What Happens When AI Starts Making Decisions From Your Business Data?

A pricing decision can be correct when it is made - and still cost the business weeks later.

In one real case, a pricing specialist importing a promotion into the ERP forgot to set one field: the end date. The discounted price stayed active after the promotion should have ended.

Nobody approved the extension. Nobody deliberately decided to keep the discount running. The system simply continued doing what it had been configured to do.

Margins on those product lines fell below plan for weeks. The problem became visible only when the warehouse ran out of stock.

This wasn't a story about a careless employee. It was a problem with a decision that kept executing after the moment anyone was actively checking whether it still made sense.

AI-driven decisions can create the same problem - only at a much greater speed and scale.

Why can a decision keep running after it stops making sense?

A person reviews a decision when it is made. In this case, the pricing specialist checked the promotional price before importing it, but missed the field that determined when the promotion should end.

After that, the system had no reason to question the decision. The price was valid, the product was valid and the promotion was valid. Nothing in the process required someone to check whether the original conditions still applied.

AI-driven decisions can widen the same gap.

A model can generate and apply decisions continuously across thousands of cases. There may be no manual step where someone reviews each decision, and no obvious moment when the original assumption gets questioned.

The speed of automation removes the pause that sometimes allows a human to notice that the context has changed.

Where does the first real signal appear?

Usually somewhere downstream.

In the pricing example, each individual sale still looked valid. The discount existed in the system. The order was processed correctly. Nothing had technically failed.

The signal appeared in inventory: products were moving faster than expected while margins were below plan.

By then, the cause was weeks old.

This is also why model performance needs to be watched over time. A peer-reviewed Scientific Reports study tested 128 model-dataset combinations across four domains and found temporal performance degradation in 91% of the cases. The researchers showed that models can deteriorate over time even when the underlying data does not contain an obvious abrupt change.

That creates a particular business risk: the system can continue producing outputs while the relationship between those outputs and current reality gradually changes.

What does keeping a person in the loop actually mean?

It doesn't mean asking someone to manually approve every decision.

That would remove much of the value of automation.

The more useful approach is to identify the conditions that deserve an automatic check.

In the pricing example, a promotion without an end date is a clear condition. The system can flag it when the promotion is imported, rather than waiting for an inventory problem weeks later.

The same principle applies to AI-driven decisions.

A company can monitor a sample of decisions, track the assumptions behind them alongside their outcomes, and define conditions that trigger a review.

For example:

• a discount has no expiry date;
• a forecast relies on data that has become too old;
• a model's error rate moves beyond an agreed threshold;
• a decision repeatedly produces an outcome outside the expected range.

The objective is simple: move the check closer to the decision instead of waiting for the business consequence.

What changes when AI makes thousands of decisions?

The risk isn't that AI suddenly makes every decision wrong.

The bigger change is the number of decisions being made and the speed at which they are applied.

A person making ten decisions a day creates ten opportunities for review.

A system making thousands creates a very different monitoring problem.

That means the controls around an AI-driven process become part of the decision itself. The business needs to know what the system is allowed to decide, which conditions should trigger a review, what evidence supports the decision, and which business outcomes should be monitored afterwards.

Without those checks, automation can make an outdated assumption very efficient.

So: what happens when AI starts making decisions from your business data?

The decisions keep moving - even when the conditions behind them have changed.

The pricing example shows that this problem exists without AI: one missing field was enough to keep a valid instruction running long after it should have stopped.

AI doesn't create that failure mode. It increases the number of decisions affected and reduces the time available to notice a problem.

That changes what businesses need to monitor.

Before handing the next business decision over to an AI system, decide in advance:

What should the system check automatically, what should trigger human review, and which business outcome would tell us that the decision is no longer working as expected?

Because the most expensive AI mistake may not be a wrong decision.

It may be a decision that remains correct according to the system - long after it has stopped being correct for the business.