The Value of Information Depends on the Decision

More information is not automatically valuable.

A supplier can send daily inventory feeds instead of weekly. A forecasting team can build another demand signal. A retailer can share sell-through data. A data engineering team can reduce a pipeline from twelve hours to fifteen minutes. All of these sound like obvious improvements.

But information has economic value only when it can change a decision, and the changed decision produces a better outcome.

That gives us a much more useful question than “Would this data be nice to have?”

What decision would change if we had it?

If nobody can answer that, the data project is already on shaky ground.

Start with the decision timeline

Suppose a retailer places a purchase order every Monday at 10:00 AM. The supplier lead time is eight weeks and the order cannot be cancelled after Tuesday.

A new demand signal arrives every Wednesday.

It may be extremely predictive. It is also useless for Monday’s committed order unless there is some recourse after Wednesday.

Timing is part of information quality.

Define the state at decision time as $S_t$, the information available as $I_t$, and the action as

$$ x_t = \pi(S_t, I_t). $$

For replenishment, $x_t$ might be an order quantity. For allocation, it might be units assigned to locations. For production, it might be capacity reserved by product. The policy $\pi$ maps what you know into what you do.

Now suppose a new signal $Z_t$ becomes available. Its value comes from comparing

$$ \pi(S_t, I_t) $$

with

$$ \pi(S_t, I_t, Z_t). $$

If the action is identical in almost every relevant state, the signal may be statistically interesting and economically worthless.

A simple supply chain example

Assume you import a product with an MOQ of 1,000 units. Current inventory plus open orders cover roughly the next ten weeks. Your policy recommends either 0, 1,000, 2,000, or 3,000 units.

A new forecasting model improves the expected eight-week demand estimate from 1,430 to 1,510 units.

That can be a meaningful statistical improvement. But if both forecasts produce the same 2,000-unit order, the immediate value of the improved information is zero for this decision.

Now imagine the new signal moves expected demand from 980 to 1,180 units and changes the decision from zero to 1,000. Suddenly the information matters a lot.

Whether that change is good still depends on what happens next. If demand really is high, the signal may prevent a stockout. If the signal is noisy, it may trigger a thousand units of unwanted inventory.

The value is not the size of the forecast change. It is the expected economic improvement from the action change.

Decision boundaries matter

Many real decisions have breakpoints.

Orders come in cases. Suppliers impose MOQs. Trucks have fixed capacities. Production lines require setup. Warehouses have thresholds. Budgets bind. Contracts have tiers.

This means the value of information is often highly nonlinear.

A tiny information update near a breakpoint can be valuable because it changes the action. A much larger update far from a breakpoint may do nothing.

This is why averaging information quality across thousands of observations can hide the places where information actually matters.

For a candidate signal, I like to measure a decision-change rate:

$$ R = \frac{#{t : x_t^{new} \neq x_t^{base}}}{#{t}}. $$

Then measure the economic value only for those changed decisions.

That immediately separates “the model predicts differently” from “the system acts differently.”

Information can have negative value in implementation

In textbook decision theory, correctly used free information should not make an optimal decision-maker worse off. You can always ignore it.

Production systems are not textbook decision-makers.

A noisy signal can cause planners to overreact. A forecast refreshed every hour can make recommendations oscillate. A new supplier feed can be stale without anyone realizing it. A model can treat a weak signal as precise. An optimizer can react aggressively because transaction costs or decision stability were omitted.

So in practice, adding information can make the implemented policy worse.

This is not an argument against information. It is an argument for evaluating the entire information-to-action pipeline.

Model the uncertainty the information actually resolves

Suppose demand is uncertain, but supplier lead time is even more uncertain.

Your team proposes spending $500,000 improving demand forecasting. Before doing that, simulate what would happen with perfect demand information while leaving lead-time uncertainty unchanged.

If performance barely improves, you have learned something important: demand uncertainty is not currently the limiting uncertainty.

Maybe the valuable project is supplier visibility instead.

The same logic applies to:

• demand,
• lead time,
• supplier yield,
• cancellations,
• returns,
• transportation delays,
• price response,
• capacity availability.

Do not ask which variable is hardest to predict. Ask which uncertainty is worth resolving for the decision.

Perfect information is a useful upper bound

A practical experiment is to give the decision system information it could never actually have and see how much performance improves.

For example, run three simulations:

1. the current policy with today’s information,
2. the same policy with the proposed improved signal,
3. an idealized version with perfect future knowledge of the uncertain quantity.

Suppose annual simulated profit is:

• current information: $42.0M,
• improved signal: $42.3M,
• perfect demand information: $42.5M.

The proposed signal captures $300K of a theoretical $500K opportunity. That is useful context.

Now suppose perfect demand information only produces $42.05M. There is little reason to fund a multi-million-dollar demand forecasting program for this decision. Even magic cannot create enough value.

Perfect-information experiments are especially useful for killing attractive but low-value projects early.

Information value depends on recourse

Consider two businesses facing the same demand uncertainty.

Business A orders overseas with a sixteen-week lead time and cannot cancel.

Business B replenishes locally every two days.

A better long-range demand signal can be enormously valuable to A and nearly irrelevant to B. Business B can wait and react.

The ability to take future actions reduces the value of knowing everything now.

This is why you need to model recourse honestly:

• Can you reorder next week?
• Can you expedite?
• Can you transfer inventory?
• Can you substitute products?
• Can you cancel an order?
• Can you change production mix?
• What does each recourse action cost?

A simulation that freezes all future decisions will usually exaggerate the value of early information.

Shared constraints change which information matters

Information value is also affected by coupling.

Suppose ten SKUs share one supplier capacity constraint. Better demand information for SKU A may not change A’s unconstrained order recommendation much, but it may reveal that scarce capacity should be moved from SKU B to SKU A.

The value appears through allocation, not through the standalone forecast.

The same happens with shared:

• vendor MOQs,
• containers,
• production capacity,
• warehouse space,
• working capital,
• transportation lanes.

Evaluate information inside the real coupled decision whenever those interactions matter.

How I would evaluate a proposed data source

Build the experiment around the production decision, not around a prediction leaderboard.

For each historical decision state or simulated state:

1. reconstruct what was known at the decision timestamp,
2. run the baseline policy,
3. add the candidate information,
4. rerun the exact same policy,
5. record whether the action changed,
6. simulate both actions against the same future scenarios,
7. calculate the economic difference.

Using common scenarios matters because it reduces simulation noise. You want the comparison to reflect the information, not different random draws.

For scenario $s$, calculate

$$ \Delta_s = V(x^{new}, s) - V(x^{base}, s), $$

where $V$ includes the economics you actually care about: margin, holding, shortage, freight, markdowns, working capital, or other relevant costs.

Then inspect the distribution of $\Delta_s$, not only its mean.

Metrics worth reporting

A good information-value report should include both decision and business metrics:

• decision-change rate,
• magnitude of changed actions,
• expected economic improvement,
• downside percentiles,
• stockouts or lost sales,
• inventory and inventory age,
• expedites and transfers,
• capacity utilization,
• working capital,
• decision stability,
• value captured relative to perfect information.

Also segment the results. Information may be valuable only for long-lead-time items, constrained vendors, launches, or high-margin products. That can justify a targeted deployment rather than forcing the signal into every decision.

Failure modes

Buying data before naming the decision

A company licenses a new dataset because it contains “valuable signals.” Nobody can identify which production action will consume it.

Do instead: write down the decision, decision owner, timing, and expected action change before purchasing the data.

Measuring prediction lift instead of decision lift

A signal improves forecast accuracy by 8%, so the project is declared successful.

Do instead: run the downstream policy and measure changed actions and economics.

Ignoring latency

The information arrives after the commitment point.

Do instead: draw the decision timeline explicitly. Record when information becomes available and when actions become irreversible.

Assuming perfect information must be enormously valuable

Teams often assume uncertainty itself is the main problem.

Do instead: run the perfect-information upper-bound experiment. Sometimes constraints, costs, or limited recourse dominate.

Evaluating one SKU at a time when constraints are shared

The information appears worthless in isolated tests even though it could improve capacity allocation across products.

Do instead: preserve the important coupling in the evaluation model.

Ignoring the cost of reacting

A more frequent signal causes constant replanning, purchase-order changes, nervous vendors, and operational churn.

Do instead: include transaction costs, frozen windows, or stability penalties where they reflect real economics.

Implementation notes

The hardest part is usually point-in-time data.

To evaluate information honestly, save the state that existed when each decision was made. That means forecast vintage, inventory, open orders, supplier commitments, costs, capacity, lead-time beliefs, and relevant business rules.

Do not backtest a January decision using a demand plan revised in March.

In production, log at least:

• decision timestamp,
• state snapshot ID,
• information source versions,
• forecast version,
• policy or optimization model version,
• recommended action,
• human override,
• eventual outcome.

That creates an audit trail from information to action to result.

It also lets you answer the question that matters when someone proposes another data source six months later: What did the last one actually change?

What to do in practice

When someone proposes better data, a better forecast, or a faster signal, do not start with the model.

Start with the decision.

Identify when the decision is made and when it becomes committed. Identify the uncertainty that drives it. Identify the recourse available afterward. Run the current policy with and without the proposed information. Preserve the real constraints. Measure how often the action changes. Simulate those changed actions under common scenarios. Put a dollar value on the difference.

Then run the perfect-information experiment to understand the ceiling.

You may discover that a modest signal is worth millions because it moves decisions near expensive breakpoints.

You may discover that an impressive forecasting improvement is worth almost nothing because the MOQ, capacity constraint, or ability to reorder next week absorbs it.

Both are good results because both tell you where to invest.

The point of information is not to know more.

The point is to make a better decision.