Policy Parameters Are Products

A surprising amount of decision science ends with a number somebody typed into a config file.

A 95% service target. A 12-week coverage horizon. A $7 stockout penalty. A 0.03 improvement threshold. A 14-day freeze window. A 20% utilization buffer. A rule that says an order must improve expected value by at least $500 before we bother changing the plan.

The optimization model can be sophisticated. The simulator can run thousands of scenarios. The forecast can be probabilistic. The solver can prove a tiny MIP gap.

And then the production policy is largely determined by six constants nobody can explain.

That is not a minor implementation detail. Those constants are part of the decision system.

If changing a number changes what the business does, that number is a product decision and an economic decision. Treat it accordingly.

Start with the decision, not the parameter

Suppose we replenish a portfolio of products from an overseas supplier. Every week we observe inventory, open purchase orders, demand information, supplier availability, and capacity. The system decides how much to order.

A simple policy might be written as

[ q_t = \max(0, S_\theta(s_t) - IP_t), ]

where:

• (IP_t) is inventory position,
• (s_t) is the current state,
• (S_\theta) is a target generated by policy parameters (\theta),
• (q_t) is the order quantity.

The parameter vector might contain

[ \theta = (h, \alpha, c_s, c_h, \epsilon), ]

with a coverage horizon (h), service parameter (\alpha), shortage cost (c_s), holding cost (c_h), and action threshold (\epsilon).

It is tempting to call these “settings.” But change (h) from 8 weeks to 16 and you may double inventory exposure. Change (c_s) and scarce capacity shifts toward different SKUs. Change (\epsilon) and the system may stop making small orders. Change the freeze window and planners may lose the ability to react to new information.

The first question is therefore not:

What should this parameter be?

It is:

What business behavior does this parameter control?

That framing prevents a lot of fake precision.

Build a parameter map

Before tuning anything, write down every parameter that can materially change the decision.

For each one, document five things:

Parameter	What it controls	Economic interpretation	Who owns it	How it is validated
Coverage horizon	How far inventory protects demand	Cost of committing early versus reacting later	Supply chain	Replay + simulation
Shortage penalty	Relative value of avoiding lost demand	Marginal underage cost	Business / finance	Economics + sensitivity
Holding cost	Cost of carrying inventory through time	Capital, storage, obsolescence	Finance / supply chain	Cost model
Change threshold	Whether small improvements trigger action	Cost of operational churn	Operations	Replay + experiment
Freeze window	How late a plan may change	Value of stability versus flexibility	Operations	Process analysis
Risk parameter	Tail-risk preference	Cost of bad outcomes beyond expected value	Business owner	Scenario analysis

This sounds basic. Do it anyway.

I have seen systems where nobody could answer whether a threshold represented economics, a technical safeguard, an old planner preference, or a value copied from a prototype three years earlier. Those are four very different things.

Separate economic parameters from numerical parameters

Not every parameter belongs in the same bucket.

A stockout cost changes the economic objective. A MIP feasibility tolerance changes how the solver interprets numerical feasibility. A simulation sample count changes estimation precision. A business approval threshold changes whether an otherwise attractive action is executed.

Mixing these together creates bad governance.

A useful classification is:

1. Economic parameters — margins, holding costs, shortage costs, expedite costs, disposal costs, capacity opportunity costs.
2. Policy parameters — target horizons, reorder thresholds, risk aversion, freeze windows, action thresholds.
3. Modeling parameters — scenario count, horizon truncation, approximation granularity, surrogate structure.
4. Solver parameters — time limit, MIP gap, cuts, heuristics, threads, tolerances.
5. Operational parameters — approval limits, minimum materiality, execution cadence, exception rules.

The ownership should differ because the meaning differs.

An OR scientist should not quietly decide that a lost sale costs $40 because it makes the simulation behave nicely. A finance leader should not choose a solver feasibility tolerance. A planner should not be expected to understand why a branch-and-bound parameter changed runtime.

Good systems make those boundaries explicit.

Some parameters should disappear

A parameter is not automatically good just because it is tunable.

If the economics can be modeled directly, do that before inventing a knob.

Suppose the business says:

We need a 95% service level.

Ask why.

Maybe the real concern is losing $60 of contribution margin when demand is missed while carrying an extra unit for a week costs $0.20. If those economics are credible, the optimizer can trade overage and underage directly. A service target may be an unnecessary proxy.

Likewise, “keep 10 weeks of supply” may simply be an old way of protecting against uncertain demand and lead time. If the decision system already represents those uncertainties and their economic consequences, a fixed weeks-of-supply target can fight the model.

Every extra policy parameter creates another dimension to calibrate, monitor, explain, and eventually debug.

Use parameters where they represent a real policy choice. Do not use them to hide economics you were too lazy to model.

When parameters are necessary, tune the policy

Many real systems cannot be reduced to one clean optimization problem. The decision may be generated by a parameterized policy and evaluated through simulation.

Write the problem that way.

Let (\pi_\theta) be the policy with parameters (\theta). Under scenario (\omega), the policy produces a sequence of decisions and a total contribution

[ J(\theta, \omega). ]

The tuning problem is

[ \max_{\theta \in \Theta} ; \mathbb{E}[J(\theta, \omega)]. ]

If downside risk matters, use something closer to

[ \max_{\theta \in \Theta} \left( \mathbb{E}[J(\theta,\omega)]

• \lambda \cdot \text{Risk}(J(\theta,\omega)) \right). ]

The important point is that you are optimizing a policy, not searching for magic constants independently.

The parameters interact.

A longer coverage horizon may look bad with one shortage penalty and good with another. A tighter action threshold may reduce churn but become dangerous when lead-time uncertainty increases. A freeze window that works for domestic suppliers may be terrible for an eight-month lead-time vendor.

One-at-a-time tuning misses those interactions.

Tune against the state distribution you will actually see

A common failure mode is tuning on a clean synthetic world and deploying into a dirty operational one.

Production states include:

• late purchase orders,
• stale forecasts,
• supplier outages,
• inventory corrections,
• demand spikes,
• MOQ activation,
• capacity shortages,
• missing data,
• planner overrides,
• items entering or leaving the assortment.

If the policy is only tuned around normal steady-state inventory, it is not really tuned for production.

Your training or simulation scenarios should deliberately include the ugly states that create expensive decisions.

That does not mean giving every pathological event equal probability. It means making sure the policy has been exercised where it can break.

Do not tune and grade on the same randomness

Simulation optimization is especially easy to fool yourself with.

Suppose policy A and policy B differ by $8,000 of simulated annual profit. That sounds meaningful until you discover the standard deviation of the difference is $25,000.

Worse, if you evaluate hundreds of parameter combinations on the same noisy objective and select the best observed result, you will select partly on noise.

The workflow should separate search from confirmation.

Use a moderate scenario set to explore the parameter space. Then take the finalists and evaluate them on fresh scenarios with substantially more replications.

When comparing nearby policies, common random numbers are useful: evaluate competing parameter sets on the same underlying demand, lead-time, and outage paths. The variance of the difference is what matters.

If

[ \Delta(\omega)=J(\theta_A,\omega)-J(\theta_B,\omega), ]

estimate the mean and uncertainty of (\Delta), not just two independent means.

A policy should not win production because it got lucky in Monte Carlo.

Tune for the actual operating regime

A single global parameter vector is attractive because it is simple.

It can also be wrong.

Consider three products:

• a stable high-volume item with a four-week lead time,
• a slow item with an eight-month lead time,
• a seasonal item with high obsolescence risk.

Why should they have the same coverage parameter, risk preference, or action threshold?

At the other extreme, giving every SKU its own 20-dimensional parameter vector is usually a disaster. You create thousands of weakly identified knobs and overfit historical noise.

The practical middle ground is often structured heterogeneity.

Parameters may vary by meaningful class:

[ \theta_i = g(z_i; \beta), ]

where (z_i) contains stable product or supplier characteristics and (g) maps them into policy parameters.

Examples include:

• lead-time class,
• margin class,
• demand-volume class,
• perishability,
• supplier reliability,
• lifecycle stage,
• MOQ structure.

Now you have a policy that adapts without becoming an ungovernable table of SKU-specific constants.

Constraints belong in the tuning problem too

The best simulated parameter set may produce behavior the business cannot execute.

A policy might maximize expected profit while causing:

• 40,000 order changes per week,
• impossible supplier volatility,
• excessive cash usage,
• warehouse congestion,
• unstable allocations,
• too many planner exceptions.

Those are not post-processing annoyances. They are constraints or costs of the policy.

You can impose constraints such as

[ \mathbb{E}[\text{WorkingCapital}(\theta)] \le B, ]

[ P(\text{Service}(\theta) < s_{min}) \le \alpha, ]

or

[ \mathbb{E}[\text{DecisionChanges}(\theta)] \le C. ]

Or, when appropriate, price the behavior directly in the objective.

The important part is to stop pretending the simulator’s headline profit metric is the whole business.

Metrics should describe behavior, not just value

For each candidate policy, I want at least three classes of metrics.

Economic metrics

• expected contribution,
• shortage cost,
• holding cost,
• expedite cost,
• disposal or markdown cost,
• working capital,
• tail loss.

Operational metrics

• order frequency,
• average order size,
• number of changed decisions,
• planner exception rate,
• capacity utilization,
• MOQ activation frequency,
• emergency actions.

Statistical metrics

• standard error of policy differences,
• scenario-level win rate,
• downside quantiles,
• sensitivity to scenario seed,
• sensitivity to parameter perturbation.

A policy that earns 0.1% more simulated profit but changes 30% of orders when one parameter moves from 1.00 to 1.01 is telling you something important.

The expected value is not the only diagnostic.

Parameter sensitivity is a production test

Once you find a good parameter vector, perturb it.

For parameter (\theta_j), evaluate

[ \theta_j(1-\delta), \quad \theta_j, \quad \theta_j(1+\delta) ]

for reasonable (\delta).

Then inspect both objective value and decisions.

There are several possible outcomes.

Flat economics, stable decisions: good. The exact parameter value probably does not matter much.

Flat economics, unstable decisions: dangerous. You may have many economically equivalent solutions causing operational churn.

Sharp economics, stable decisions: investigate. The parameter may control a small number of very expensive cases.

Sharp economics, unstable decisions: this parameter deserves serious monitoring and governance.

This is much more useful than reporting “the optimizer selected 0.8734.”

Four decimals do not imply four decimals of knowledge.

Put parameter versions in the production logs

If a production decision cannot be reproduced, debugging becomes archaeology.

Every decision run should log at least:

run_id
model_version
policy_version
parameter_set_id
parameter_values
forecast_vintage
state_snapshot_time
scenario_generator_version
solver_configuration
execution_time
recommended_action
executed_action

Do not overwrite a config value in place and hope nobody asks what happened three months later.

Version the parameter set.

If an order suddenly jumps from 2,000 to 11,000 units, you should be able to answer whether the state changed, the forecast changed, the code changed, or the policy parameters changed.

Without that lineage, every incident turns into a meeting full of guesses.

Changes need a promotion process

A parameter change can be as consequential as a code change.

Treat material changes accordingly.

A reasonable promotion path is:

1. Economic review — does the parameter still represent the intended business tradeoff?
2. Historical replay — how would the new policy have behaved on known states?
3. Simulation evaluation — how does it perform under uncertainty and stress scenarios?
4. Decision diff — which actual decisions change, by how much, and why?
5. Operational review — can the changed behavior be executed?
6. Limited rollout — expose a controlled portion of the decision population.
7. Monitoring — compare realized behavior with the expected distribution.

The decision diff is especially important.

Do not tell an operator, “Expected simulated value improved 1.8%.”

Show them:

• 82% of decisions are unchanged,
• 11% increase modestly,
• 5% decrease,
• 2% change materially,
• the material changes are concentrated in long-lead-time items with high shortage economics.

Now somebody can review the policy like a business system instead of a black box.

Failure modes I would expect in a real implementation

1. The parameter has no owner

Everybody uses it. Nobody is accountable for its meaning.

Fix: assign an owner based on what the parameter represents, not who happened to write the code.

2. The value came from a prototype

A scientist picked something reasonable to get the first demo working. Three years later it is still there.

Fix: maintain a parameter register with source, rationale, owner, validation method, and last review date.

3. Parameters compensate for model errors

A shortage penalty is inflated because lead-time uncertainty is missing. A coverage horizon is extended because the forecast underestimates promotions.

Fix: distinguish real policy preferences from compensating hacks. Fix the underlying model when possible.

4. The policy is over-tuned

Hundreds of knobs are fitted to historical outcomes until the backtest looks amazing.

Fix: reduce dimensionality, use structured parameters, regularize complexity, and confirm on fresh scenarios or time periods.

5. Tuning ignores operational cost

The simulation optimizes inventory economics but creates constant plan churn.

Fix: measure and price decision instability or constrain it explicitly.

6. Nobody knows when a parameter changed

A dashboard looks strange and the investigation starts with, “Did somebody change something?”

Fix: immutable parameter versions tied to every production run.

7. A single value is treated as universal truth

One global threshold is applied across radically different suppliers and products.

Fix: test whether economically meaningful segmentation improves decisions without creating unmanageable complexity.

What I would do in practice

If I inherited a production optimization system tomorrow, I would not start by tuning the solver.

I would inventory the knobs.

I would search the codebase, configuration service, notebooks, dashboards, and orchestration jobs for every number that can change a decision. I would classify each number as economic, policy, modeling, solver, or operational. I would ask who owns it, where it came from, what behavior it controls, and when it was last validated.

Then I would rank the parameters by decision sensitivity.

Which parameters actually move orders, allocations, schedules, or recommendations? Which ones barely matter? Which ones affect a handful of extremely expensive cases? Which ones create operational instability?

I would focus validation effort there.

For policy parameters, I would build a replay and simulation harness that can evaluate candidate parameter sets on identical states and random paths. I would separate search from confirmation. I would log decision diffs, not just objective values. I would stress the ugly states: outages, capacity shortages, stale data, long lead times, MOQ activation, and extreme demand.

Finally, I would version the resulting policy as a first-class production artifact.

Because that is what it is.

The bigger point

Decision science is often presented as a progression from data to forecast to optimization to decision.

Production systems are messier. The final behavior is usually a composition of forecasts, optimization, heuristics, thresholds, economics, approval rules, and configuration.

That composition is the policy.

A beautiful mathematical model does not rescue a bad policy parameter. A calibrated simulator does not rescue a threshold nobody understands. A proven optimal solution does not help if an arbitrary config value points the model at the wrong economic tradeoff.

If a number changes the decision, treat the number like it matters.

Give it an interpretation. Give it an owner. Test it under uncertainty. Measure its effect on actual decisions. Version it. Monitor it.

Policy parameters are not decoration around the model.

They are part of the product.