The Search Space Is Part of the Model

A lot of simulation optimization discussions start too late.

Someone has a simulator. It can evaluate an ordering policy, a production plan, a staffing rule, or a network configuration under uncertainty. Then the conversation immediately jumps to algorithms:

• Bayesian optimization?
• Local search?
• Genetic algorithm?
• Reinforcement learning?
• Random search?

Those are reasonable questions.

But they are usually not the first question.

The first question is:

What decisions are you allowing the search to consider?

That sounds almost trivial. It is not.

In many real systems, the hardest part of simulation optimization is not evaluating a candidate. It is deciding what a candidate actually is, how candidates are generated, and which parts of the decision space should never be searched in the first place.

The search space is not plumbing around the model.

The search space is part of the model.

Start with the decision, not the optimizer

Suppose you manage inventory for 20,000 items across a network of fulfillment locations.

You have a simulator that can replay demand, lead times, cancellations, capacity constraints, and replenishment behavior. For any proposed policy, it can estimate:

• contribution margin,
• lost sales,
• holding cost,
• inventory turns,
• service level,
• working capital,
• expedites,
• transfers,
• and operational violations.

Great.

Now what exactly are you optimizing?

One possibility is a static vector of target inventory positions:

[ \theta = (T_1, T_2, \ldots, T_n) ]

Another is a parameterized policy:

[ x_t = \pi(S_t; \theta) ]

where:

• (S_t) is the state when the decision is made,
• (x_t) is the action,
• (\pi) is the decision rule,
• and (\theta) contains tunable parameters.

A third possibility is to use the simulator at decision time. Instead of tuning one policy offline and deploying it forever, the system generates candidate actions now, simulates their consequences, and chooses among them:

[ x_t^* = \arg\min_{x \in \mathcal{C}(S_t)} \hat{J}(x \mid S_t) ]

Here, (\mathcal{C}(S_t)) is the set of candidates generated for the current state.

That candidate set is doing enormous work.

If it is bad, the simulator can be perfect and the optimization can still be useless.

The simulator does not create decisions by itself

A simulator answers a conditional question:

If I do this, what might happen?

It does not automatically answer:

What should I try?

That gap is where many simulation projects get stuck.

A team builds a beautiful digital representation of the operation. It can reproduce inventory trajectories, queues, shortages, production delays, and service outcomes. Then someone asks it to make a decision.

The simulator has no idea what to do next.

You need a mechanism that proposes actions.

For a replenishment problem, candidate actions might be:

• order nothing,
• order to the current target,
• order to a higher target,
• order to a lower target,
• shift units from one location to another,
• delay the order one cycle,
• accelerate the order,
• use an alternate supplier,
• or choose a joint combination of those actions.

The important point is that the candidate generator encodes beliefs about what good decisions look like.

If you only simulate five variations around the current plan, you are not doing unrestricted optimization. You are doing local improvement around the current plan.

That may be exactly right.

But say what it is.

A practical example: constrained ordering

Consider a buyer who must allocate 50,000 units of available supply across 2,000 item-location combinations.

For item (i), let:

[ x_i = \text{units ordered} ]

A basic constrained decision might be:

[ \sum_i x_i \leq 50{,}000 ]

with additional restrictions such as:

[ x_i \in {0, q_i, 2q_i, \ldots} ]

for case packs (q_i), plus minimum order quantities, supplier constraints, receiving capacity, cash limits, and inventory caps.

The uncertainty includes:

• future demand,
• lead time,
• supplier fill rate,
• cancellations,
• returns,
• substitution,
• and downstream capacity.

The simulator can evaluate any feasible allocation (x).

But the number of feasible allocations is enormous.

You cannot simulate all of them.

So what do you do?

This is the actual simulation optimization problem.

Not just simulation.

Not just optimization.

Search under an expensive, noisy evaluation function.

The three jobs of a candidate generator

A useful candidate generator has three jobs.

1. Produce feasible decisions

Do not waste expensive simulation runs on actions the operation cannot execute.

If the warehouse can receive 10 trucks tomorrow, a candidate requiring 18 trucks should not survive to the simulator unless the purpose is explicitly to price the value of extra capacity.

If orders must be in case packs, generate case-pack-feasible candidates.

If a supplier has a minimum order quantity, handle it before simulation.

If two actions are mutually exclusive, encode that.

This is one reason MILP can work extremely well inside a simulation optimization system. The MILP does not have to model every uncertain consequence perfectly. It can generate high-quality feasible candidates while the simulator handles the messy dynamics.

2. Produce meaningfully different decisions

Ten candidates that differ by one unit are not ten useful candidates.

If simulation noise is larger than the economic difference between the candidates, you are spending compute to compare numerical dust.

Candidate diversity should be measured in operational terms.

For example:

• different service-risk positions,
• different cash usage,
• different supplier mixes,
• different network flows,
• different timing choices,
• or different exposure to uncertain demand.

The point is not random diversity for its own sake.

The point is to explore decisions that represent genuinely different economic bets.

3. Concentrate effort where value can change

The full decision space may be huge, but the economically interesting region is often much smaller.

Suppose ordering 0 to 100 units for an item produces large changes in shortage risk, while ordering 400 to 500 units only adds excess inventory.

Do not search both regions with equal intensity.

Use structure.

That structure might come from:

• marginal value curves,
• critical-ratio logic,
• dual values,
• historical decisions,
• deterministic optimization,
• forecast quantiles,
• policy breakpoints,
• or previous simulation results.

A good search system is not embarrassed to use domain knowledge.

It should be embarrassed to ignore it.

Candidate generation can be an optimization model

One of the most useful architectures is a two-layer system.

The first layer proposes candidates quickly.

The second layer evaluates them realistically.

For example, a MILP might generate candidate allocations by solving:

[ \max_x \sum_i \tilde{v}_i(x_i) ]

subject to operational constraints.

The value functions (\tilde{v}_i) do not need to be perfect. They may be approximations derived from forecasts, economics, or previous simulations.

Then the simulator evaluates the resulting candidate under richer uncertainty:

[ \hat{J}(x) = \frac{1}{N}\sum_{s=1}^{N} C(x, \omega_s) ]

where (\omega_s) is scenario (s).

Now vary the candidate-generation assumptions:

• different risk penalties,
• different service targets,
• different scenario weights,
• different inventory caps,
• different shadow prices,
• different budget levels.

Each solve produces a structurally coherent candidate.

This is often much better than randomly perturbing thousands of decision variables and hoping something useful appears.

Search over actions or search over policies?

This distinction matters.

Search over actions

You are making a decision now.

Examples:

• how much to order today,
• which loads to accept,
• where to send constrained supply,
• which jobs to schedule next.

The simulator is used at decision time.

The state is current and specific:

[ S_t = (I_t, O_t, D_t, C_t, \ldots) ]

You generate candidate actions for this state, simulate them, and select one.

This can be powerful when decisions are high-value and simulation is fast enough.

Search over policy parameters

You are tuning a rule that will make many future decisions.

Examples:

• reorder thresholds,
• safety-stock multipliers,
• escalation rules,
• allocation weights,
• or dispatch priorities.

Now the candidate is a parameter vector (\theta), and the simulator evaluates the policy over many states and time periods.

The goal is:

[ \theta^* = \arg\min_{\theta \in \Theta} \mathbb{E}[C(\pi(\cdot;\theta), \omega)] ]

These are not the same problem.

Do not accidentally build an offline policy-tuning system when the business needs a decision now.

And do not run an expensive online simulation search every morning if a stable policy can produce nearly the same value in milliseconds.

Questions to ask before choosing the search algorithm

Before debating Bayesian optimization versus local search, answer these questions:

• What exactly is one candidate?
• Is the candidate an action, a plan, a policy, or a parameter vector?
• How many dimensions does it have?
• Which variables are discrete?
• Which decisions are coupled?
• What makes a candidate infeasible?
• Can feasibility be repaired cheaply?
• How expensive is one simulation evaluation?
• How noisy is the estimated objective?
• Can common random numbers be used?
• Can candidates be evaluated in parallel?
• How often must a decision be made?
• How different is tomorrow’s state from today’s?
• Can previous search results be reused?
• What is the cost of returning a merely good answer instead of the best observed answer?

These questions determine the architecture.

The fashionable algorithm comes later.

Uncertainty changes the search problem

Simulation objectives are often noisy.

Suppose candidate A has estimated profit of $10.2 million and candidate B has estimated profit of $10.1 million.

That does not mean A is better.

If each estimate has a standard error of $300,000, the ranking is mostly noise.

This creates a second allocation problem:

How much simulation effort should each candidate receive?

A naive system gives every candidate 10,000 replications.

That is often wasteful.

A better system might:

1. evaluate many candidates cheaply,
2. eliminate obviously bad candidates,
3. allocate more replications to close contenders,
4. stop when additional precision is unlikely to change the decision.

Use common random numbers when comparing candidates. If candidate A and candidate B are evaluated under the same demand, lead-time, and disruption scenarios, much of the environmental noise cancels in the comparison.

The metric that matters is often not the absolute estimate:

[ \hat{J}(A) ]

but the paired difference:

[ \hat{\Delta}_{A,B} = \frac{1}{N}\sum_s \left[C(A,\omega_s)-C(B,\omega_s)\right] ]

That is a much more useful object for decision making.

Constraints belong in the search, not in a cleanup script

A common failure mode looks like this:

1. generate an attractive candidate,
2. simulate it,
3. discover it violates capacity,
4. patch it with a repair heuristic,
5. simulate the repaired version,
6. report the result as if the optimizer chose it.

Now the actual decision rule is the optimizer plus the repair heuristic.

The repair logic may be doing more economic work than the optimizer.

If repairs are unavoidable, treat them as part of the policy and evaluate them explicitly.

Better yet, generate feasible candidates when possible.

For supply chain systems, this often means separating constraints into three groups:

Hard constraints should never be violated: physical capacity, case packs, legal restrictions, material balance.

Soft constraints may be violated at a price: overtime, expedites, service misses, preferred supplier share.

Uncertain constraints depend on future outcomes: receiving congestion, storage overflow, service attainment.

Do not flatten all three into one giant penalty function unless you have a good reason.

Metrics for the search system itself

Teams often measure only the final simulated objective.

That misses whether the search process is healthy.

Track:

• best objective found versus simulation budget,
• time to first feasible candidate,
• candidate rejection rate,
• duplicate candidate rate,
• fraction of compute spent on candidates that were obviously dominated,
• improvement over the current production decision,
• improvement over a simple baseline policy,
• ranking stability as replications increase,
• decision stability across forecast updates,
• and wall-clock time to a deployable answer.

For an online decision system, also track regret from latency.

A solution that is theoretically better but arrives after the ordering cutoff has infinite practical stupidity.

Common failure modes

Searching the raw variable space

A 20,000-dimensional vector of order quantities is not a friendly search space.

Random perturbation will mostly create nonsense.

Search a lower-dimensional policy, a structured neighborhood, or candidates generated by an optimization model.

Making the candidate generator too conservative

If every candidate is a tiny variation of the current plan, the search cannot discover a materially different strategy.

Local search is useful only if the local region contains good decisions.

Making the candidate generator too creative

The opposite failure is generating bizarre decisions because they are mathematically different.

Diversity without operational plausibility burns simulation budget.

Comparing candidates on different random worlds

Candidate A gets easy demand scenarios. Candidate B gets hard ones. The system declares A better.

Use paired scenarios whenever possible.

Optimizing simulator noise

The search keeps revisiting candidates that got lucky early.

Use replication, uncertainty estimates, and reevaluation of incumbents.

Ignoring decision latency

A search that takes six hours for a decision needed in 20 minutes is not an optimization system.

It is a research project.

Tuning the policy on one historical replay

A policy can overfit a simulation environment just like a machine-learning model can overfit a dataset.

Use multiple periods, regimes, and stress scenarios. Keep final evaluation scenarios out of the tuning loop.

A practical architecture

For many real supply chain problems, I would start with something like this:

Step 1: define the state

Capture only information available when the decision is made:

• on-hand inventory,
• open orders,
• forecast distribution,
• lead-time state,
• capacity,
• cash or supply limits,
• and relevant operational commitments.

Step 2: define the action

Be explicit about what can change now.

Do not call forecast values, inventory balances, or future outcomes decision variables.

Step 3: build a fast candidate generator

Use a deterministic model, marginal values, policy rules, or structured neighborhoods to produce feasible candidates.

Generate tens or hundreds of useful candidates, not millions of random ones.

Step 4: evaluate with common scenarios

Run candidates through the same uncertain futures.

Measure economic outcomes, operational failures, and risk.

Step 5: adapt the simulation budget

Kill bad candidates early. Spend compute on close decisions.

Step 6: stress the finalists

Evaluate the best few candidates under tail scenarios, regime shifts, and model misspecification.

Step 7: return a decision with context

Do not return only:

Candidate 47 wins.

Return:

• the recommended action,
• expected economic improvement,
• downside exposure,
• binding constraints,
• closest alternative,
• and the conditions under which the recommendation would change.

That is much more useful to an operator.

What to do in practice

If you already have a simulator and want it to help make decisions, do not start by shopping for a search library.

Take one real decision and write down:

1. the state available now,
2. the action you can actually take,
3. the constraints that define feasibility,
4. the uncertain events that occur after the action,
5. the economic metric that determines whether the action was good,
6. and the mechanism that will propose alternatives.

Then build the smallest candidate generator that can produce meaningfully different, feasible actions.

A simple MILP that generates 20 strong candidates can beat a sophisticated black-box optimizer wandering through a terrible search space.

A hand-built neighborhood can beat a generic metaheuristic if it reflects how the operation can actually change.

A parameterized policy can beat online search if the decision must be made in milliseconds.

And online simulation can beat a fixed policy when the current state is unusual, the decision is valuable, and the simulator is fast enough to matter.

The search algorithm matters.

But it only searches the world you give it.

Design that world carefully.