Profile Before You Optimize

The Solver Is an Easy Suspect

A planning system takes 40 minutes to run.

The first reaction is usually predictable:

• tune the solver,
• change the MIP gap,
• add more threads,
• buy a bigger machine,
• replace the algorithm.

Sometimes that is correct. Often it is not.

A real decision pipeline does much more than call an optimizer. It reads data, cleans it, builds business structures, generates scenarios, creates model objects, solves, simulates, aggregates results, and writes outputs. If the solver takes six minutes and preprocessing takes twenty-four, a week of solver tuning is mostly theater.

This sounds obvious. In practice, teams routinely optimize the component they understand best instead of the component consuming the time.

The rule is simple:

Do not optimize runtime until you know where the runtime goes.

That means profiling the full path from raw inputs to final decision, not staring at the solver log and guessing.

Frame the Performance Problem Correctly

Before touching code, define what is actually slow.

“The model is slow” is not a useful statement.

Ask:

• Is the delay in data loading?
• Is a large business object being rebuilt every run?
• Is scenario generation expensive?
• Is model construction slow?
• Is the mathematical solve slow?
• Is the simulator slow per replication?
• Are too many candidates being evaluated?
• Is serialization or output writing the bottleneck?
• Is the system slow only for certain products, locations, or horizons?
• Did runtime increase because the problem grew, or because the architecture changed?

The performance problem should be written as a measurable statement.

For example:

The daily replenishment run takes 52 minutes at the 95th percentile. Profiling shows 29 minutes are spent rebuilding the MOQ eligibility structure, 11 minutes generating scenarios, 8 minutes solving, and 4 minutes elsewhere.

Now there is something to work on.

Without that decomposition, “make it faster” is just an invitation to random engineering.

The Decision Pipeline Is the Unit of Analysis

A useful mental model is:

raw data
   ↓
validated state
   ↓
static business structures
   ↓
uncertainty model / scenarios
   ↓
candidate decisions or optimization model
   ↓
solve / search
   ↓
simulation or policy evaluation
   ↓
selection
   ↓
operational output

Each arrow costs time. Each box may contain work that can be reused, cached, parallelized, approximated, or removed.

The mistake is treating the solver call as if it were the whole system.

In supply chain applications, some of the most expensive work happens before the first decision variable exists. A system may build product-location networks, sourcing eligibility, MOQ hierarchies, calendars, substitution graphs, bill-of-material structures, or pack-size trees. Those objects can be large and expensive to construct.

If they are rebuilt inside every simulation replication or candidate evaluation, the architecture may dominate the algorithm.

A Concrete Example: The MOQ Tree

Suppose an inventory simulator evaluates candidate ordering policies.

Each candidate is tested across many stochastic demand paths. Before simulating inventory, the code builds an MOQ structure that represents rules such as:

• item minimums,
• supplier minimums,
• order-level minimums,
• case packs,
• nested product groups,
• exceptions by location.

The team assumes the simulation is slow because it runs thousands of replications.

Profiling shows something else:

Total runtime:                 100%
Build MOQ structure:            57%
Simulate inventory transitions: 21%
Generate demand paths:          10%
Score outcomes:                  6%
Other:                           6%

The obvious optimization target is not the simulation math. It is the structure being rebuilt around it.

But “cache the tree” is not automatically safe.

You need to know whether the object is truly static.

Ask:

• Does the tree depend only on master data?
• Does it depend on the candidate policy?
• Does it depend on the sampled demand path?
• Does downstream code mutate the object?
• Are references shared across replications?
• Can a cached object be treated as immutable?
• If it changes, can we cache a template and copy only the mutable state?

This is where performance work becomes modeling work.

You are identifying what is state, what is decision-dependent, what is uncertainty-dependent, and what is static structure.

Separate Static, Dynamic, and Stochastic Objects

A useful classification is:

Static objects

These do not change during the decision run.

Examples:

• network topology,
• product hierarchy,
• supplier eligibility,
• case-pack relationships,
• fixed calendars,
• model index maps.

These are strong caching candidates.

Decision-dependent objects

These change when the candidate action or policy changes.

Examples:

• activated lanes,
• selected suppliers,
• order quantities,
• policy parameters,
• capacity reservations.

These usually need to be rebuilt or updated per candidate.

Stochastic objects

These change by scenario or replication.

Examples:

• demand paths,
• lead-time realizations,
• disruptions,
• returns,
• yield.

These must vary when evaluating uncertainty, although the random draws themselves may be reused across candidates through common random numbers.

Mutable state

These evolve through time.

Examples:

• on-hand inventory,
• pipeline inventory,
• backlog,
• open orders,
• remaining capacity.

These must not accidentally leak from one replication into another.

This classification often reveals the runtime design immediately. Expensive static structures should generally not be rebuilt inside the innermost stochastic loop.

Put Timers Around Real Boundaries

Start with coarse profiling before using sophisticated tools.

Measure major stages:

load_inputs
validate_inputs
build_static_structures
generate_scenarios
build_model
solve_model
simulate_candidates
aggregate_results
write_outputs

For simulation optimization, go one level deeper:

for candidate in candidates:
    prepare_candidate
    for scenario in scenarios:
        reset_state
        simulate
        score

The loop structure matters.

If build_static_structures is accidentally inside the scenario loop, the problem may be architectural rather than algorithmic.

Collect at least:

• total wall-clock time,
• call count,
• mean time per call,
• p50, p95, and maximum time,
• share of total runtime,
• input size associated with each call.

A function taking 50 milliseconds is irrelevant if called once. It is a crisis if called ten million times.

Runtime Is Usually a Product, Not a Constant

For many decision systems, total runtime can be approximated as:

runtime ≈ candidates × scenarios × periods × work_per_transition

That equation is more useful than saying “the simulator is slow.”

Suppose:

• 500 candidate policies,
• 1,000 demand paths,
• 52 weekly periods,
• 0.2 milliseconds per transition.

That is 26 million transitions before counting setup costs.

Now imagine an expensive tree build is performed once per candidate-scenario pair. Even a small setup cost becomes enormous.

Performance work should ask two separate questions:

1. Can we make each unit of work cheaper?
2. Can we reduce the number of units of work?

The second question is often more powerful.

The Search Policy Is Also a Runtime Decision

In simulation optimization, runtime is not only an engineering property. It is partly determined by the search policy.

You choose:

• how many candidates to generate,
• how many scenarios to use initially,
• which candidates deserve more simulation,
• when to stop evaluating a bad candidate,
• when the incumbent is good enough.

A naive design gives every candidate the same simulation budget.

That is usually wasteful.

A better structure might be:

1. Generate 500 candidates.
2. Evaluate all candidates on 20 common scenarios.
3. Drop clearly dominated candidates.
4. Evaluate the top 50 on 200 scenarios.
5. Evaluate the top 5 on 5,000 scenarios.
6. Select using confidence intervals and business risk metrics.

The simulator did not become faster. The decision process became smarter about where to spend computation.

That distinction matters.

Uncertainty Changes How You Measure Performance

A deterministic optimizer can often be benchmarked on a fixed instance.

A stochastic decision system is trickier.

Runtime may depend on:

• number of scenarios,
• path length,
• disruption frequency,
• amount of backlog,
• number of active products,
• policy behavior.

Some policies may create more events than others. A stockout-heavy policy may trigger different logic than a high-inventory policy. A disruption scenario may create expensive recovery behavior.

So benchmark with representative workloads, not one convenient test case.

Use a fixed scenario bank when comparing implementations. Otherwise a code change and a random-seed change can be confused with each other.

Common random numbers are useful here for the same reason they are useful in policy comparison: they reduce noise in the comparison.

Constraints Can Create Hidden Runtime

Business constraints are not free just because they are written outside the solver.

Consider:

• MOQs,
• case packs,
• truckload rounding,
• supplier calendars,
• shelf life,
• substitution,
• capacity,
• multi-echelon dependencies.

Teams often preprocess these rules into large lookup structures. That can be the right design. But the preprocessing cost must be visible.

A useful question for every expensive constraint structure is:

At what frequency can the underlying information actually change?

If supplier eligibility changes monthly, rebuilding the full eligibility graph for every simulation replication makes no sense.

If remaining capacity changes every period, that state cannot simply be cached globally.

The frequency of change should influence the architecture.

Do Not Cache Until You Understand Mutation

Caching is powerful and dangerous.

The classic failure mode is caching an object that downstream code mutates.

Candidate A changes the object. Candidate B receives the modified version. Results become order-dependent. The system gets faster and wrong.

Before caching, verify:

• object ownership,
• mutation behavior,
• thread safety,
• serialization cost,
• invalidation rules,
• memory cost.

Possible patterns include:

Immutable shared object

Build once and share everywhere.

Best when the object is truly read-only.

Cached template plus cheap copy

Build the expensive structure once, then copy only the mutable parts.

Useful when construction is expensive but per-run state is small.

Versioned cache

Key the cache by the inputs that define the structure.

For example:

cache_key = hash(
    supplier_rules,
    product_hierarchy,
    location_config,
    calendar_version
)

If those inputs change, rebuild. Otherwise reuse.

Incremental update

Modify only the part affected by new data.

Useful for large networks where a small number of entities change between runs.

The right pattern depends on the semantics of the object, not on the convenience of the caching library.

Metrics That Actually Matter

Do not track only average runtime.

For a production decision system, track:

• end-to-end runtime,
• p50 and p95 runtime,
• runtime by stage,
• calls per stage,
• time per candidate,
• time per scenario,
• time per state transition,
• memory usage,
• cache hit rate,
• model build time,
• solver time,
• incumbent time,
• total candidates evaluated,
• total scenario replications,
• decision quality versus compute budget.

That last metric is important.

A 50% runtime reduction is not useful if decision quality collapses. A 10x increase in runtime may be justified if the economic improvement is material and the decision is made only once per month.

Performance is part of the decision economics.

Measure Quality Against Compute

For optimization and simulation systems, plot decision quality against runtime.

For example:

5 minutes  → expected cost: $12.8M
10 minutes → expected cost: $11.9M
20 minutes → expected cost: $11.6M
40 minutes → expected cost: $11.55M
80 minutes → expected cost: $11.54M

Now the business tradeoff is visible.

The difference between 20 and 80 minutes may not matter. Or it may matter enormously if the system must respond to intraday disruptions.

Do not optimize runtime in isolation from the cadence of the decision.

A decision made every five minutes has a different compute budget from a quarterly network design study.

Common Failure Modes

Tuning the solver before profiling

The team spends days changing parameters. The solver was 12% of runtime.

Profiling only one run

One product-location combination is not representative of production scale.

Measuring functions but ignoring call counts

A cheap function dominates because it sits inside the deepest loop.

Caching mutable state

The system becomes fast and subtly wrong.

Rebuilding static structures per scenario

Architecture multiplies setup cost by the simulation budget.

Parallelizing the wrong bottleneck

More workers increase contention, serialization, or memory pressure without reducing the dominant cost.

Reducing scenarios blindly

Runtime improves, but the selected policy is now mostly simulation noise.

Optimizing average runtime only

The p95 run misses the operational deadline.

Ignoring model build time

The solver log looks fast while millions of variables and constraints take forever to construct.

Benchmarking with changing randomness

A stochastic difference is mistaken for an engineering improvement.

A Practical Workflow

When a decision system is too slow, use this order.

1. Define the operational requirement

How fast does the decision actually need to be made?

2. Measure end to end

Start the clock at raw input and stop it at usable output.

3. Decompose the pipeline

Find the major stages before diving into individual functions.

4. Inspect the deepest loops

Look for expensive work multiplied by candidates, scenarios, periods, or entities.

5. Classify expensive objects

Static, decision-dependent, stochastic, or mutable state.

6. Remove repeated work

Cache, precompute, reuse, or incrementally update only when semantics allow it.

7. Reduce unnecessary evaluations

Use screening, adaptive budgets, stopping rules, and better candidate generation.

8. Optimize the real hotspot

Only now should you rewrite code, change data structures, parallelize, or tune the solver.

9. Re-profile

The bottleneck moves. Yesterday’s 60% problem may become tomorrow’s 5% problem.

10. Validate decision equivalence

Faster is not better if the decisions changed for the wrong reason.

What to Do in Practice

The next time someone says the optimizer is slow, do not immediately open the solver parameter manual.

Put timers around the full pipeline.

Find the largest block.

Then ask what kind of work it is:

• static structure,
• decision logic,
• stochastic sampling,
• mutable state,
• mathematical search,
• output overhead.

That classification tells you much more than the function name.

The biggest runtime improvement may come from a stronger formulation. It may come from fewer candidates. It may come from adaptive simulation. It may come from caching a structure that should never have been rebuilt. It may come from removing a layer of code that does no useful decision work at all.

The point is not that solver tuning is useless.

The point is that tuning the wrong 10% of the system is useless.

Profile the decision pipeline first. Then optimize what is actually expensive.