Building an F1 Strategy Prototype: The Search Layer, Part 4

If you are coming in here first, Part 1 frames F1 strategy as a combinatorial decision space, Part 2 builds the simulator, and Part 3 generates legal branches.

This part is about the layer that sits around those pieces.

The simulator answers:

If we follow this strategy under this scenario, what happens?

The decision model answers:

Which actions or plans are legal enough to consider?

The search layer asks:

Which action should we take now, given the futures we can simulate?

That sounds like "find the best strategy", but the useful framing is narrower. The search layer does not discover truth. It spends a limited compute budget comparing reachable futures, then returns a recommendation with enough evidence that a person can trust it, question it, or override it.

The search layer owns attention

In Part 3, the decision model reduced a raw candidate set to 22 legal toy plans for a 24-lap race. That is small enough to simulate exhaustively. A real race state is not that polite.

The full tree can include:

• pit now, stay out, cover, extend, push, or manage
• different compounds and stop counts
• rival reactions
• traffic windows
• Safety Car and VSC timing
• tyre degradation uncertainty
• weather transitions
• two-car team interactions

The job of search is to decide where attention goes.

It should not simulate every branch equally just because the branch exists. Some futures are illegal. Some are legal but strategically thin. Some deserve a quick estimate. Some deserve many rollouts because the decision is close.

That is the practical difference between a simulator and a strategy system. A simulator can evaluate a candidate. A search layer decides which candidates deserve evaluation.

Start with batch enumeration

The first search method should be boring: enumerate legal plans, simulate each one, rank the outcomes.

def rank_plans(initial_state, plans, params):
    results = []

    for plan in plans:
        final_state = simulate_race(initial_state, plan, params)
        results.append((score(final_state), plan))

    return sorted(results, reverse=True)

This is not sophisticated, but it is a strong baseline. It creates a clean habit:

1. Generate alternatives.
2. Apply the same simulator to each alternative.
3. Score the resulting state.
4. Compare the outcomes.

That already beats a surprising amount of hand-wavy strategy thinking. Instead of saying "lap 14 feels like the stop", the model has to say "lap 14 wins under these assumptions and this objective."

For a dry race toy model, the candidate set might contain plans like:

M-H, stop on lap 10
M-H, stop on lap 12
M-S, stop on lap 15
M-S-H, stop on laps 8 and 18
M-H-S, stop on laps 10 and 19

That is not the race yet. It is the planning surface. The search layer turns that surface into ranked evidence.

Scoring is part of the model

The ranking is only meaningful if the score is honest.

Possible objectives include:

• minimise total race time
• maximise expected points
• maximise probability of beating a target rival
• maximise podium probability
• minimise downside risk
• trade expected result against variance

None of those is universally correct. A leader may prefer the branch with a slightly lower mean result if it protects against the painful downside. A car running P11 may need a high-variance strategy because a safe P11 is still no points. A team fighting one championship rival may care more about finishing ahead of that car than about a theoretical average finishing position.

So the score function should be explicit:

def score(outcome):
    points_value = points_for_position(outcome.position)
    rival_bonus = 2.0 if outcome.beats_target_rival else 0.0
    downside_penalty = 1.5 if outcome.position >= 8 else 0.0

    return points_value + rival_bonus - downside_penalty

The constants are placeholders. The structure is the point. The model is not simply asking "which branch is fastest?" It is asking "which branch best matches the strategic objective?"

That distinction matters because strategy has context. A fast plan with a large failure mode may be the right answer in one race and the wrong answer in another.

Add uncertainty with Monte Carlo

A deterministic simulator can make strategy look cleaner than it is. It produces one future. Races do not offer one future.

Monte Carlo simulation changes the question:

How does this strategy perform across many plausible race futures?

For each candidate plan, sample uncertain inputs:

• tyre degradation
• pit stop loss
• traffic after rejoin
• lap-time noise
• rival pace
• rival pit timing
• Safety Car or VSC timing
• weather transition timing, if the prototype includes weather

Then simulate the plan many times:

def evaluate_plan(plan, scenario_sampler, runs):
    outcomes = []

    for _ in range(runs):
        scenario = scenario_sampler()
        final_state = simulate_race(plan, scenario)
        outcomes.append(score(final_state))

    return summarise(outcomes)

The summary should not only report the mean. Expected value is useful, but risk often decides the call.

Useful outputs include:

• mean score
• mean gap to target rival
• probability of beating the target rival
• probability of finishing above a threshold
• probability of a painful downside
• worst-decile result

For example:

Strategy A:
  average finish: P6.1
  P5 or better: 18%
  P8 or worse: 9%

Strategy B:
  average finish: P6.5
  P5 or better: 31%
  P8 or worse: 24%

Strategy A is more stable. Strategy B has more upside and more downside. Neither answer is complete without race context.

A tiny Monte Carlo prototype

The support script for this post reuses the same 24-lap toy setup from Part 3. It generates legal plans, samples simplified race scenarios, simulates each plan 500 times, and ranks by a small score function.

The scenario model is intentionally plain:

• the target rival uses a fixed medium-hard plan
• degradation varies per run
• pit loss varies per run
• traffic appears in short windows
• a Safety Car may reduce pit loss if it lands near a stop
• the score rewards position and beating the target rival, then penalises downside

The script is not calibrated F1 machinery. It is a search-layer demonstration.

uv run scripts/f1_strategy/search_layer.py

Monte Carlo search layer
scenario: 24 laps, start=medium, candidate_plans=22, runs_per_plan=500
target rival plan: pit_laps=(11), compounds=(hard)

rank  pit_laps   compounds        gap    beat     P5+     P8-   score
   1  (10)       (hard)        -3.635 100.0% 100.0%   0.0%  12.000
   2  (11)       (hard)        -3.848 100.0% 100.0%   0.0%  12.000
   3  (12)       (hard)        -4.263 100.0% 100.0%   0.0%  12.000
   4  (14)       (soft)        -7.502  92.6%  88.2%   0.6%  11.447
   5  (15)       (soft)        -7.819  90.0%  86.4%   2.0%  11.258
   6  (16)       (soft)        -7.433  88.0%  85.0%   1.0%  11.185
   7  (9)        (hard)        -2.191  83.4%  83.4%  15.0%  10.479
   8  (13)       (hard)        -2.615  84.8%  72.0%  11.2%  10.440

The interesting result is not the exact ranking. The useful part is the disagreement between metrics.

The soft-tyre branches have better mean gap to the rival in this toy model. But they also carry more variance because they are exposed to traffic and late-stint degradation. The hard-tyre branches score better because the objective rewards robustly beating the target rival and avoiding downside.

That is the search layer doing real work. It is not just finding the lowest simulated race time. It is ranking futures according to a declared objective.

Receding-horizon strategy

Given my aerospace background and inconvenient attachment and obsession with control systems, I was probably always going to drag this prototype towards control theory at some point. The useful concept here is receding-horizon control: make a forward-looking plan, apply only the first move, then re-plan when the state changes.

Batch enumeration works when the strategy can be written upfront. Live strategy is more awkward.

At lap 12, the team may choose to stay out. At lap 13, a rival pits. At lap 14, traffic opens. At lap 15, the driver reports graining. At lap 16, a VSC changes the pit-loss calculation. The correct plan is not solved once at the start. It is re-solved as the state changes.

This is a receding-horizon problem:

observe current state
generate legal actions
simulate futures from those actions
choose the best immediate action
apply only that action
observe the new state
repeat

The system may simulate to the end of the race, but it only commits to the next action. That is important. A model can prefer "stay out now" while still expecting to pit later if the window remains open.

The output of search should therefore be an immediate recommendation, not a sacred full-race script.

Recommended action: stay out

Reason:
  Pitting now has a strong mean pace result, but it rejoins into traffic
  in 63% of sampled futures.

  Staying out keeps the overcut branch open and reduces the probability
  of falling behind the target rival.

That format is more useful than a naked answer. It gives the strategist a reason to inspect the model instead of guessing what it has optimised.

When tree search starts to fit

Enumeration is enough while the candidate set is small. Once the model starts choosing immediate actions under changing race state, the structure becomes a tree.

Lap 14:
  stay out
    rival pits
      cover next lap
      extend
    rival stays out
      pit for undercut
      hold track position
  pit now
    fit hard
      manage to the end
      push to clear traffic
    fit soft
      attack now
      accept late degradation risk

The tree gets large quickly. Monte Carlo Tree Search is one reasonable tool for this shape.

If I were implementing it directly, I would start with the four standard stages:

select a promising branch
expand one action
simulate a rollout
backpropagate the result
repeat until the budget is used

The minimum model is a node that stores a race state, the action that led to it, its children, and the rollout values seen so far:

class Node:
    def __init__(self, state, action=None, parent=None):
        self.state = state
        self.action = action
        self.parent = parent
        self.children = {}
        self.visits = 0
        self.total_value = 0.0

    @property
    def mean_value(self):
        return self.total_value / self.visits if self.visits else 0.0

Selection is where the tree balances exploitation and exploration. The usual starting point is UCT:

from math import log, sqrt

def uct(child, exploration=1.4):
    if child.visits == 0:
        return float("inf")

    parent_visits = child.parent.visits
    return child.mean_value + exploration * sqrt(log(parent_visits) / child.visits)

For this F1 prototype, I would avoid putting too much race logic directly into UCT. The formula should stay mostly generic. It only decides which branch deserves another rollout. The F1-specific judgement belongs in the things UCT is sampling:

• the rollout model, where Safety Car timing, tyre degradation, traffic, and rival reactions are sampled
• the reward function, where beating a target rival, downside risk, and Safety Car optionality are valued
• the diagnostics, where the model reports why a branch was visited or avoided

That separation matters because "balance Safety Car risk" is not a single constant. A high exploration value does not know what a Safety Car is. It only says: keep testing uncertain branches for longer before committing compute to the apparent favourite.

The support script for making that visible is deliberately small:

uv run scripts/f1_strategy/uct_risk_sweep.py

It runs a toy UCT search from lap 8, compares stay out against pit hard, and samples a few pieces of uncertainty: whether a Safety Car creates a cheaper future stop, and whether the rival covers, offsets, stops late, or holds track position.

UCT risk sweep
scenario: lap=8, start=medium, simulations=700, seed=23
uncertainty: sampled Safety Car timing and rival cover/offset/late/hold reactions

   c  recommend action    visits    mean   bonus    beat    down     SC+   cover
0.15  stay out  stay out     690   0.689   0.015  94.3%   3.2%  51.3%   0.0%
0.15            pit hard      10   0.564   0.121  90.0%  10.0%   0.0%  30.0%
0.70  stay out  stay out     621   0.692   0.072  94.7%   3.2%  52.2%   0.0%
0.70            pit hard      79   0.556   0.202  88.6%  10.1%   0.0%  27.8%
1.40  stay out  stay out     547   0.697   0.153  95.1%   2.6%  51.7%   0.0%
1.40            pit hard     153   0.560   0.290  88.9%   9.2%   0.0%  34.0%
2.20  stay out  stay out     474   0.696   0.259  94.7%   2.7%  54.0%   0.0%
2.20            pit hard     226   0.580   0.375  90.7%   7.5%   0.0%  34.5%

The recommendation stays the same in this toy setup, but the search behaviour changes. With c=0.15, the tree almost immediately exploits stay out. With c=2.20, it spends about a third of the budget checking pit hard, even though that branch has lower mean value. That extra attention is useful when the model is uncertain about rival reaction or a rare race-control event. It is wasteful when the branch is already well understood.

This is the practical UCT knob: it does not make the strategy risk-seeking or risk-averse by itself. It changes how much evidence the search demands before it stops asking awkward questions.

Then the search loop maps cleanly onto the race model:

def choose_action(root_state, budget):
    root = Node(root_state)

    for _ in range(budget):
        node = select(root)
        child = expand(node)
        outcome = rollout(child.state)
        backpropagate(child, outcome)

    return best_child(root).action

The mapping is almost too neat. legal_actions(state) is the Part 3 decision model. transition(state, action) is the simulator step. rollout(state) is a noisy run to the end of the race. backpropagate updates the branch values.

The catch is that a useful implementation quickly becomes less tidy than the sketch. You need state snapshots, reset behaviour, action masks, rollout policy control, reward scaling, tree bookkeeping, and preferably a way to inspect the tree after it makes a recommendation.

For this prototype, the more useful work is in shaping the F1 environment, not in writing another generic MCTS class. So the support script uses gymcts, a Monte Carlo Tree Search package built for Gymnasium-style environments.

That fits the direction of the prototype:

• Part 2 already moved the simulator towards discrete event thinking.
• Part 3 already framed strategy as legal branches under constraints.
• Part 4 needs a look-ahead search that can step through those branches and roll forward.

The useful bridge is to wrap the race strategy state as a Gymnasium environment:

class F1StrategyEnv(gym.Env):
    def action_masks(self):
        return np.array([
            can_stay_out(self.state),
            can_pit_soft(self.state),
            can_pit_hard(self.state),
        ])

    def step(self, action):
        next_state = apply_strategy_action(self.state, action)
        reward = score(next_state)
        return observation(next_state), reward, done, False, info

Then gymcts owns the MCTS mechanics:

env = F1StrategyEnv(seed=11)
env.reset(seed=11)

wrapped_env = DeepCopyMCTSGymEnvWrapper(
    env,
    action_mask_fn=lambda wrapped: wrapped.unwrapped.action_masks(),
)

agent = GymctsAgent(
    env=wrapped_env,
    number_of_simulations_per_step=180,
    clear_mcts_tree_after_step=False,
)

action, node = agent.perform_mcts_step()

The first tree-search support script is:

uv run scripts/f1_strategy/gymcts_search_tree.py

gymcts F1 strategy tree search
state: lap=8, start=medium, simulations=180, safety_car_lap=13
recommended next action: pit hard

action      visits    value
pit hard        92    0.323
stay out        87    0.316

The same script can also ask gymcts to render the tree it searched:

uv run scripts/f1_strategy/gymcts_search_tree.py --show-tree --tree-depth 2

The gymcts visualisation renderer prints a terminal tree. It shows the visit count N, the average node return Q_v, the action a that leads to a node, the best rollout return seen from that node, and the UCT score used during traversal.

Here is a cleaned excerpt from the same run:

MCTS tree visualisation, depth=2
(N=180, Q_v=0.25, best=0.36)
+-- (a=0, N=87, Q_v=0.23, best=0.36, UCT_v0=0.56)
|   +-- (a=0, N=12, Q_v=-0.15, best=0.05, UCT_v0=0.59)
|   |   +-- (a=0, N=3, Q_v=-0.34, best=-0.30, UCT_v0=0.60)
|   |   +-- (a=2, N=8, Q_v=-0.06, best=0.05, UCT_v0=0.57)
|   +-- (a=2, N=74, Q_v=0.30, best=0.36, UCT_v0=0.59)
|       +-- (a=0, N=73, Q_v=0.30, best=0.36, UCT_v0=0.58)
+-- (a=2, N=92, Q_v=0.27, best=0.35, UCT_v0=0.56)
    +-- (a=0, N=91, Q_v=0.27, best=0.35, UCT_v0=0.55)
        +-- (a=0, N=90, Q_v=0.27, best=0.35, UCT_v0=0.55)

In this environment:

• a=0 means stay out
• a=1 means pit soft
• a=2 means pit hard

The root summary looks close because MCTS has kept both first moves alive. The shape underneath is the useful bit. After pit hard, nearly all visits continue with stay out. After stay out, the tree still spends meaningful effort on a later pit hard branch.

This is a different shape from the earlier Monte Carlo enumerator. The enumerator compares complete plans equally. The MCTS version starts from the current lap, allocates more visits to the branch that looks useful, and returns the next action.

That hides most of the important work, but in the right place. MCTS is not magic. The rollout policy matters. The action generator matters. The score function matters. The rival model matters. The exploration rule matters.

Still, the fit is useful because MCTS does not spend equal effort everywhere. It can allocate more rollouts to branches that look promising while still exploring alternatives enough to avoid getting trapped by the first good-looking plan.

The SimPy bridge changes the question

There's a cleaner bridge back to the race simulator in Part 2, and it is not rhetorical. It's structural and hiding in the shape of the problem itself. It appears once the Gymnasium wrapper stops treating the transition as just a compact state update. It's the same bridge that made SimPy useful in Part 2.

The first gymcts script is useful, but its environment transition is still a compact Python function. It updates the race state directly. That is fine for proving the MCTS interface, but it does not use the event structure that made SimPy useful in the simulator post.

The more interesting version is a bridge between gymcts, the Gymnasium environment, and the SimPy-based race simulator. It looks like this:

gymcts
  chooses an action

Gymnasium environment
  exposes observations, action masks, rewards, and reset/copy behaviour

SimPy
  advances the next race segment with event timing, pit resources, and queues

That is the symbiosis. gymcts does not need to know how a pit box works. It only needs a Gymnasium environment it can copy, step, and score. The environment does not need to implement MCTS. It only needs to turn an action into a next state. SimPy owns the awkward timing details inside that transition.

The support script for that bridge is:

uv run scripts/f1_strategy/simpy_gymcts_bridge.py

In this toy setup, the environment starts on lap 8. The ego car can stay out, pit soft, or pit hard. A teammate is scheduled to use the same team pit box on lap 10, the rival pits on lap 11, and a Safety Car window exists around lap 13. Each MCTS step advances the next lap through a small SimPy event simulation rather than a direct state update.

SimPy + gymcts F1 strategy bridge
state: lap=8, start=medium, simulations=220, safety_car_lap=13
simpy events: shared ALT pit box, teammate pit lap=10, rival pit lap=11
recommended next action: stay out

action      visits    value
stay out       181    0.763
pit hard        38    0.549

That is the interesting part: the recommendation changes.

The pure gymcts script recommends pit hard:

gymcts F1 strategy tree search
state: lap=8, start=medium, simulations=180, safety_car_lap=13
recommended next action: pit hard

action      visits    value
pit hard        92    0.323
stay out        87    0.316

The SimPy-backed version recommends stay out.

This demonstrates that simulator fidelity can change the search recommendation.

This is not proof that staying out is the real strategic answer. It is a toy model with toy parameters. But it is a useful result because the difference comes from the transition model. Once the search layer sees event timing, a shared pit box, queue delay, and a future Safety Car-shaped pit-loss change, it values the branches differently.

That is exactly why the simulator boundary matters. Search is only as good as the futures it can roll forward. If the transition function hides pit-lane contention, MCTS cannot reason about pit-lane contention. If the transition function exposes it through SimPy, the tree search can allocate visits around it.

For an F1 prototype, I would reach for tree search when these conditions appear:

• actions must be chosen one lap at a time
• rival reactions change the value of later actions
• Safety Car or weather branches matter
• the legal plan set is too large for equal simulation
• a rollout policy can provide plausible default decisions

Until then, enumeration plus Monte Carlo is easier to inspect. Once the prototype needs sequential decisions, a Gymnasium wrapper plus gymcts is a cleaner next step than hand-rolling a generic tree-search engine. Once the prototype also needs event timing, that wrapper should not bypass the simulator. It should step through it.

Explain the recommendation

The search output should not be:

PIT NOW

It should be closer to:

Recommended action: stay out

Compared actions:
  pit hard now:
    mean score: 8.9
    probability of beating target rival: 54%
    probability of P8 or worse: 19%

  stay out:
    mean score: 9.6
    probability of beating target rival: 63%
    probability of P8 or worse: 11%

Reason:
  Pitting now gains tyre life but rejoins behind traffic in most sampled futures.
  Staying out keeps the overcut branch open and has lower downside under the
  current traffic model.

That explanation does three things.

First, it states the action. Second, it shows the alternatives. Third, it exposes the reason in terms the model can be challenged on.

If the traffic model is wrong, the recommendation may change. Good. That is exactly the assumption a strategist should inspect.

What this layer should produce

For this prototype, I want the search layer to return:

• a recommended immediate action
• ranked alternatives
• outcome distributions, not only means
• the score function used for ranking
• the main assumptions behind the recommendation
• the branches that were pruned or ignored
• enough metadata to replay the result

The replay point matters. Search systems are easy to make non-reproducible by accident. Random seeds, sampled scenarios, model versions, and scoring parameters should be captured when a recommendation is produced. Otherwise, the model becomes hard to audit.

Dynamic programming, heuristics, and optimisers

Other optimisation tools can help, but I would be careful about where they sit.

Dynamic programming is attractive if the state can be made small and Markovian enough. In a toy race, a state might be (lap, compound, tyre_age, pit_count, gap_bucket). The model can cache values for repeated states and avoid simulating the same suffix repeatedly.

The problem is state size. Once traffic, rival behaviour, Safety Cars, tyre inventory, and weather are included, the state representation can become too large or too approximate. Dynamic programming is still useful as a mental model: do not recompute equivalent futures if you can cache them. But I would not force the whole race into a neat table too early.

Heuristics are also useful. A search layer can prune branches with simple rules:

• do not pit into an obvious traffic window unless the tyre delta is large
• do not spend deep rollouts on a two-stop branch that cannot recover pit loss
• do not consider soft tyres for a final stint outside the useful window
• prefer branches that keep a Safety Car option open when the downside is small

Those rules should be labelled as pruning, not legality. That distinction from Part 3 still matters. A legal branch can be pruned because it is not worth compute in this scenario, but the model should be able to say that is what happened.

Packages such as OR-Tools, SCIP, HiGHS, Pyomo, PuLP, SciPy, and CVXPY can all fit around the search layer:

• OR-Tools CP-SAT can generate discrete legal plans or choose among candidate strategies.
• SCIP, HiGHS, Pyomo, or PuLP can handle small linear or mixed-integer selection problems.
• SciPy can fit tyre degradation and pace parameters used by the simulator.
• CVXPY can help if a convex subproblem appears naturally.

Several commercial solvers also exist, with varying performance advantages over the previously mentioned free and open-source options: AMPL, Gurobi, Hexaly, IBM CPLEX, and FICO Xpress are common examples.

The outer shape remains simulation plus search. I would not ask one optimiser to understand the whole race on day one. A more practical pattern is:

generate candidate strategies
simulate each strategy under sampled scenarios
summarise outcome distributions
choose among candidates subject to risk and team constraints

That is less dramatic than "the optimiser solves the race." It is also more likely to survive contact with the model's assumptions.

Other paths to explore

Artificial Intelligence (AI) and Machine Learning (ML) would be another natural layer to explore, but I would use it around the simulator before asking it to replace the simulator. Gradient-boosted tree methods such as XGBoost, LightGBM, CatBoost, and even the older AdaBoost family, or more cutting-edge transformer-based models, could help estimate pieces of the model: tyre degradation, traffic loss, pit-loss variation, rival reaction probability, or the value of a rollout state. The choice depends on the data shape. LightGBM is attractive when the dataset is large and training speed matters. CatBoost is useful when categorical features are important and the first baseline needs to be strong without much feature surgery. XGBoost is still the balanced workhorse: fast enough, customisable, and supported by a large ecosystem. AdaBoost is simpler and can still be useful on smaller, cleaner classification problems. In this framing, ML would not remove the need for search. It would make the simulator, pruning rules, or rollout policy less hand-tuned.

The caveat

A search layer can make a weak model look authoritative. More rollouts do not fix bad assumptions. They only evaluate the assumptions more thoroughly.

If tyre degradation is wrong, Monte Carlo will confidently rank the wrong futures. If the rival model is naive, tree search will optimise against an opponent that does not exist. If the score function rewards the wrong thing, the recommendation can be internally consistent and strategically poor.

That is why this series keeps the boundaries explicit:

simulator:      turns strategy and scenario into consequences
decision model: generates legal and sensible branches
search layer:   allocates compute and ranks futures
score function: states what "better" means

Across the four parts, the prototype has moved from framing the decision space, to simulating futures, to generating legal branches, and finally to ranking those futures under uncertainty.

The useful endpoint is not a model that says "trust me." It is a model that says:

Given these assumptions, this action has the best risk-adjusted distribution.

Here are the alternatives.

Here is why the ranking changes if the assumptions change.

That is the part I keep coming back to. F1 strategy is not just about predicting the fastest future. It is about keeping the right futures alive while the race is still moving.

And that feels like the right place to leave this series. Before the models, the branches, the simulations, and the slightly obsessive control-systems framing, every race still begins with the same clean moment of uncertainty:

It's lights out and away we go.