Meridian: testing fleet operations before launch
A simulation and experimentation platform for fleet, charging, dispatch, and market-expansion decisions.
Meridian is a self-serve product for the moment before a fleet-wide change ships. It combines probabilistic demand forecasting, constraint-aware optimization, and discrete-event simulation so an operations team can ask what a policy change costs before finding out in production.
Simulation architecture, the forecasting and dispatch models, the FastAPI service, and the TypeScript frontend.
Test a fleet change
Engineering brief
- Architecture
- A monorepo: a Python FastAPI service running the simulation, and a Next.js TypeScript frontend. A LightGBM quantile model forecasts demand, scenario generation samples nights from the fitted spread, a SimPy model runs the night, and an OR-Tools solver assigns vehicles. Pydantic models are mirrored as TypeScript types.
- The hard part
- Making a run reproducible. A run is determined by its config and seed and carries an input snapshot with policy, model and engine versions, so a result someone disputes can be replayed exactly, including on another machine.
- Testing
- 25 Pytest cases across config, demand, dispatch, runner behaviour and reproducibility. The reproducibility test caught a real bug: the served-zone set was a Python set, and per-process string hash randomisation fed a different order into the sampler, so identical seeds diverged across machines while the suite stayed green.
- Known limitation
- No routing engine: distance is straight-line times a circuity factor at one average speed, which is the largest single source of error. Demand is synthetic, there is no significance testing on the difference between arms, and nothing is calibrated against a change that actually shipped.
“It depends on the night” is a real answer, and a useless one
An operations team wants to expand a service area, prioritise airport pickups, or change how vehicles are dispatched. Someone asks what it will cost. The honest answer is that it depends on demand, on how much charging capacity is free that evening, and on where the fleet happens to be sitting when the demand arrives.
A spreadsheet becomes brittle once demand, vehicle state, charging queues, and dispatch policy interact over time. So these decisions get made from intuition and a partial rollout, and the failure mode is consistent: the change looks fine for a week, then a busy Friday exposes a constraint nobody was watching, usually somewhere other than where the change was made.
Meridian exists to move that discovery earlier. It runs the change against many sampled nights, reports the spread rather than a point estimate, and returns a decision with the reasoning and the guardrails attached.
Three teams, one artifact
The hard part of a tool like this is that the three groups who need it want different things from the same run, and a product that serves only one of them gets ignored by the other two.
Operations
Decide whether a policy or service-area change is safe to roll out, and know what to watch if it goes to pilot.
Data science
Check that the forecast underneath a recommendation is calibrated, and that its uncertainty reached the answer.
Engineering
Understand what a policy change actually requires, and reproduce a result that looks wrong.
The run output is built so one artifact answers all three: a verdict and guardrails on top for operations, the calibration and spread in the middle for data science, and a reproducible snapshot at the bottom for engineering.
What the system refuses to do
The recommendation is a rule, not a model
A learned recommender would be less useful here, because the disagreement is almost never about the arithmetic. It is about the target. An operator has to be able to read why a change was called, disagree with a threshold, edit it in the config, and re-run. Making the verdict a transparent rule over the experiment’s own targets keeps that argument possible.
“Pilot” is the verdict that earns its keep
A binary launch-or-reject is too crude for this decision. A change that moves the primary metric a long way without quite reaching its goal is a pilot candidate: the goal may be wrong, or the remaining gap may be closable with capacity rather than policy. What earns a rejection is a change that does not meaningfully move the metric it was designed to move.
No metric is a bare point estimate
Every figure in the product is a mean with a P10–P90 band, and the uncertainty chart plots one bar pair per simulated night. That is not decoration. The central finding in the flagship experiment is that the proposed change clears its target on a typical night and breaches a guardrail on a busy one, and a table of means would have hidden it.
Both arms see the same night
Each replication samples demand once and runs baseline and proposed against that identical draw. Pairing removes weather from the comparison, so any difference between the arms is policy. It also buys far more signal per second of compute than simply running more independent nights.
Zones are a schematic, not a map
The service area is invented, so drawing it over real cartography would imply a real market. Circles sized by demand and filled by completion rate show which parts of the network a change helps and which it strains, which a choropleth of a real city would obscure.
Experiments are files, not database rows
An experiment definition is a YAML file that moves through pull request review like the code it tests. That is also what makes a config version and an input snapshot mean something: a result is tied to a definition someone approved.
Forecast, sample, simulate, decide
A LightGBM quantile model learns P10, P50, and P90 demand for every zone and every fifteen-minute interval. Those quantiles are not a chart: scenario generation draws each night's arrivals from the fitted spread, so a zone with a genuinely uncertain forecast moves more than a zone with a tight one, and forecast uncertainty reaches the recommendation instead of stopping at a visualisation.
A SimPy model then runs the night. Vehicles are processes, chargers are a resource with real capacity so queueing emerges from contention rather than assumption, and riders leave if nobody arrives. Two dispatch policies compete on identical demand: greedy nearest-available, and a batched min-cost assignment solved with OR-Tools that encodes airport priority and a battery reserve as cost rather than as hard rules. Whether the optimisation earns its complexity is the question, so both are first-class.
A run is determined by its configuration and seed, and carries an input snapshot, policy version, model version, and engine version. Re-running reproduces the numbers exactly, including on another machine.
experiments/*.yaml ──▶ FastAPI ──▶ per replication:
(versioned config) sample_scenario ← P10/P50/P90 spread
FleetSim (SimPy) ← vehicles, chargers, riders
└ dispatch ← greedy | OR-Tools
aggregate → bands
recommend → verdict
│
Next.js UI ◀─────────────────────────────── ┘
library · workspace · resultsBuilding it surfaced more design errors than writing it. Dispatch was assigning vehicles twenty-two minutes away to riders with eleven minutes of patience, so both arms threw away trips for no reason. The optimiser went infeasible whenever pending requests outnumbered idle vehicles and quietly fell back to greedy, which meant the optimised arm was running the baseline during exactly the busy periods it existed for. Repositioning aimed one interval ahead while crossing the metro took longer than one interval, so vehicles arrived after the demand they were sent for.
The one worth keeping was reproducibility. The set of served zones was a Python set, and CPython randomises string hashing per process, so iterating it fed a different order into the destination draw on every run. Identical configs with identical seeds diverged between machines while passing every test, because an in-process repeat cannot catch it. The fix is small. Finding it required not trusting a green suite.
Late-night airport expansion
Can airport priority and service-area expansion meet a seven-minute pickup target without overloading overnight charging?
The baseline is current operations: 220 vehicles, two depots, thirty-six chargers, nearest-available dispatch, existing service area. The proposed arm keeps the same fleet and adds an airport-priority queue, a fifteen percent battery reserve for airport trips, demand-aware repositioning, and three new zones.
| Metric | Baseline | Proposed | Change |
|---|---|---|---|
| Airport P90 pickup, curbside | 9.8 min | 7.4 min | −24% |
| Trip completion rate | 90.8% | 87.2% | −3.6 pts |
| Completed trips | 2,578 | 2,618 | +41 |
| Deadhead share of miles | 26.8% | 31.6% | +4.8 pts |
| P90 charger queue | 6.1 min | 12.1 min | +99% |
| Cost per completed trip | $3.61 | $3.74 | +3.4% |
Pilot recommended, and the reason is not dispatch
Airport pickup improves substantially and the fleet completes more trips overall. But trip completion rate drops three and a half points as expansion adds demand faster than the policy absorbs it, deadhead rises by nearly five points, and under upper-tail demand the charger queue reaches roughly double its baseline and breaches the twelve-minute guardrail.
The useful part is where the constraint turned out to live. The experiment was designed as a dispatch question, and dispatch is not what limits it. More vehicles finish a busier night needing charge at the same time, and thirty-six stalls cannot absorb that. A fleet-wide launch would have found this on a Friday.
So the recommendation is a staged pilot on the airport zone and two expansion zones, with charger headroom monitored nightly and depot capacity added before the full service area follows. The product also declines to overclaim: the proposed arm closes most of the gap to the seven-minute target without reaching it, and the summary says so rather than moving the goalposts.
Demand is synthetic
The model is trained on generated history, so its holdout metrics measure whether it learned its own generator, not whether it would forecast a real market. Everything downstream treats the model as a versioned artifact, so swapping in a real trip feed is a substitution rather than a rewrite.
There is no routing engine
Distance is straight-line times a circuity factor at a single average speed. Absolute pickup times are indicative; the comparison between two arms under identical geography is the result. This is the largest single source of error in the numbers.
No significance testing yet
With twenty replications, a small difference between arms is not distinguishable from noise, and the product does not currently say so. It is the most important missing piece of statistical rigour, and it is first on the roadmap for that reason.
Unit economics are illustrative
Cost per completed trip uses assumed vehicle-hour, per-mile, and charging figures. The direction of the change is meaningful; the level is arbitrary and labelled as such in the product.
Uncalibrated against reality
No backtest against a known past change exists, so Meridian is directionally useful and quantitatively unvalidated. The documentation says that in those words rather than leaving it implied.
What I would build next
Ordered by what would hurt first if this had to support real decisions rather than demonstrate how they could be supported.
- 01
Real demand, on a schedule
Replace generated history with a trip feed and retrain on a cadence, with drift monitoring. The model is already a versioned artifact with its coverage reported in every run, so this slots in without touching the simulation.
- 02
Confidence on the difference, not the outcome
Report intervals on the delta between arms, add power analysis to size replication counts, and stop sweeps early once the answer is settled. This is what turns “looks better” into “is better”.
- 03
A job queue
Runs move to a worker pool with persisted status and object-storage output, so a two-hundred-replication sweep is not an HTTP request. Run history stops being process memory.
- 04
Routing and time-of-day speeds
A real routing engine removes the biggest approximation in the model and makes absolute pickup times defensible rather than indicative.
- 05
Calibration against a known change
Backtest a change that already shipped and check the simulation’s predicted direction and magnitude. Until that exists, the tool supports decisions but cannot claim accuracy.
- 06
Charging fidelity
Charge curves rather than a linear rate, per-stall power limits, and depot electrical constraints. The flagship experiment is charging-bound, so this is where the answer is most sensitive to the model being wrong.