Balancing a Simulation with a Harness Instead of Playtesting

I've been building a ranch-management simulator — forage, weather, breeding, herd health, a market — as a browser game with a strictly separated simulation core. Early on it had a problem every simulation game hits: it was unfair, and nobody could say exactly why. Careful play went bankrupt. Neglect was indistinguishable from strategy. The traditional fix is weeks of playtesting and vibes-driven tuning. We did something else: we made balance a regression-tested engineering problem.

The precondition: a deterministic core

The simulation core is framework-free and deterministic by construction. One seeded PRNG whose entire state is a 32-bit integer stored inside the world state, so a save resumes the identical random stream; nine simulation systems that run in a fixed, asserted order every tick. This isn't a convention — it's enforced by lint rules that ban framework imports and platform globals from the core, each ban carrying its own written rationale. Determinism was originally about save integrity. It turned out to be the key to everything that followed.

The calibration harness

Because the core is deterministic and headless, we could write a harness that plays the game by policy. Three scripted players — passive (does nothing), managed (sound ranching: sell steers in fall, cull, feed in winter scarcity), and negligent (overstocks and ignores health) — each run for five simulated years across multiple seeds, reporting net worth, deaths, calf crop, drought exposure, and feed spend, year by year.

The baseline diagnosis was brutal and, crucially, specific: the managed policy went bankrupt on day 1,442. The passive herd lost about 100 head — 98 of them to disease — collapsing from 60 animals to 9 in two years. Zero of the game's objectives were reachable by any policy. A playtester would have said "it feels impossible." The harness said where: the forage system, not the economy. Sixty cows on the reachable acreage stripped the standing crop to zero by the first winter, body condition pinned at minimum, disease risk (derived from body condition) maxed into a death spiral — and a crashed pasture could never regrow against grazing pressure.

One constant

The crux fix was a single number: the seed-bank regrowth floor went from 10 to 300 lb of dry matter per acre. That one constant creates a sustainable-stocking breakpoint at roughly 80–90 animal units: a ~70-head herd's range now heals each spring, while a 120-head herd stays crashed — which is exactly the divergence you want between a managed player and a careless one. Fifteen constants moved in total (five forage, five health, four weather, and a cows-per-bull ratio that had been silently halving the calf crop), but the seed bank was the lever.

Results on the headline seed: managed went from −$36.4k to +$79.5k, breakeven on day 424 instead of bankruptcy on day 1,442, with zero disease deaths. And the part I care most about: negligent still goes bankrupt (day 291) and passive still bleeds. A balance pass that makes every strategy viable hasn't balanced the game, it's deleted it. The tuning had to preserve failure — and the harness proves it does, on every seed, on every future change, because the whole thing runs in CI in seconds.

Documenting the bugs you're not fixing

The tuning pass worked around four structural issues, and instead of hiding them, the decision record lists them: feed costs are double-charged in one path ("still a bug" — the tune makes managed play rarely buy feed, which sidesteps it); passive play can't reach true insolvency because there's no debt interest; proportional selling leaks breeding stock over a long horizon; and at the time of the pass, the herd was locked to a quarter of the map. Written-down known-wrongness beats discovered wrongness every time — the next contributor reads the list instead of re-deriving it from confusing behavior.

Why this beats playtesting (for this class of problem)

Playtesting finds feel problems — pacing, clarity, delight. It's terrible at finding systemic problems, because humans can't hold five years of compounding forage math in their heads, and every playthrough is an unrepeatable sample. The harness gives you repeatable seeds, scripted counterfactuals, and numbers attached to causes. Ten minutes after any core change, three simulated ranchers have already lived fifteen years under the new rules and filed a report. I'd build the harness first on any future simulation project — before the game is fun, before it's even fair, make it measurable.