Context
LOUD is a top-down zombie roguelite prototype I'm building in Godot with Claude Code. Weapons fire on their own, loud weapons raise a Heat meter that brings more zombies, and from 06:00 you decide when to extract with the salvage you're carrying. Death loses all of it.
Nobody had played the newer builds yet, and the project's rule is that the game is tuned only from playtest evidence. Still, design questions kept coming up that a few playtests can't answer: does the shotgun cost more health than it's worth, is there a difficulty cliff when Brutes arrive, does staying past 06:00 pay? Each of these needs hundreds of matches played the same way, and the bot playing them has to be good enough that its results mean something.
What I built
The lab
- Bots on the real game. A match is the real arena scene with every real
system. The player reads input through a
PlayerInputobject; a bot swaps in aBotInputand sets only what a gamepad can: a stick vector, a BoomBox throw and target, and interact pressed or held. Upgrade cards are picked by pressing the real menu buttons. - Fair observations. Bots see zombies inside the camera view, with velocities estimated from two looks, plus the lanes and objectives the HUD shows. No off-screen enemies, spawn schedules or RNG state.
- Faster, and provably the same. Headless with
--fixed-fps 60, every frame is one 60 Hz physics tick and the engine runs ticks as fast as the CPU allows. Nothing is skipped or scaled. A speed check replays the same matches at real time, headless and in a window, and compares a state hash every second: 52 of 52 identical. - Utility bots. Twenty times a second a bot scores 17 moves (16 directions or standing still) on predicted threat, crowding, walls, goals and open space. Seventeen JSON profiles, from a random baseline to an aggressive and a tactical bot, change those weights without code.
- A batch runner (
loudsim.py, Python standard library only) that spreads matches over parallel Godot workers, recovers from crashes and hangs, resumes an interrupted run, and writes everything to SQLite. - Reports as standalone HTML, with what was measured, the statistics, hypotheses and the next experiments kept in separate sections.
What the studies found
- The first balance study (1,978 matches): for these bots offence beats defence, the shotgun costs health, the bat costs kill speed, and damage taken steps up at 2:00 when Brutes arrive. Two thirds of deaths were near a wall.
- Experiment 1 asked whether those wall deaths came from poor bot navigation. A wall-aware layer moved 17 points of deaths off the walls without making the bots die less, so wall deaths were a symptom of being overrun. The bigger finding was a bug in the bots' perception: one line appended to a copy of an array, so every bot counted at most one zombie per 96 px cell. The verdict: trust the first study's directions, not its magnitudes.
Training the bot
The tactical bot is the lab's most complete policy, with escape lookahead, range keeping and both BoomBox modes, but its weights were hand-tuned: someone picked the best of five variants. Training replaces that with a search.
- What learns: 23 of the tactical bot's settings (17 movement, 2 BoomBox, 4 extraction), starting from the hand-tuned values.
- How: an evolution strategy. Each generation plays the current settings plus seven mirrored pairs (a random nudge in both directions), ranks each pair, and steps toward the better side.
- Fitness: mean salvage banked per match, zero on death, over Pistol, Shotgun and Bat starts. That's the game's own goal.
- No memorizing: every candidate plays the same seeds within a generation, and the seeds change every generation.
- Three seed pools: training seeds drawn fresh each generation, a fixed validation set every five generations that picks the checkpoint, and a test set touched once, at the end.
The technical article covers the bot, the determinism work, the perception bug and the training loop in detail, with the code.
How I worked with Claude Code
Claude Code (Claude Opus 5.5) wrote the lab, ran the batches and wrote the study reports; every commit carries its co-author line. My part was what to build, which questions to ask, and what the results were allowed to mean:
- The phases. I scoped each one: the lab itself, Experiment 1 on wall deaths, the throughput work on the M4 Pro, and the training.
- The budget. After the noise measurement I chose eight seeds per candidate over four, doubling the run for a more reliable ranking.
- What the bots may decide. Bot results suggest experiments; they don't change the game. Tuning stays with playtests.
- The compute. The long batches ran lid-closed on my MacBook, at the worker count the lab measured as fastest.
Scope
- Bots, not people. No human playtest data exists for these builds, so every number describes these bots on that build.
- Old perception. The training ran with the original perception, the one that undercounts dense crowds. A retrain with the fix is a separate dataset.
- No ceiling yet. The learning curve was still rising at generation 30, and one setting ended on the edge of its allowed range.
- A weak starting point. The hand-tuned bot extracted in only a quarter of runs, so much of the gain was there to find.
Decisions and trade-offs
- Bots that play the real game through its input layerover a simplified simulator written for the botsA separate simulator answers questions about itself. Here the bot swaps in for the player's input device and the arena, spawns, Heat, weapons and extraction run unchanged, so a result is a result about the game.cost · Speed is set by one thread of GDScript per match, about 30× real time, and two determinism bugs had to be fixed in the game's own code.
- Bots that see only what is on screenover giving them the full game stateA bot that reads off-screen zombies or spawn timers would rate the game easier than any player could find it. Bots get the camera view, Charger lanes and the HUD's arrows, and a profile that needs more must say so per match.cost · Fair perception is code too, and it had a bug: for two studies every bot saw at most one zombie per grid cell.
- Every comparison paired on shared seedsover independent random samples per variantWith the same seeds for every variant, seed luck cancels and a difference belongs to the change. The reports use paired bootstrap intervals and sign-flip permutation tests.cost · Determinism has to hold everywhere: seeded streams per system, move directions quantized to 16, throw targets to whole pixels.
- An evolution strategy over the bot's 23 settingsover reinforcement learning with a neural networkThe bot already scores its moves with tunable weights. Searching those weights needs no new model, runs on the Python standard library, and the result reads as which settings moved and by how much.cost · It can only rebalance what the hand-written scoring already does; it can't invent a behaviour the bot has no term for.
- A rank-based mirrored evolution strategy (OpenAI-ES form)over a cross-entropy methodOn a synthetic test with match-like noise, the cross-entropy method collapsed its spread and closed 34% of the gap to the optimum; the ranked mirrored pairs closed 74%.cost · Fifteen candidates per generation, so every step costs 360 matches.
- Disjoint training, validation and test seed pools, with the test pool used onceover choosing the best variant on the seeds it was tuned onEarlier hand tuning picked a variant that extracted in 30% of runs on its tuning seeds and 17% on fresh ones. With a held-out test, validation predicted 553 salvage per match and the test gave 554.cost · 1,800 extra matches that teach the bot nothing.
- Eight seeds per candidate per generationover four, at half the computeMeasured before the run: with four seeds, a mirrored pair's better side matched a 30-seed answer 76% of the time; with eight, about 88%.cost · The run took 3 h 34 min instead of about half that.
Results
On 900 held-out seed pairs (300 seeds × three weapons), the trained bot banked 434 more salvage per match than the hand-tuned one (95% CI +394 to +477) and was better with every weapon. It got there mostly by staying past 06:00 for the score multiplier while its health held, and by dying early far less often: 411 of 900 runs died before 06:00, against 640.
- Salvage banked per match, hand-tuned → trained bot (900 held-out pairs)
- 120 → 554
- Extraction rate on the same pairs
- 25% → 48%
- Training run: 30 generations, 0 invalid matches
- 11,640 matches in 3 h 34 min
- Sustained throughput on a 14-core M4 Pro
- 99 matches/min
- Accelerated runs identical to real-time runs, digest for digest
- 52 / 52
- Bots play the real arena with every real system and touch only the controller: a stick, a throw, an interact button, and the real upgrade-menu buttons
- Same build, seed and bot give the same match: any match replays from its recorded inputs, checked against a state hash every second
- A batch runner with crash recovery, resume, thermal monitoring and a check that the player's save is never touched
- Standalone HTML reports with paired-seed statistics: bootstrap intervals, permutation tests and Holm correction
- Two studies (1,978 and 3,600 matches) and a training run with disjoint training, validation and test seed pools
What I learned
- A bot's result describes the bot. The biggest effect in the wall study wasn't the walls, it was a bug in what the bots could see.
- Keep a test set you only touch once. Hand tuning on one set of seeds once overstated a bot by 13 points; with a held-out test, validation and test agreed to one salvage.
- Measure the noise before spending the compute. The 76% versus 88% check set the seed budget before a single training match ran.