diff --git a/CLAUDE.md b/CLAUDE.md index 64da2b9..681c1c1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -48,6 +48,7 @@ precise than anything you can infer. 3. **RULE: no stand-ins.** The full logic IS in the pseudocode; a "gap" is an unfilled stub, not a hole. Never write placeholder logic — read the decomp. Bring-up scaffolding is marked + temporary. 4. **Verify honestly.** `static_assert`-lock layouts; run env-gated; read `content\_YYYYMMDD.log`; cdb on crashes. + **Verify against the FIELD COMPOSITION, not a constructed proxy** — benches + the verified bar: `context/test-harness.md`. Tag claims with the evidence tier; flag T3/T4. 5. **Persist insights.** A genuinely new finding → add it to the right `context/*.md` (+ the `docs/` ledger for detail), with an evidence tier. Keep the knowledge base current — this is a mandate, @@ -65,6 +66,7 @@ precise than anything you can infer. | BGF geometry, LODs, CONN/PCONN, ramps | `context/bgf-format.md` | | A layout/linkage/databinding BUG | `context/reconstruction-gotchas.md` | | The reconstruction method / workflow | `context/reconstruction-method.md` | +| **Running a BENCH / verifying a fix** — harness contract, 2-node pattern, what counts as VERIFIED | `context/test-harness.md` | | Walking, gait, ground model, collision | `context/locomotion.md` | | Subsystems, the factory, heat/weapons/power | `context/subsystems.md` | | Damage zones, targeting, firing, death | `context/combat-damage.md` | diff --git a/context/build-and-run.md b/context/build-and-run.md index 81ef08f..c9c575e 100644 --- a/context/build-and-run.md +++ b/context/build-and-run.md @@ -94,6 +94,10 @@ run\run.cmd [EGG] # default DEV.EGG; cd's to content\ and runs btl4.exe -eg ## Local benches MUST launch like a player (bench parity, 2026-07-30) +> Parity is one rule of the harness. The full harness contract, the two-node +> pattern, and the VERIFICATION DOCTRINE (field composition over constructed +> proxies) live in [[test-harness]] — read that before writing any bench. + Every shipped launcher (`play_solo.bat`, `join*.bat`, `play_steam.bat`) sets the SAME three things, and a local bench that omits them is not testing the game the field plays: diff --git a/context/reconstruction-method.md b/context/reconstruction-method.md index 00f74ce..f15ef83 100644 --- a/context/reconstruction-method.md +++ b/context/reconstruction-method.md @@ -58,4 +58,5 @@ the ground-model decode (10 agents), the alarm-unification (8), the gauge-widget ## Key Relationships - Bug classes: [[reconstruction-gotchas]] (check FIRST). Reference data: [[decomp-reference]]. +- Verification: [[test-harness]] — the bench contract + what counts as VERIFIED (field composition, not proxies). - Why it's needed: [[source-completeness]]. diff --git a/context/test-harness.md b/context/test-harness.md new file mode 100644 index 0000000..01d1762 --- /dev/null +++ b/context/test-harness.md @@ -0,0 +1,140 @@ +--- +id: test-harness +title: "The Test Harness — how benches run, and what counts as verified" +status: living +source_sections: "scratchpad/night6/bench_common.sh (the contract); build-and-run.md §bench-parity; nights 6-9 bench scripts; the #86/#110 verification failure" +related_topics: [build-and-run, reconstruction-method, multiplayer, reconstruction-gotchas, experience-levels] +key_terms: [bench, harness, field-composition, scalpel] +open_questions: [] +--- +# The Test Harness — how benches run, and what counts as verified + +Two subjects, deliberately in one file: the MACHINERY (how to launch and read a +bench) and the DOCTRINE (what a bench must show before a fix may be called +fixed). The doctrine exists because the machinery was used wrong more than once. + +## THE DOCTRINE: verify the FIELD COMPOSITION, not a constructed proxy + +**The archetype failure (#86 → #110).** "Weapons on damaged arms keep firing" +was fixed and verified by destroying the WEAPON directly (`BT_KILL_SUBSYS`). +The field scenario was the ARM ZONE dying while the weapon stayed healthy — a +path the bench never took. The claim "fixed" stood for two playtests while the +players kept hitting the bug; the actual defect (a stubbed cascade walk, +gotcha #26) sat exactly in the gap between the proxy and the field. **A fix +verified only against a reproduction you constructed is not verified.** + +Rules, in order of application: + +1. **Reproduce the reported scenario first** — the player's composition, not a + convenient adjacent one. If the report is "peer's arm shot off in MP", the + proof is a two-node run where PEER FIRE kills the arm (`mp_armgrind.sh`), + not a solo self-damage run. +2. **Scalpels are for isolation, not for verdicts.** Deterministic hooks + (`BT_SELF_DAMAGE_ZONE`, `BT_KILL_SUBSYS`, `BT_FORCE_SEEK`, `BT_ARMOR_FORCE`) + are the right way to LOCATE a defect. Before a scalpel result supports a + "fixed" claim, prove PATH IDENTITY to the field composition (e.g. the + self-damage harness dispatches a real `Entity::TakeDamageMessage` — the + same message + handler a peer round takes — so the scalpel shares every + line from the handler down; verified 2026-08-02, and then the field + composition was STILL run). +3. **If the symptom is multiplayer, the proof is two nodes.** Master-side + correctness says nothing about what a peer sees; replication is its own + layer (records, edge-detects, instance gates). #94, #84 and #110 each had a + peer-side half invisible to any solo bench. +4. **If the symptom is visual, the proof is pixels** — `BT_SHOT_EVERY` frames + diffed against a control region, not a log line saying the value changed + (gotcha #23: a colour change that logs perfectly and renders nothing). +5. **Coverage claims need a sweep, not an instance.** "Works for every weapon + type / chassis" = enumerate the axis and measure each point (all 3 fire + gates through a zone death; all 8 chassis' authored cascade flags — + `zonesweep.sh`). Per-chassis behaviour usually lives in AUTHORED DATA, so + fixed code on one chassis proves nothing about the others. +6. **Chase every anomaly in a passing run to ground.** The Owens rack cascade + also killing the sensor suite looked like a walk bug; it was authored + skeleton data (the mast is a child of the rack). A pass with an + unexplained extra effect is not yet a pass. + +## THE MACHINERY + +### The contract: `scratchpad/night6/bench_common.sh` +The single source of truth for launching a node THE WAY A PLAYER'S SHORTCUT +DOES — read the file, its comments carry the incident history. Summary: +- `bt_player_env` — `BT_PLATFORM=glass`, `BT_START_INSIDE=1` (cockpit, not + chase cam), `BT_DEV_GAUGES=1`. Copied verbatim from the shipped .bats; + `bt_assert_player_env` warns on drift. +- `bt_launch [args]` — one node, env-scoped, PID + tracked in `/tmp/bt_bench_pids.$$`. +- `bt_expert_egg` — shipped eggs are ALL `experience=expert`; a novice egg + silences heat/crits/jams and is only legal for the gimp bench (which must + say so out loud). See [[experience-levels]]. +- `bt_kill_ours` — teardown kills ONLY this run's PIDs (a blanket + `taskkill /IM` once shot down two live sessions). + +### Single-node bench skeleton +```bash +. /c/git/bt411/scratchpad/night6/bench_common.sh +cd /c/git/bt411/content || exit 1 +taskkill //F //IM btl4.exe > /dev/null 2>&1; sleep 2 # stale-node clear +sed "s/^map=.*/map=grass/; s/^time=.*/time=day/; 0,/^vehicle=.*/s//vehicle=$V/" MP.EGG > X.EGG +export BT_=... # diagnostics + scalpels +bt_launch x.log X.EGG 0x03 +sleep ; taskkill //F //IM btl4.exe; sleep 2 # then grep x.log +``` +The vehicle sed must replace the WHOLE line (`s/^vehicle=.*/`), or you mint +`vehicle=thr1bhk1` and the spawn FATALs. Chassis codes: ava1 bhk1 lok1/2 mad1/2 +own1 snd1 thr1 vul1 (own1 has NO arm zones — rack zones instead). + +### Two-node bench skeleton (the MP pattern) +```bash +bt_assert_player_env +bt_expert_egg MP.EGG X.EGG +BT_MP_LOG=1 bt_launch x_b.log X.EGG 0x0C -net 1601 # node B first +sleep 2 +BT_MP_LOG=1 bt_launch x_a.log X.EGG 0x03 -net 1501 # node A +sleep 5 +python ../tools/btconsole.py X.EGG 127.0.0.1:1501 127.0.0.1:1601 & # the relay STARTS the mission +sleep ; kill $relay; bt_kill_ours +``` +- A `-net` node renders NOTHING until `btconsole.py` (the headless relay) + starts the mission — a black window is pre-mission, not a hang. +- Affinity: TWO logical processors per node, disjoint sets (`0x03`/`0x0C`); + one LP starves the gauge executive ([[multiplayer]] peer-shakiness fix). +- Give the SHOOTER dense fire (`BT_AF_PERIOD=3` + `BT_AF_MISSILE=1`) and the + victim sparse (`=9`) when one side must lose a grind. Unthrottled autofire + trips the FailureHeat all-weapons brick and combat dies. +- Combat drive: `BT_GOTO=enemy BT_GOTO_STOP=` — 100 ends in a visual + ram scrum; 180 gives a standing exchange. +- Read each claim on the node that can see it: damage on the VICTIM's log, + replication on the OBSERVER's (`[zone-repl]`, `[mlrec]`, mirrored `DET`s). + +### Process hygiene (every one of these cost a session) +- **Stale-node taskkill FIRST.** A leftover node joins the next lobby as a + third instance and poisons the run. +- **Never double-background.** `run_in_background` + an inner `&` orphans the + script mid-startup: nodes launch, the script's own cleanup never runs. +- **Teardown kill order closes windows one at a time** — a "crashed" window at + the end of a timed run is usually the bench's own taskkill. +- **Stale exe:** `LNK1104` / a bench ignoring new diagnostics = the previous + process still holds `btl4.exe`, or the build silently didn't run — check the + `btl4.vcxproj ->` line printed, then rerun. +- Bench artifacts (`*.EGG` copies, `*_NNN.png`, bench logs) must be deleted + from `content/` before any dist cut; field logs are never committed. + +### Reading logs without fooling yourself +- **A capped/throttled diagnostic that is silent is NOT evidence of absence** + — the #99 `[seqrun]` shared counter "proved" a sequence never ran; it was + throttle starvation. Per-object throttles for per-object questions. +- **Alarm-line counters are not trend lines** — `free=0` printed AT the + failure is true by definition; the 30s census carries the trend (#32, three + separate times). +- **Log the actor's NAME at every refusal/decision** (`'PPC' fire REFUSED`); + anonymous counters (`FIRED #20`) cannot support attribution claims. +- Field logs have NO diagnostic gates set — a fix whose acceptance evidence + sits behind an env var is unverifiable in the field. Spawn-time summaries + (one line, ungated) answer questions retroactively; per-frame traces stay + gated. + +## Key Relationships +- Uses: [[build-and-run]] (parity, env gates, BT_SHOT capture) · [[experience-levels]] (expert vs novice gating) +- Informs: [[reconstruction-method]] (step 4 "verify honestly" — this file is the how) +- Incident sources: [[reconstruction-gotchas]] §23 (pixels), §26 (silent stubs); [[multiplayer]] (replication layers)