diff --git a/scratchpad/crash_20260726_2316_notes.md b/scratchpad/crash_20260726_2316_notes.md new file mode 100644 index 0000000..21e2660 --- /dev/null +++ b/scratchpad/crash_20260726_2316_notes.md @@ -0,0 +1,105 @@ +# Games-night crash, 2026-07-26 ~23:14-23:17 (build 4.11.584 / 2b0506d+) + +Evidence preserved from the session transcript — BOTH operator_1.log and +operator_relay.log truncated on restart before they could be copied. (Fix +that: logs must rotate, not truncate.) + +## Timeline (relay, 23:08 session — log now lost) +- 23:08:28 session up, roster 8 seats +- 23:10:14 LAUNCH trim 6/8; ALL SIX READY in 8s (map=cavern — the artrucks + fix verified live); RunMission pair 23:10:35/39; clock 600s +- 23:14:07 game[172.56.165.252 host=5 tag=10.99.0.4] dropped: closed (one + player's client died mid-round) +- ~23:15 operator reports EVERYONE froze/crashed (mid-round, ~5 min early) +- 23:16:25 pods re-ACKing eggs (mass relaunch); operator's own pod dropped + "mid-load" = the crash below; 23:17:33 Elengil dropped mid-load too +- 23:16:34 re-staged egg = 22 chunks (7 pilots incl. newcomer Draco) +- HELLO storms with stale hostIDs 8/9/10/12 -> REJOIN identity resyncs (the + 8-seat template vs trimmed-6 remap; recovered as designed) + +## The captured crash (operator's pod, during the 23:16 RELOAD, not mid-round) +``` +[cyl] table 'lok2' layers=7 +[score] pilot roster built: 7 scoring pilot(s) +[crash] UNHANDLED EXCEPTION code=0xc0000005 addr=0x58990d (btl4+0x4990d) access=0 target=0x1d0 +[crash] stack: btl4+0x4990d btl4+0x14caf btl4+0x16cc9 btl4+0x11f13 btl4+0xdc041 + btl4+0x9c0ce btl4+0x29e55 btl4+0x28556 btl4+0x98b76 btl4+0x98c5f + btl4+0x121373 btl4+0xdb7d6 btl4+0xcc6a btl4+0x17e80c (ntdll...) +``` +PDB-resolved (dist/BT411_4.11.584.pdb): +``` +btl4+0x4990d CreateReservoirSubsystem+0x13d <- null->0x1d0 read +btl4+0x14caf Mech::Mech+0x91f +btl4+0x16cc9 Mech::Make+0x29 +btl4+0x11f13 BTL4Application::MakeViewpointEntity+0x63 +btl4+0xdc041 Application::MakeAndLinkViewpointEntity+0x11 +``` +Operator's own mech = blkhawk (loaded fine 5x earlier tonight), so the poison +is likely roster/egg STATE, not the operator's chassis. + +## The 23:19 egg (captured) — TWO anomalies +``` +p1 vulture Draco <- first appearance tonight (new player) +p2 lok2 SAURON +p3 bhk1 Elengil <- DUPLICATE +p4 sunder RajelAran +p5 bhk1 Elengil <- DUPLICATE (ghost seat!) +p6 thr1 Conn Man <- first appearance tonight +p7 blkhawk epilectrik +``` +A DUPLICATE PILOT (Elengil on seats 3 AND 5) — formed during the post-crash +chaos (she crashed 23:17:33 and rejoined; reclaim-by-identity missed under the +ABORT churn). The operator asked earlier "is our seat code letting multiple +claims happen?" — inverted form confirmed: one identity, two seats. + +## Hypotheses for the load crash (untested, in order) +1. DUPLICATE PILOT in the egg -> mission construction reads a half-claimed + roster entry -> CreateReservoirSubsystem gets null spec (+0x1d0). Explains + everyone crashing regardless of own mech. +2. A first-appearance chassis (vulture / thr1 / madcat-in-egg) with a broken + reservoir spec in the RES. +3. The mid-round freeze (~23:15) is a SEPARATE bug: mass stall after one peer + dropped mid-mission (reliable-channel wait on a dead host?) — 23:14:07 drop + precedes the freeze by ~1 min. Unproven; logs lost. + +## TODO (post-night) +- Repro solo: 1-seat eggs per suspect chassis (vulture, thr1, madcat); then a + crafted DUPLICATE-pilot egg. Convict via the same stack. +- Read CreateReservoirSubsystem in decomp/reconstruction: what lives at +0x1d0 + of WHICH pointer; make the factory fail loud, not deref null. +- Seat code: reclaim-by-identity under ROUND-ABORT churn can double-seat one + identity. Find the race, add a same-identity sweep before assign. +- Mid-mission peer-drop: what do surviving pods do? (mass freeze suspect) +- LOGS MUST ROTATE, NOT TRUNCATE (operator_1.log, operator_relay.log) — tonight + lost the crash block and the whole 23:08 session to truncation. +- Matchlog HDR should include map= (had to dig the map out of egg diffs). + +## Round ~23:24-23:34 datapoints (clean full round, clock expiry, 4 matchlogs) +- vulture (Draco) and thr1 (Conn Man) LOADED AND PLAYED -> both CLEARED as + reservoir-crash suspects -> duplicate-pilot egg hypothesis strengthened. +- Draco CLOSED HIS WINDOW mid-round: round stayed stable for everyone else -> + a mid-round exit does not by itself cause the mass freeze; 23:15's trigger + is still unknown (timing? who? state?). +- NEW BUG (identity swap): round-end reshuffle walk-up-assigned SAURON onto + Draco's HELD seat (.6). Assign line stamped callsign='SAURON' but the egg + kept the HELD pref -> SAURON's plasma showed 'Draco', and his score this + round would credit 'Draco'. Fix: a walk-up assign to a DIFFERENT identity + must overwrite the held pref (holds are for the SAME identity returning). + +## OWENS LASER CRASH (operator report at night's end, ~23:45) +- Player report: a user in OWENS crashed when firing lasers -- "again"; a + previous (unfiled, verbal) report exists of the same thing. Operator + believes it was Conn Man both times. +- CONFIRMED: the 23:14:07 mid-round drop (172.56.165.252) WAS Conn Man (IP -> + callsign 4/4 tonight). So the likely chain: owens laser crash -> his client + died 23:14:07 -> mass freeze ~23:15 -> relaunch storm -> duplicate-pilot egg + -> reservoir load crash. One bug, three acts. +- He switched to thr1 immediately after (classic switch-away-from-crasher). +- Owens was in NO played round tonight until then (only the never-launched + 22:49 artrucks staging) -- consistent with the old report. +- EVIDENCE TO COLLECT from Conn Man: content\join.log (has the [crash] stack; + resolve vs dist/BT411_4.11.584.pdb) + his crash-round matchlog + (content\matchlog_* stamped ~00:08-00:14 HIS clock; crash = never uploaded; + final FIRE lines = the weapon at the moment of death). +- REPRO PATH: solo + BT_FORCE_MODEL=owens (btl4mssn.cpp), fire lasers. If + deterministic -> local stack + real debugging -> faithful fix.