BT411/scratchpad/crash_20260726_2316_notes.md

Games-night crash, 2026-07-26 ~23:14-23:17 (build 4.11.584 / 2b0506d+)

Evidence preserved from the session transcript — BOTH operator_1.log and operator_relay.log truncated on restart before they could be copied. (Fix that: logs must rotate, not truncate.)

Timeline (relay, 23:08 session — log now lost)

• 23:08:28 session up, roster 8 seats
• 23:10:14 LAUNCH trim 6/8; ALL SIX READY in 8s (map=cavern — the artrucks fix verified live); RunMission pair 23:10:35/39; clock 600s
• 23:14:07 game[172.56.165.252 host=5 tag=10.99.0.4] dropped: closed (one player's client died mid-round)
• ~23:15 operator reports EVERYONE froze/crashed (mid-round, ~5 min early)
• 23:16:25 pods re-ACKing eggs (mass relaunch); operator's own pod dropped "mid-load" = the crash below; 23:17:33 Elengil dropped mid-load too
• 23:16:34 re-staged egg = 22 chunks (7 pilots incl. newcomer Draco)
• HELLO storms with stale hostIDs 8/9/10/12 -> REJOIN identity resyncs (the 8-seat template vs trimmed-6 remap; recovered as designed)

The captured crash (operator's pod, during the 23:16 RELOAD, not mid-round)

[cyl] table 'lok2' layers=7
[score] pilot roster built: 7 scoring pilot(s)
[crash] UNHANDLED EXCEPTION code=0xc0000005 addr=0x58990d (btl4+0x4990d) access=0 target=0x1d0
[crash] stack: btl4+0x4990d btl4+0x14caf btl4+0x16cc9 btl4+0x11f13 btl4+0xdc041
        btl4+0x9c0ce btl4+0x29e55 btl4+0x28556 btl4+0x98b76 btl4+0x98c5f
        btl4+0x121373 btl4+0xdb7d6 btl4+0xcc6a btl4+0x17e80c (ntdll...)

PDB-resolved (dist/BT411_4.11.584.pdb):

btl4+0x4990d  CreateReservoirSubsystem+0x13d     <- null->0x1d0 read
btl4+0x14caf  Mech::Mech+0x91f
btl4+0x16cc9  Mech::Make+0x29
btl4+0x11f13  BTL4Application::MakeViewpointEntity+0x63
btl4+0xdc041  Application::MakeAndLinkViewpointEntity+0x11

Operator's own mech = blkhawk (loaded fine 5x earlier tonight), so the poison is likely roster/egg STATE, not the operator's chassis.

The 23:19 egg (captured) — TWO anomalies

p1 vulture  Draco        <- first appearance tonight (new player)
p2 lok2     SAURON
p3 bhk1     Elengil      <- DUPLICATE
p4 sunder   RajelAran
p5 bhk1     Elengil      <- DUPLICATE (ghost seat!)
p6 thr1     Conn Man     <- first appearance tonight
p7 blkhawk  epilectrik

A DUPLICATE PILOT (Elengil on seats 3 AND 5) — formed during the post-crash chaos (she crashed 23:17:33 and rejoined; reclaim-by-identity missed under the ABORT churn). The operator asked earlier "is our seat code letting multiple claims happen?" — inverted form confirmed: one identity, two seats.

Hypotheses for the load crash (untested, in order)

1. DUPLICATE PILOT in the egg -> mission construction reads a half-claimed roster entry -> CreateReservoirSubsystem gets null spec (+0x1d0). Explains everyone crashing regardless of own mech.
2. A first-appearance chassis (vulture / thr1 / madcat-in-egg) with a broken reservoir spec in the RES.
3. The mid-round freeze (~23:15) is a SEPARATE bug: mass stall after one peer dropped mid-mission (reliable-channel wait on a dead host?) — 23:14:07 drop precedes the freeze by ~1 min. Unproven; logs lost.

TODO (post-night)

• Repro solo: 1-seat eggs per suspect chassis (vulture, thr1, madcat); then a crafted DUPLICATE-pilot egg. Convict via the same stack.
• Read CreateReservoirSubsystem in decomp/reconstruction: what lives at +0x1d0 of WHICH pointer; make the factory fail loud, not deref null.
• Seat code: reclaim-by-identity under ROUND-ABORT churn can double-seat one identity. Find the race, add a same-identity sweep before assign.
• Mid-mission peer-drop: what do surviving pods do? (mass freeze suspect)
• LOGS MUST ROTATE, NOT TRUNCATE (operator_1.log, operator_relay.log) — tonight lost the crash block and the whole 23:08 session to truncation.
• Matchlog HDR should include map= (had to dig the map out of egg diffs).

Round ~23:24-23:34 datapoints (clean full round, clock expiry, 4 matchlogs)

• vulture (Draco) and thr1 (Conn Man) LOADED AND PLAYED -> both CLEARED as reservoir-crash suspects -> duplicate-pilot egg hypothesis strengthened.
• Draco CLOSED HIS WINDOW mid-round: round stayed stable for everyone else -> a mid-round exit does not by itself cause the mass freeze; 23:15's trigger is still unknown (timing? who? state?).
• NEW BUG (identity swap): round-end reshuffle walk-up-assigned SAURON onto Draco's HELD seat (.6). Assign line stamped callsign='SAURON' but the egg kept the HELD pref -> SAURON's plasma showed 'Draco', and his score this round would credit 'Draco'. Fix: a walk-up assign to a DIFFERENT identity must overwrite the held pref (holds are for the SAME identity returning).

OWENS LASER CRASH (operator report at night's end, ~23:45)

• Player report: a user in OWENS crashed when firing lasers -- "again"; a previous (unfiled, verbal) report exists of the same thing. Operator believes it was Conn Man both times.
• CONFIRMED: the 23:14:07 mid-round drop (172.56.165.252) WAS Conn Man (IP -> callsign 4/4 tonight). So the likely chain: owens laser crash -> his client died 23:14:07 -> mass freeze ~23:15 -> relaunch storm -> duplicate-pilot egg -> reservoir load crash. One bug, three acts.
• He switched to thr1 immediately after (classic switch-away-from-crasher).
• Owens was in NO played round tonight until then (only the never-launched 22:49 artrucks staging) -- consistent with the old report.
• EVIDENCE TO COLLECT from Conn Man: content\join.log (has the [crash] stack; resolve vs dist/BT411_4.11.584.pdb) + his crash-round matchlog (content\matchlog_* stamped ~00:08-00:14 HIS clock; crash = never uploaded; final FIRE lines = the weapon at the moment of death).
• REPRO PATH: solo + BT_FORCE_MODEL=owens (btl4mssn.cpp), fire lasers. If deterministic -> local stack + real debugging -> faithful fix.

Conn Man detail (2026-07-27, via operator): TRUCKS vs OPPONENTS

Firing at TRUCKS (scenery): no crash. Firing at an OPPONENT: crash. Both spawn hit effects, so the delta is the PLAYER-HIT path on the shooter's machine: score/damage attribution (SCORE/FIRE messages), target tracking, or the victim REPLICANT's damage/death handling (a kill -> teardown windows = the 515 class). Q for Conn Man when he sends join.log: did it crash on a HIT or at the moment of a KILL?

Post-night fix session (2026-07-27, 00:00-00:25) -- status of each thread

• GUARDS LANDED + BUILD CLEAN: SortAndReloadNameBitmaps / LoadOrdinalBitmaps / LoadNameBitmaps (round-stop path) + the Reservoir ctor's linkedSinks.Resolve() (the captured crash site, now '[spawn] FATAL-AVOIDED' + inactive-copy degrade).
• STOP-PATH VERIFIED LIVE: offset-port rig (1520), full launch -> mission -> StopMission -> both pods relaunched clean through the guarded resort, 0 faults.
• DUP-PILOT REPRO: NEGATIVE. A minimal 2-pod egg with the ghost shape (same callsign+vehicle both seats) loaded and PLAYED. The 23:16 trigger therefore needs more than duplicate callsigns (7-pilot shape? abort-churn state? the tag-level ghost?) -- hypothesis stays OPEN; the guard defends the site regardless, and the FATAL-AVOIDED line self-identifies if it ever fires.
• Seat identity swap: FIXED (LEFT-grace window) + 7-check regression suite.
• Logs rotate (.1) instead of truncating -- both destroyers removed.
• Map dropdown = the authentic Mac-console catalog (8 maps); relay warns on phantom maps at egg load and names the map after 60s of 0/N ready.
• Gitea #35 updated twice (occurrence + trucks-vs-opponents discriminator). STILL OWED: Conn Man's join.log (the laser-crash stack); live games-night verification of tonight's five fixes.

#35 kill-storm repro attempts (2026-07-27 00:45-01:05) -- NEGATIVE, insurance armed

• Theory refined: the six-beam volley vs a target dying MID-VOLLEY (owens = the biggest simultaneous event burst; trucks never die/tear down; the 515 class in the beam-hit/interest-teardown neighborhood).
• BT_MP_FORCE_DMG parameterized (= sets per-tick damage) + rig_killstorm.ps1 (owens autofire vs a respawning victim). TWO 4-minute storms: the respawn cycle limits the harvest to 1-2 kill windows per run -- zero crashes, zero guard fires. Combined with July's 254-volley negative: the window needs a slow machine's timing (the Surface), not more volleys on this box.
• FIELD NET now armed on three layers: the crash self-report (584+), join.bat keeping join.old.log, and the sweep guards self-identifying. The next field occurrence CANNOT be lost.
• BEST NEXT MOVE: hand the reporter the storm rig -- his machine has the right timing profile. A one-bat package of rig_killstorm is feasible.