BT411/docs/GHOST_MECH_ANALYSIS.md

# The GHOST MECH / zombie-wreck failure — field analysis 2026-07-29

**Status:** mechanism established [T1/T2]; the missing link (why the drop-zone handshake fails) is
still open. Field session: 2026-07-29 night, Steam MP, 5 players, build 4.11.642. Raw logs and the
verbatim agent findings are in `scratchpad/night6/` (uncommitted — they contain machine names and
Steam identities). Tracker: #81 (zombie wreck), #57 (deathPending latch), #45 (tally replication).

## The one-sentence version
A death whose **drop-zone reply never arrives** leaves the respawn cycle stranded, so the mech is
never reset and never repainted — and because the render-side wreck swap is **one-way**, the pilot
keeps driving a burning hulk that sinks out of the world after ~18 s and then cannot be drawn at
all. The "wreckage moving and shooting" report and the "invisible ghost" report are **the same
failure at two different ages**.

## Established from the field logs [T1]
- Exactly **8 death cycles** in the whole session. Every one has the same 3-line signature at
  consecutive lines: `death cycle START` → `*** DESTROYED ***` → `WARNING: death ... SWALLOWED`.
- **Only 2 of 8 reached `RESET at drop zone`.** A stranded cycle predicts ghosting **perfectly**:
  8/8 cycles, 10/10 player-matches, in both directions.
- Match 2: all four non-host players died, all four stranded, zero RESETs anywhere = "everybody
  ghosted". **The host did not ghost because he never died** — his log holds no death cycle at all.
  (There is nothing special about being host; an earlier reading of this was wrong.)
- Match 1: three deaths, two stranded, one completed. The completion is the control case, and it is
  the "self-corrected for one player" the testers reported.
- ⚠ **The SWALLOWED warning is benign.** It fires on 8 of 8 deaths, including both clean
  completions and a build-641 death, so its base rate is 100% and its correlation with ghosting is
  zero. It is not even a second death — it is the same death re-entering the notify from inside the
  cycle. Do not treat it as a signal (an earlier reading did).
- ⚠ **Player IDs are per-match**: node = lobby token octet + 1, host always `.1`. `player 3:1` is a
  different human in every match. Any cross-machine identification must decode this first.
- ⚠ An earlier "three spurious RESETs on the host" finding was an **aggregation artifact** — one
  log file spans ~18 sessions and each session legitimately logs one spawn RESET.

## The render mechanism [T1, code]
- `btl4vid.cpp:1255-1300` — the wreck sink is a **pure timer**, nothing about movement: 0.25 s
  reveal, then `sink = -0.025·t²`, and at `-8.0` the hulk/debris/flames get `SetDrawObj(NULL)` plus
  `[BTrender] wreck buried (sink complete)`. That threshold is reached at **t ≈ 18 s**.
- The wreck objects hang off the mech's own render tree as an offset, so **they travel with the
  pilot** — hence a moving, burning, shooting wreck for the first ~18 s.
- `mech4.cpp:1827` — a healthy respawn calls `BTRebuildMechModel` on Reset precisely because "the
  wreck swap is one-way on the render side". No reply → no Reset → no rebuild → the hulk sinks and
  the player is invisible but still simulated and driveable.
- Peers apply the wreck swap correctly and in a bit-identical order across machines, so the ghost is
  **not** an entity peers never learned about — it is an entity whose presentation is frozen dead
  and then hidden.

## The stall itself [T0 WinTesla — NOT proven authentic]
`engine/MUNGA/DROPZONE.cpp`: a slot is granted only if `IsAvailable()` — busy for `DOWN_TIME = 5 s`
after use, and any other player's vehicle within 2 m marks it busy. **When no slot is free the
handler reposts the message to itself every 0.1 s at `MaxEventPriority` and never replies**, so the
requester's `deathPending` latch is never cleared.
⚠ **Provenance:** `engine/MUNGA` is **WinTesla**, the later Windows port of MUNGA taken from updated
Red Planet (which converted the Division hardware to software). **The 1995 game never ran it.** So
this behaviour is authoritative for what we compile, *not* for what the pods did. BT's own
`munga/dropzone.cpp` is **absent from the decomp export** — the 1995 assignment policy is an
unrecovered gap. Recovering it (byte-scan + windowed disasm, the technique that recovered
`FUN_004a6344`) is the open task.

## 2026-07-30 dig: the respawn handshake, walked end to end
Re-read the whole chain (death → +5 s re-post → engine hunt → DropZone grant → reply → Reset) and
**measured** the parts that were assumptions. Results, in order of importance:

**1. The engine hunt RETRIES EVERY 2 SECONDS, forever, while the mech is dead** [T0]
(`PLAYER.cpp:325-327` posts the message back to itself unconditionally; its only state-based exit is
`GetSimulationState() == DropZoneAcquiredState`, which never happens — see 3). The DropZone also has
a resend net: a repeat request from the same requester + same `deathCount` re-grants the **same**
slot even while it is busy (`DROPZONE.cpp:182-194`). **So "a transient no-slot" and "one lost reply"
are both ELIMINATED as causes** — the request keeps being re-sent and re-granted every 2 s. A
permanent strand needs a permanent reason.

**2. THE SILENT DISCARD — the one remaining unlit path, now instrumented** [T1]
`BTPlayer::DropZoneReplyMessageHandler` is `if (!playerVehicle) … else if (deathCount ==
message->deathCount) … else { return; }`. That final branch was a **bare return with no logging**:
the drop zone granted a spot and replied, the numbers disagreed, and the reply was dropped —
`deathPending` stays latched, the mech is never `Reset`, and the pilot is a permanent ghost **with
zero trace in the log**. That is exactly why 6 of 8 field cycles stranded leaving no evidence. Now
always-on (`[ghost] DROP-ZONE REPLY DISCARDED …`) and it prints the **direction** of the mismatch,
which decides the fix: `msgDeath < ours` = genuinely stale (dropping is correct); `msgDeath > ours` =
**our counter is behind and we threw away a live respawn** — which points straight at #45 (the death
tally does not replicate correctly). Deliberately NOT "recovered" yet: guessing the direction would
be a stand-in, and the wrong guess respawns a mech that is still alive.

**3. ⚠ LANDMINE — do not convert the two `Set_Alarm_Level` raw writes to `SetSimulationState()`.**
`Set_Alarm_Level` is an **empty stub** (`btstubs.cpp:87`), so the death path's
`Set_Alarm_Level(this+0x2c, 1)` and the reply path's `(this+0x2c, 2)` are **no-ops** today. Their
values decode perfectly against `Player`'s enum (`PLAYER.h:273-279`) as `DropZoneAcquiredState`(1)
and `VehicleTranslocatedState`(2) — `DropZoneAcquiredState = Entity::StateCount`, and `Entity` adds
no states, so it is **1**. That makes "obviously these should be `SetSimulationState` calls" a very
attractive and **catastrophic** fix: the engine hunt is gated on `GetSimulationState() !=
DropZoneAcquiredState`, so setting state 1 on death would stop `AssignDropZone` from ever being
dispatched and make **every** pilot ghost permanently.
Measured: our `BTPlayer::simulationState` is at **0x24**, not 0x2c, and the write leaves it at 0
(`[ghost] death raw-write(+0x2c,1): simState 0 -> 0 (hunt gate still open)`). In the binary `+0x2c`
is the **Simulation-base alarm** — the same field the mech side calls `graphicAlarm` (`@0x4ac126`:
"owner alarm+0x2C -> level 9"). Our layout models that only as a `Mech` member, so a `BTPlayer` has
nowhere to put it; the stub is harmless until we recover what reads a *Player's* alarm in the binary
(likely a cockpit/HUD respawn indicator). [T1 offsets, T3 purpose]

**Revised theory of the ghost:** not drop-zone starvation (see below) and not a lost reply (see 1),
but a **`deathCount` mismatch between the player and the reply**, silently discarding a live
respawn — MP-specific in a way solo structurally cannot reproduce (solo respawn works every time;
6 of 8 MP cycles stranded), and directly coupled to the known #45 replication defect.

## What the instrumentation already killed
`slots=8` (measured, `[dz] POOL`). An 8-slot pool **cannot** be cooldown-starved by 5 players (at
most 5 slots on cooldown at once), so the shared `dropzone=one` in `tools/eggmodel.py:42` is **not**
the bottleneck. Suspicion moves to the handshake: a lost reply, or a `deathCount` mismatch — the
engine gates the hunt on `message->deathCount == deathCount` and the DropZone's resend safety net is
keyed on `lastDeathCount == message->deathCount`, so one mismatch breaks both silently (cf. #45).

## Instrumentation shipped (`ba6756c`) — always-on, no env var needed
| line | meaning |
|---|---|
| `[dz] POOL name= slots= downTime= proximityBlock=` | once per mission: the pool size |
| `[dz] GRANTED slot N to entity E death#D waited=Xs` | one per respawn; `waited` is the headline; tags `(GHOST RECOVERED)` past 15 s |
| `[dz] STALL` / `[dz] GHOST LIKELY` | escalating, rate-limited per waiter; names **why** each slot is busy (age + last user) — this discriminates cooldown saturation from proximity blocking |
| `[dzreq] player P asking ... try=N msgDeath= ourDeathCount=` | requester half; flags `*** MISMATCH ***` |
`BT_DROPZONE_LOG=1` adds a verbose per-request slot dump for bench work.

## Next actions, in order
1. **Recover BT's 1995 drop-zone policy** from `content/BTL4OPT.EXE` (decomp gap). The key question:
   when no slot was available, did BT also repost forever and never reply? If yes, the stall is
   authentic and our divergence is elsewhere; if no, implement what BT did.
2. **Read the next field logs' `[dz]`/`[dzreq]` lines** — they answer cooldown-vs-proximity and
   lost-reply-vs-deathCount-mismatch directly.
3. Ask the testers one question that discriminates a second defect: **did a flaming mover stay
   visible for more than ~20 s?** If yes, the sink tick is not running for that entity (updater
   early-return, or a render-tree rebuild resetting `wreckAge`) — a second bug on top of this one.
4. Separately filed/needed: one machine **stopped processing the effect/death-transition stream**
   57 % into a match and never recovered (0 explosions / 0 wreck swaps / 0 burials while the other
   four logged 4/4/4) — that machine was playing a stale world, and it is why that player reported
   the whole "desync cluster".

## The desync cluster is four separate things, not one bug [T2]
- **Throttle reads zero** — local, and present in the *good* sessions too. ⚠ Unresolved nuance: the
  internal `thr` field is 0 in every field `[drive]` sample while mechs moved, but reads 0.8 in solo
  benches; whether that is the same thing the player saw on his gauge is **not** established.
- **No weapon audio** — local audio-source starvation caused by the **map's ambient emitters**
  (`arena2_morning` ≈ 10.6 effects/s × ~2.5 voices pins the 256-source pool); thousands of
  `ACQUIRE FAILED ... pool is exhausted`. Unrelated to crits or the network.
- **Weapons fire without damage** — replication staleness. The shooter resolves damage and awards
  its own score locally, so a shooter always *sees* hits even when the victim applies nothing.
- **Other mechs lagging/misplaced** — the same stale-world condition as item 4 above.

## Why tonight and not before
Nothing in 642 broke multiplayer. Critical hits landing for real (first time) made deaths far more
frequent, and every death is a chance for this stall — so a rare race became a common one. Fewer
players ⇒ fewer deaths ⇒ clean matches, which is exactly what the testers observed.