From 51dff6bd5aeeb12d1f016b353f634e554e05c397 Mon Sep 17 00:00:00 2001 From: Joe DiPrima Date: Thu, 6 Aug 2026 11:12:20 -0500 Subject: [PATCH] handoff: #60 closed (census + re-export) Co-Authored-By: Claude Fable 5 --- scratchpad/SESSION_HANDOFF_20260803.md | 15 ++++++++++++++ scratchpad/night12/census60.py | 16 +++++++++++++++ scratchpad/night12/close60.py | 27 ++++++++++++++++++++++++++ 3 files changed, 58 insertions(+) create mode 100644 scratchpad/night12/census60.py create mode 100644 scratchpad/night12/close60.py diff --git a/scratchpad/SESSION_HANDOFF_20260803.md b/scratchpad/SESSION_HANDOFF_20260803.md index e7ad1db..60c749f 100644 --- a/scratchpad/SESSION_HANDOFF_20260803.md +++ b/scratchpad/SESSION_HANDOFF_20260803.md @@ -466,3 +466,18 @@ touch is a different region from the fog work). Merged --no-ff, symbols confirme (4.11.801 boots, fog resolves, no exceptions). Zip verified; notes: dist/RELEASE_NOTES_4.11.801_discord.md (adds the Cyd item). #138 can be closed when field-confirmed tonight. + +## 2026-08-06: #60 CLOSED -- gap census + RE-EXPORT (e69c0d7, f44be87) +Installed Ghidra 12.1.2 + JDK 21 at %LOCALAPPDATA%\bt411-tools (8.3 SHORT paths required in the +runner -- Ghidra's .bat expands %JAVA_HOME% unquoted, profile has a space). ExportGaps.java = +old output contract + gap-fill (E8 targets / data->code ptrs / census starts, fixpoint). +RESULT: coverage 87.3->93.5%, dark code 90.4->40.8KB, functions 6267->6472 (+205, 0 failures). +Every historically-dark celebrity now has pseudocode incl. @4c05c4 VehicleDead (the #60 opener). +The re-export CONFIRMED the crouch hand-reconstruction field-for-field AND revealed a missed +branch (airborne auto-rise, mode 3|4 && legState 1 -> forced rise) -- implemented + re-benched. +PROMOTED to canonical reference/decomp/; old export kept at reference/decomp/archive_2025export/ +(⚠ addresses stable, part/LINE citations are NOT -- cite @ADDR). +NEW LEAD: @0x4c0904 = the MASTER BTPlayer Performance (team resolve, EndMission console post, +score heartbeat); our @0x4c083c PlayerSimulation attribution needs re-checking. +Pipeline: tools/ghidra_reexport.sh -> tools/gapcensus.py -> tools/gapdiff.py. +NEXT: polish list (killed-by console line, burning wrecks, marker beacon) + field logs from 801. diff --git a/scratchpad/night12/census60.py b/scratchpad/night12/census60.py new file mode 100644 index 0000000..9bb6a8b --- /dev/null +++ b/scratchpad/night12/census60.py @@ -0,0 +1,16 @@ +import sys +sys.path.insert(0, r'C:\git\bt411\scratchpad\night7') +import gitea + +gitea.comment(60, """**THE GAP CENSUS IS DONE** (e69c0d7): `reference/decomp/GAP_CENSUS.md` + `gap_census.tsv`, regenerated any time by `tools/gapcensus.py`. + +Headline: .text is 892 KB; the index covers 87.3%; **428 dark regions = 90 KB of real code Ghidra never made into functions**. Indexed-but-unexported = 0, so this issue's original finding generalizes: the whole gap class is "missing from the index," exactly like VehicleDead was. Game-side (bt/ + bt_l4/) dark is ~54 KB — 66 regions already visited by past raw-disasm digs, **159 never touched by anything**. + +Validation: all six addresses that historically burned us (VehicleDead @4c05c4, Searchlight ToggleLamp @4b838c, the TakeDamage death tail, the master-perf region, the myomer integrator, the duck consumer) land inside census regions, and the two most-cited regions are the two that produced the most reconstructions. + +Top uncharted leads (spot-checked as real code): the ~9 KB l4splr|btmssn cluster (a dispatch-table state machine — likely BTMission's unexported heart), and a 613-byte btplayer hole right before the ctor (prime suspect for the deferred mission-review id-0x18 sender). Full lead list: `phases/phase-04-gap-census.md`. + +Process change (reconstruction-gotchas #20): the census is now the FIRST stop before any "absent from the export" claim — export silence over a dark region proves nothing without a byte-scan. + +Remaining half of this issue: a Ghidra re-export forcing disassembly at the 332 discovered function starts (scripts ready in `reference/ghidra_scripts/`; no Ghidra install on this box — candidate for Nick's machine). Leaving open for that half.""") +print('posted') diff --git a/scratchpad/night12/close60.py b/scratchpad/night12/close60.py new file mode 100644 index 0000000..f116811 --- /dev/null +++ b/scratchpad/night12/close60.py @@ -0,0 +1,27 @@ +import sys +sys.path.insert(0, r'C:\git\bt411\scratchpad\night7') +import gitea + +gitea.close(60, """**BOTH HALVES DONE — closing** (census e69c0d7, re-export f44be87). + +Installed Ghidra 12.1.2 + JDK 21 locally and rebuilt the export with a new `ExportGaps.java` that keeps the old output contract exactly (shard format, index columns — every tool, grep and `@ADDR` citation still works) and adds a gap-fill pass first: force disassembly + `createFunction` at E8 call targets landing outside any function, data→code pointers at a plausible prologue, and the 332 starts the census discovered — iterated to a fixpoint. + +| | before | after | +|---|---|---| +| index coverage of .text | 87.3% | **93.5%** | +| dark REAL code | 90.4 KB | **40.8 KB** (−54.8%) | +| game-side dark (bt/ + bt_l4/) | 53.1 KB | **21.1 KB** | +| functions / pseudocode | 6,267 | **6,472** (+205, 0 decompile failures) | + +**`BTPlayer::VehicleDeadMessageHandler @0x4c05c4` — the function whose absence opened this issue — now has pseudocode**, along with every other address that historically burned us (ToggleLamp @4b838c, the TakeDamage death tail, the master-perf region, the myomer integrator, the btplayer hole @4c0904). + +Two things worth flagging beyond the numbers: +1. **The re-export independently CONFIRMED this week's hand-decoded crouch reconstruction, field for field** (mapPosture/duckState/squatCapable/myomerEffectiveness, the novice gate, SetLegAnimation(2/3), ForceUpdate(8)+(1), the stability alarm). Good news for every raw-disasm reconstruction we've shipped. +2. **It also caught a branch the raw pass missed** — an airborne auto-rise (`mode 3|4 && legState 1 → forced rise`) that the hand disassembly stopped just short of. Implemented and re-benched the same day. That is exactly the failure mode this issue was about, now caught by tooling instead of by a player. + +**Process:** `reference/decomp/GAP_CENSUS.md` maps what is still dark; per reconstruction-gotchas §20 the rule is now cheap — look the address up, and only if it is in a dark region does "absent from the export" need byte-scan backup. + +⚠ **Note for anyone citing the decomp:** the old export is preserved at `reference/decomp/archive_2025export/`. Addresses are stable across both, but line numbers and shard membership are NOT — cite `@ADDR`, not `part_0NN.c:LINE`. + +Pipeline for future re-exports: `tools/ghidra_reexport.sh` → `tools/gapcensus.py` → `tools/gapdiff.py`. Remaining 40.8 KB dark (mostly engine TUs we compile from source) plus the new `@0x4c0904` master-Performance lead are tracked in `phases/phase-04-gap-census.md`.""") +print('closed')