handoff: #60 closed (census + re-export)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Joe DiPrima
2026-08-06 11:12:20 -05:00
co-authored by Claude Fable 5
parent f44be87ab2
commit 51dff6bd5a
3 changed files with 58 additions and 0 deletions
+15
View File
@@ -466,3 +466,18 @@ touch is a different region from the fog work). Merged --no-ff, symbols confirme
(4.11.801 boots, fog resolves, no exceptions). Zip verified; notes:
dist/RELEASE_NOTES_4.11.801_discord.md (adds the Cyd item). #138 can be closed when
field-confirmed tonight.
## 2026-08-06: #60 CLOSED -- gap census + RE-EXPORT (e69c0d7, f44be87)
Installed Ghidra 12.1.2 + JDK 21 at %LOCALAPPDATA%\bt411-tools (8.3 SHORT paths required in the
runner -- Ghidra's .bat expands %JAVA_HOME% unquoted, profile has a space). ExportGaps.java =
old output contract + gap-fill (E8 targets / data->code ptrs / census starts, fixpoint).
RESULT: coverage 87.3->93.5%, dark code 90.4->40.8KB, functions 6267->6472 (+205, 0 failures).
Every historically-dark celebrity now has pseudocode incl. @4c05c4 VehicleDead (the #60 opener).
The re-export CONFIRMED the crouch hand-reconstruction field-for-field AND revealed a missed
branch (airborne auto-rise, mode 3|4 && legState 1 -> forced rise) -- implemented + re-benched.
PROMOTED to canonical reference/decomp/; old export kept at reference/decomp/archive_2025export/
(⚠ addresses stable, part/LINE citations are NOT -- cite @ADDR).
NEW LEAD: @0x4c0904 = the MASTER BTPlayer Performance (team resolve, EndMission console post,
score heartbeat); our @0x4c083c PlayerSimulation attribution needs re-checking.
Pipeline: tools/ghidra_reexport.sh -> tools/gapcensus.py -> tools/gapdiff.py.
NEXT: polish list (killed-by console line, burning wrecks, marker beacon) + field logs from 801.
+16
View File
@@ -0,0 +1,16 @@
import sys
sys.path.insert(0, r'C:\git\bt411\scratchpad\night7')
import gitea
gitea.comment(60, """**THE GAP CENSUS IS DONE** (e69c0d7): `reference/decomp/GAP_CENSUS.md` + `gap_census.tsv`, regenerated any time by `tools/gapcensus.py`.
Headline: .text is 892 KB; the index covers 87.3%; **428 dark regions = 90 KB of real code Ghidra never made into functions**. Indexed-but-unexported = 0, so this issue's original finding generalizes: the whole gap class is "missing from the index," exactly like VehicleDead was. Game-side (bt/ + bt_l4/) dark is ~54 KB — 66 regions already visited by past raw-disasm digs, **159 never touched by anything**.
Validation: all six addresses that historically burned us (VehicleDead @4c05c4, Searchlight ToggleLamp @4b838c, the TakeDamage death tail, the master-perf region, the myomer integrator, the duck consumer) land inside census regions, and the two most-cited regions are the two that produced the most reconstructions.
Top uncharted leads (spot-checked as real code): the ~9 KB l4splr|btmssn cluster (a dispatch-table state machine — likely BTMission's unexported heart), and a 613-byte btplayer hole right before the ctor (prime suspect for the deferred mission-review id-0x18 sender). Full lead list: `phases/phase-04-gap-census.md`.
Process change (reconstruction-gotchas #20): the census is now the FIRST stop before any "absent from the export" claim — export silence over a dark region proves nothing without a byte-scan.
Remaining half of this issue: a Ghidra re-export forcing disassembly at the 332 discovered function starts (scripts ready in `reference/ghidra_scripts/`; no Ghidra install on this box — candidate for Nick's machine). Leaving open for that half.""")
print('posted')
+27
View File
@@ -0,0 +1,27 @@
import sys
sys.path.insert(0, r'C:\git\bt411\scratchpad\night7')
import gitea
gitea.close(60, """**BOTH HALVES DONE — closing** (census e69c0d7, re-export f44be87).
Installed Ghidra 12.1.2 + JDK 21 locally and rebuilt the export with a new `ExportGaps.java` that keeps the old output contract exactly (shard format, index columns — every tool, grep and `@ADDR` citation still works) and adds a gap-fill pass first: force disassembly + `createFunction` at E8 call targets landing outside any function, data→code pointers at a plausible prologue, and the 332 starts the census discovered — iterated to a fixpoint.
| | before | after |
|---|---|---|
| index coverage of .text | 87.3% | **93.5%** |
| dark REAL code | 90.4 KB | **40.8 KB** (54.8%) |
| game-side dark (bt/ + bt_l4/) | 53.1 KB | **21.1 KB** |
| functions / pseudocode | 6,267 | **6,472** (+205, 0 decompile failures) |
**`BTPlayer::VehicleDeadMessageHandler @0x4c05c4` — the function whose absence opened this issue — now has pseudocode**, along with every other address that historically burned us (ToggleLamp @4b838c, the TakeDamage death tail, the master-perf region, the myomer integrator, the btplayer hole @4c0904).
Two things worth flagging beyond the numbers:
1. **The re-export independently CONFIRMED this week's hand-decoded crouch reconstruction, field for field** (mapPosture/duckState/squatCapable/myomerEffectiveness, the novice gate, SetLegAnimation(2/3), ForceUpdate(8)+(1), the stability alarm). Good news for every raw-disasm reconstruction we've shipped.
2. **It also caught a branch the raw pass missed** — an airborne auto-rise (`mode 3|4 && legState 1 → forced rise`) that the hand disassembly stopped just short of. Implemented and re-benched the same day. That is exactly the failure mode this issue was about, now caught by tooling instead of by a player.
**Process:** `reference/decomp/GAP_CENSUS.md` maps what is still dark; per reconstruction-gotchas §20 the rule is now cheap — look the address up, and only if it is in a dark region does "absent from the export" need byte-scan backup.
⚠ **Note for anyone citing the decomp:** the old export is preserved at `reference/decomp/archive_2025export/`. Addresses are stable across both, but line numbers and shard membership are NOT — cite `@ADDR`, not `part_0NN.c:LINE`.
Pipeline for future re-exports: `tools/ghidra_reexport.sh` → `tools/gapcensus.py` → `tools/gapdiff.py`. Remaining 40.8 KB dark (mostly engine TUs we compile from source) plus the new `@0x4c0904` master-Performance lead are tracked in `phases/phase-04-gap-census.md`.""")
print('closed')