Joe DiPrimaandClaude Fable 5
e69c0d7aa8
tools/gapcensus.py (deterministic): index-vs-.text interval math, export cross-check, function-start discovery inside dark regions (E8 call targets + data-section code pointers), pad exclusion, TU attribution, repo-citation flags. Output: reference/decomp/GAP_CENSUS.md + gap_census.tsv. Headline: .text 892KB, index covers 87.3%; 428 dark regions = 90KB REAL code (indexed-but-unexported = 0 -- the gap class is purely 'never indexed'). Game-side dark ~54KB: 66 regions visited by past digs, 159 NEVER TOUCHED. Validation: all six historically-bitten dark addresses (VehicleDead, ToggleLamp, death tail, master-perf, myomer integrator, duck consumer) land inside census regions; the two most-cited regions are the two that produced the most reconstructions. Top uncharted leads (spot-checked real code): the ~9KB l4splr|btmssn cluster (dispatch-table state machine -- likely BTMission's unexported heart); the 613B btplayer hole before the ctor (mission-review id-0x18 sender suspect); btl4app tails; heat|mechmppr + mechweap|btplayer boundaries. Full log: phases/phase-04-gap-census.md. Re-export half deferred (no local Ghidra; scripts ready). KB: source-completeness census section + CLAUDE.md lookup row + decomp-reference tools entry; consult the census BEFORE any 'absent from the export' claim (gotcha #20). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
55 lines
3.6 KiB
Markdown
55 lines
3.6 KiB
Markdown
# Phase 04 — The Export Gap Census (#60)
|
|
|
|
**Date:** 2026-08-06 · **Trigger:** export-gap blindness caused 4 wrong conclusions
|
|
(reconstruction-gotchas §20), 2 falsely [T1]. **Goal:** turn "we trip over gaps when we
|
|
arrive at them" into a known inventory.
|
|
|
|
## Method
|
|
`tools/gapcensus.py` (deterministic; re-run after any re-export):
|
|
1. **Index coverage** — `functions_index.tsv` (6,267 functions with sizes) interval-merged
|
|
against the PE `.text` extent → raw dark regions (bytes belonging to NO indexed function).
|
|
2. **Export coverage** — `/* @ADDR */` headers in `all/part_*.c` vs the index.
|
|
3. **Discovery** — function starts inside dark regions from E8 call targets + data-section
|
|
code pointers (vtables/handler tables/Performance pointers), with caller counts.
|
|
4. **Annotation** — pad-byte exclusion (CC/90/00), nearest TU attribution from `file=` tags,
|
|
and `visited` flags (address cited anywhere in game/ context/ docs/).
|
|
|
|
## Headline results
|
|
- `.text` = 892 KB. Index covers **87.3%**. Dark: **428 regions, 105 KB raw / 90 KB real code**.
|
|
- **Indexed-but-unexported = 0** — the entire gap class is "Ghidra never made it a function"
|
|
(consistent with #60's original find: VehicleDead was missing from the *index*, not filtered
|
|
from the export).
|
|
- 332 function starts discovered inside dark regions (call-graph + data-pointer evidence).
|
|
- **By TU family (real code):** munga/ 28.5 KB (engine — source in repo, LOW priority),
|
|
**bt/ 24.0 KB + bt_l4/ 14.9 KB + boundary 15.6 KB = ~54 KB of GAME-side dark**, munga_l4/ 9.5 KB.
|
|
- **Game-side dark regions: 66 already visited by past digs, 159 never touched.**
|
|
|
|
## Validation
|
|
All six known dark celebrities land inside census regions: the VehicleDead handler
|
|
(`0x4c05c4`), Searchlight ToggleLamp (`0x4b838c`), the TakeDamage death tail (`0x4a07b8`,
|
|
inside the famous `0x49ffc8-0x4a1232` stretch — 89 repo citations), the master-perf region
|
|
(`0x4a9b5a-0x4ab188` — 69 citations; crouch/myomers/posture all came from here), the myomer
|
|
integrator (`0x4b8be3`), the duck consumer (`0x4aa011`). The two most-cited dark regions are
|
|
exactly the two that produced the most reconstructions — the census finds the right things.
|
|
|
|
## Top uncharted leads (zero citations, real code, spot-checked)
|
|
1. **`0x480a2a-0x487ca0` cluster (~9 KB across 6 regions), l4splr|btmssn boundary** — head
|
|
spot-check @0x486467: real `enter 0x3c` function driven by a global-index dispatch table
|
|
(`[0x4fef88]` → table `0x501588`) — mission-state-machine flavored. Likely the un-exported
|
|
heart of **BTMission** (round lifecycle / mode carousel).
|
|
2. **`0x4c08fc-0x4c0bc8` (613 B), btplayer, immediately before the BTPlayer ctor** — real
|
|
function reading `player+0x24c`; prime suspect for the deferred **mission-review post
|
|
(id 0x18)** sender and/or death-cycle siblings.
|
|
3. **`0x4d41ae-0x4ddc7c` btl4app regions (~2.6 KB)** — app/session plumbing (attract/config?).
|
|
4. **`0x4af691-0x4af9cf` (707 B), heat|mechmppr boundary** — heat-family tail or mapper head.
|
|
5. **`0x4bdd7b-0x4be078` (673 B), mechweap|btplayer boundary** — weapon-family virtuals.
|
|
|
|
## Systemic fix status
|
|
- The inventory (`reference/decomp/GAP_CENSUS.md` + `gap_census.tsv`) is now the FIRST stop
|
|
before any "absent from the binary/export" claim — see gotcha §20's checklist.
|
|
- **Ghidra re-export** remains the second half: no local Ghidra install on this machine;
|
|
`reference/ghidra_scripts/` (ExportAll.java etc.) are ready when one is available (or on
|
|
Nick's machine). A re-export with current auto-analysis + forced disassembly of the 332
|
|
discovered starts would convert most of the 90 KB to pseudocode. Until then: raw disasm
|
|
per region, as practiced.
|