Files
firestorm/MW4COMPARE/tools/decompile/constants.py
T
83478b7666 Add MW4COMPARE: .mw4 decompiler toolchain and V4H comparison harness
Tooling built to recover editable source for six 'Mech chassis that exist
in the parallel FS_Build_V4H build but not in this repo. Reverse-engineers
every compiled record type in the .mw4 package format back to the .data /
.instance / .subsystems / .damage / .contents / .torso / .engine /
.armature sources the content pipeline consumes.

Nothing here is wired into the game build. It is a standalone analysis
harness run from Linux.

Package format
--------------
"#VBD" container. Directory records are [len][name][FILETIME][origSize]
[storedSize][offset], payload base at dword 0x0C. A record is stored raw
when storedSize == origSize, otherwise LZW (9->12-bit LSB-first codes,
256=clear, 257=EOF, dict from 258), per Database.cpp:451.

GameModel records are flat /Zp4 structs following the C++ inheritance
chain Entity(0) -> Mover(28) -> MWObject(80) -> Vehicle(664) -> Mech(756),
1636 bytes total. CreateMessage records follow Replicator -> Entity ->
Mover -> MWMover -> MWObject -> Vehicle -> Mech from start=16 (the
undeclared Connection__Message header), ending at 341 and padded to 344.

tools/decompile/
----------------
  datamap.py        header-driven layout engine; CHAIN + ANCHORS
                    {Vehicle:664, Mech:756} assert the struct offsets
  mw4msg.py         CreateMessage reader/walker
  data.py           .data      constants.py  define/table symbol resolution
  damage.py         .damage    contents.py   .contents
  smallmodel.py     .torso + .engine         instance.py  .instance
  armature.py / armature_parts.py  .armature + armaturedata/armaturevideo
  assembly.py       joint hierarchy renderer
  make_generic_doll.py  builds generic MFD/Radar damage dolls
  verify_*.py       per-type round-trip verifiers

Verified round-trip across all 64 shared chassis:
  .armature      2938/2976 pages     .subsystems  7579/7585 keys
  .data map      6071/6071 values    .data trip   8291/8306 keys
  .damage        6605/6605 keys      .contents    7480/7480 keys
  .torso+.engine 1280/1280 keys      .instance     896/896 keys, 64/64 pages
  armature_parts 1202/1202 .data, 1149/1202 .video

Layout-discovery lessons (documented in DECOMPILING.md)
-------------------------------------------------------
- Never let a field map be discovered by the values that verify it. A
  value-matching pass reported 4288/4288 while mis-assigning 34 keys. The
  map was rebuilt from header declaration order, anchored on uniquely
  resolved fields.
- Read the factory, not the data. 12 .data fields and 5 Torso fields are
  declared plain Stuff::Scalar but multiplied by Radians_Per_Degree in
  Mech_Tool.cpp:889 / Torso_Tool.cpp.
- Strip typedefs before walking a header. A stray `typedef int AttributeID;`
  masked a missing ClassID - two 4-byte errors cancelling out, caught only
  by the ANCHORS assertion.
- A verifier that silently narrows its own input reports success. Braced
  blocks must be hidden before splitting pages, replacing both CR and LF,
  because a `Shadow={...}` block contains a line reading `[shadow]` and
  splitlines() also splits on bare CR.
- NSWIZZLE is undefined, so the #else branch is live and orders members
  differently. bool is 1 byte; char x[MaxStringLength] is 256.
- V4H carries stale Mech IDs (their Atlas is 5, ours 6), so 64 of 65 shared
  chassis are off by one; --retarget-ids emits $(M_<Chassis>)/$(IDS_<Chassis>).

reports/ holds generated diffs. The two ~5 MB manifest-*.tsv intermediates
are gitignored; regenerate everything with run-comparison.sh.

Co-authored-by: Claude Opus 5 (Anthropic) <noreply@anthropic.com>
Co-authored-by: GitHub Copilot <copilot@github.com>
2026-08-08 16:47:57 -05:00

85 lines
3.2 KiB
Python

#!/usr/bin/env python3
"""
constants.py - reverse tables for the `.data` keys authored as symbolic names.
Four `[GameData]` keys are written in the source as a symbol rather than a
literal, and the record stores only the resolved integer. Emitting a faithful
`.data` therefore needs the int -> symbol direction:
MechID $(M_Annihilator) MechLabHeaders.h #define M_Annihilator 0
TechType $(Tech_IS) MechLabHeaders.h #define Tech_IS 0
NameIndex $(IDS_Annihilator) MissionLang.defines #define IDS_ANNIHILATOR 576
MoveTypeFlag LEGJUMPMOVETYPE MWObject.hpp enum + MWObject_Tool.cpp:20
MoveTypeFlag is the odd one out: a bare token, not `$(...)`, matched by the
`stricmp` chain in `MWObject__GameModel::ConvertStringToMoveType`. Its values
come from the anonymous enum at MWObject.hpp:136, where LEG is 0 and the order
is NOT the same as the stricmp chain, so the enum is the authority.
The `.defines` and `.h` files spell symbols in mixed case while sources
reference them in any case, so lookups here are case-insensitive.
"""
import re
REPO = "/home/rich/Repositories/firestorm/Gameleap"
MECHLAB_HEADERS = REPO + "/mw4/Content/ShellScripts/MechLabHeaders.h"
MISSIONLANG_DEFINES = REPO + "/mw4/Content/Defines/MissionLang.defines"
# MWObject.hpp:136. Declaration order is the value order.
MOVE_TYPE = [
"LEGMOVETYPE", "LEGJUMPMOVETYPE", "TRACKMOVETYPE", "WHEELMOVETYPE",
"FLYERMOVETYPE", "HOVERMOVETYPE", "HELIMOVETYPE", "NONEMOVETYPE",
"DROPSHIPMOVETYPE", "WATERMOVETYPE",
]
_DEFINE = re.compile(r'^\s*#define\s+(\w+)\s+(-?\d+)\s*$', re.M)
def defines(path, prefix):
"""-> {value: [symbols]} for every `#define <prefix>NAME <int>` in a file.
Values are not unique: IDS_FIRSTSKIN and IDS_WOLFHOUND are both 501, so a
single winner cannot be picked here without knowing which mech is being
emitted.
"""
txt = open(path, encoding="latin-1", errors="replace").read()
out = {}
for name, val in _DEFINE.findall(txt):
if name.lower().startswith(prefix.lower()):
out.setdefault(int(val), []).append(name)
return out
def tables():
"""-> {sourceKey: {intValue: [sourceToken]}} for the four symbolic keys."""
wrap = lambda d: {v: [f"$({s})" for s in syms] for v, syms in d.items()}
return {
"MechID": wrap(defines(MECHLAB_HEADERS, "M_")),
"TechType": wrap(defines(MECHLAB_HEADERS, "Tech_")),
"NameIndex": wrap(defines(MISSIONLANG_DEFINES, "IDS_")),
"MoveTypeFlag": {i: [s] for i, s in enumerate(MOVE_TYPE)},
}
def symbol(tbl, key, value, hint=None):
"""Pick the source token for a value, preferring one naming the chassis."""
syms = tbl.get(key, {}).get(value)
if not syms:
return None
if hint and len(syms) > 1:
h = norm(hint)
for s in syms:
if h and h in norm(s):
return s
return syms[0]
def norm(tok):
"""Compare symbols ignoring case and $() wrapping."""
return re.sub(r'[^a-z0-9_]', '', tok.lower())
if __name__ == "__main__":
for key, tbl in tables().items():
print(f"{key:14s} {len(tbl):5d} values e.g. {list(tbl.items())[:2]}")