Files
firestorm/MW4COMPARE/tools/decompile/instance.py
T
83478b7666 Add MW4COMPARE: .mw4 decompiler toolchain and V4H comparison harness
Tooling built to recover editable source for six 'Mech chassis that exist
in the parallel FS_Build_V4H build but not in this repo. Reverse-engineers
every compiled record type in the .mw4 package format back to the .data /
.instance / .subsystems / .damage / .contents / .torso / .engine /
.armature sources the content pipeline consumes.

Nothing here is wired into the game build. It is a standalone analysis
harness run from Linux.

Package format
--------------
"#VBD" container. Directory records are [len][name][FILETIME][origSize]
[storedSize][offset], payload base at dword 0x0C. A record is stored raw
when storedSize == origSize, otherwise LZW (9->12-bit LSB-first codes,
256=clear, 257=EOF, dict from 258), per Database.cpp:451.

GameModel records are flat /Zp4 structs following the C++ inheritance
chain Entity(0) -> Mover(28) -> MWObject(80) -> Vehicle(664) -> Mech(756),
1636 bytes total. CreateMessage records follow Replicator -> Entity ->
Mover -> MWMover -> MWObject -> Vehicle -> Mech from start=16 (the
undeclared Connection__Message header), ending at 341 and padded to 344.

tools/decompile/
----------------
  datamap.py        header-driven layout engine; CHAIN + ANCHORS
                    {Vehicle:664, Mech:756} assert the struct offsets
  mw4msg.py         CreateMessage reader/walker
  data.py           .data      constants.py  define/table symbol resolution
  damage.py         .damage    contents.py   .contents
  smallmodel.py     .torso + .engine         instance.py  .instance
  armature.py / armature_parts.py  .armature + armaturedata/armaturevideo
  assembly.py       joint hierarchy renderer
  make_generic_doll.py  builds generic MFD/Radar damage dolls
  verify_*.py       per-type round-trip verifiers

Verified round-trip across all 64 shared chassis:
  .armature      2938/2976 pages     .subsystems  7579/7585 keys
  .data map      6071/6071 values    .data trip   8291/8306 keys
  .damage        6605/6605 keys      .contents    7480/7480 keys
  .torso+.engine 1280/1280 keys      .instance     896/896 keys, 64/64 pages
  armature_parts 1202/1202 .data, 1149/1202 .video

Layout-discovery lessons (documented in DECOMPILING.md)
-------------------------------------------------------
- Never let a field map be discovered by the values that verify it. A
  value-matching pass reported 4288/4288 while mis-assigning 34 keys. The
  map was rebuilt from header declaration order, anchored on uniquely
  resolved fields.
- Read the factory, not the data. 12 .data fields and 5 Torso fields are
  declared plain Stuff::Scalar but multiplied by Radians_Per_Degree in
  Mech_Tool.cpp:889 / Torso_Tool.cpp.
- Strip typedefs before walking a header. A stray `typedef int AttributeID;`
  masked a missing ClassID - two 4-byte errors cancelling out, caught only
  by the ANCHORS assertion.
- A verifier that silently narrows its own input reports success. Braced
  blocks must be hidden before splitting pages, replacing both CR and LF,
  because a `Shadow={...}` block contains a line reading `[shadow]` and
  splitlines() also splits on bare CR.
- NSWIZZLE is undefined, so the #else branch is live and orders members
  differently. bool is 1 byte; char x[MaxStringLength] is 256.
- V4H carries stale Mech IDs (their Atlas is 5, ours 6), so 64 of 65 shared
  chassis are off by one; --retarget-ids emits $(M_<Chassis>)/$(IDS_<Chassis>).

reports/ holds generated diffs. The two ~5 MB manifest-*.tsv intermediates
are gitignored; regenerate everything with run-comparison.sh.

Co-authored-by: Claude Opus 5 (Anthropic) <noreply@anthropic.com>
Co-authored-by: GitHub Copilot <copilot@github.com>
2026-08-08 16:47:57 -05:00

151 lines
5.7 KiB
Python

#!/usr/bin/env python3
"""
instance.py - decompiler for a mech `<chassis>.instance` file.
One page named after the chassis, holding the model/armature/subsystem/damage
references and the mechlab bar ratings. The record is a single
`Mech__CreateMessage`, so the layout comes from the CreateMessage chain rather
than the GameModel one:
Replicator -> Entity -> Mover -> MWMover -> MWObject -> Vehicle -> Mech
`datamap.chain_layout(..., start=16)` computes it -- 16 because the
`Connection__Message` header (messageLength, priority, flags) sits in front and
is not declared in any of these classes. The result ends at 341 and pads to
exactly the 344-byte record, and every offset established independently while
decoding `.armature` lands on the nose: classID 16, replicatorID 24,
localToParent 28, dataListID 84, alignment 88, jointName 152.
Two parser gaps had to be closed to get there, both fields typed with names the
member regex did not know: `Stuff::RegisteredClass::ClassID` / `ReplicatorID` in
the Replicator base, and `Entity__ExecutionStateEngine::FactoryRequest` /
`ObjectID` in Entity. Missing them silently shifted everything after offset 76
by 8 bytes while still producing a plausible-looking table.
python3 instance.py <chassis-record-dir> [-o out.instance]
python3 instance.py --verify
"""
import argparse, os, re, struct, subprocess, sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import datamap, mw4msg, subsystems
MANIFEST = "/home/rich/Repositories/FS_Ours_extracted/_manifest.tsv"
C = datamap.CODE
CHAIN = [
("Replicator__CreateMessage", C + "/mw4/Libraries/Adept/Replicator.hpp"),
("Entity__CreateMessage", C + "/mw4/Libraries/Adept/Entity.hpp"),
("Mover__CreateMessage", C + "/mw4/Libraries/Adept/Mover.hpp"),
("MWMover__CreateMessage", C + "/mw4/Code/MW4/MWMover.hpp"),
("MWObject__CreateMessage", C + "/mw4/Code/MW4/MWObject.hpp"),
("Vehicle__CreateMessage", C + "/mw4/Code/MW4/Vehicle.hpp"),
("Mech__CreateMessage", C + "/mw4/Code/MW4/Mech.hpp"),
]
CONNECTION_HEADER = 16
EXEC_STATE = {1: "NeverExecuteState", 2: "AlwaysExecuteState", 6: "ActiveState"}
# Entity.hpp:894
ALIGNMENT = {0: "DefaultAlignment", 1: "Player", 2: "Enemy",
3: "Team1", 4: "Team2", 5: "Team3", 6: "Team4"}
# source key -> message member, in the order 61 of 89 sources use
KEYS = [
("Model", "dataListID"),
("ExecutionState", "executionState"),
("Armature", "armatureStreamResourceID"),
("Subsystems", "subsystemStreamResourceID"),
("DamageObjects", "damageStreamResourceID"),
("Alignment", "alignment"),
("CurrentHeat", "currentHeat"),
("CurrentCoolant", "currentCoolant"),
("MaxCoolant", "maxCoolant"),
("DoesHaveInstanceName", "doesHaveInstanceName"),
("PowerRating", "m_powerBar"),
("ArmorRating", "m_armorBar"),
("SpeedRating", "m_speedBar"),
("HeatRating", "m_heatBar"),
]
def record(mech_dir):
for fn in os.listdir(mech_dir):
if fn.lower().endswith(".instance"):
return open(os.path.join(mech_dir, fn), "rb").read()
raise SystemExit(f"no .instance record in {mech_dir}")
def relative(path, chassis):
"""References are written relative to the mech folder."""
if not path:
return path
return re.sub(rf'^mechs[\\/]{re.escape(chassis or "")}[\\/]', '', path, flags=re.I)
def num(x):
return f"{int(x)}" if float(x) == int(x) else f"{x:g}"
def decompile(mech_dir, manifest=None):
"""-> (pageName, [(key, value)])"""
manifest = manifest if manifest is not None else subsystems.load_manifest(MANIFEST)
blob = record(mech_dir)
layout, _ = datamap.chain_layout(CHAIN, {}, start=CONNECTION_HEADER)
msgs = list(mw4msg.walk(blob, start=0))
if len(msgs) != 1:
raise SystemExit(f"expected one message in {mech_dir}, got {len(msgs)}")
msg = msgs[0][1]
name = datamap.read(msg, *layout["jointName"])
folder = os.path.basename(mech_dir.rstrip("/"))
kv = []
for key, member in KEYS:
off, typ, size = layout[member]
val = datamap.read(msg, off, typ, size)
if member == "executionState":
kv.append((key, EXEC_STATE.get(val, str(val))))
elif member == "alignment":
kv.append((key, ALIGNMENT.get(val, str(val))))
elif typ == "ResourceID":
path = manifest.get(val >> 16)
kv.append((key, relative(path, folder) if path else ""))
elif typ == "bool":
kv.append((key, "yes" if val else "no"))
elif typ in datamap.FLOATS:
kv.append((key, num(val)))
else:
kv.append((key, str(val)))
return name, kv
def emit(name, kv):
lines = [f"[{name}]"] + [f"{k}={v}" for k, v in kv]
return "\r\n".join(lines) + "\r\n"
def main():
ap = argparse.ArgumentParser()
ap.add_argument("mech_dir", nargs="?")
ap.add_argument("-o", "--output")
ap.add_argument("-m", "--manifest", default=MANIFEST)
ap.add_argument("--verify", action="store_true")
args = ap.parse_args()
if args.verify:
here = os.path.dirname(os.path.abspath(__file__))
raise SystemExit(subprocess.call([sys.executable,
os.path.join(here, "verify_instance.py")]))
if not args.mech_dir:
ap.error("mech_dir required")
name, kv = decompile(args.mech_dir, subsystems.load_manifest(args.manifest))
text = emit(name, kv)
if args.output:
with open(args.output, "wb") as fh:
fh.write(text.encode("latin-1"))
print(f"wrote {args.output}")
else:
sys.stdout.write(text)
if __name__ == "__main__":
main()