Tooling built to recover editable source for six 'Mech chassis that exist
in the parallel FS_Build_V4H build but not in this repo. Reverse-engineers
every compiled record type in the .mw4 package format back to the .data /
.instance / .subsystems / .damage / .contents / .torso / .engine /
.armature sources the content pipeline consumes.
Nothing here is wired into the game build. It is a standalone analysis
harness run from Linux.
Package format
--------------
"#VBD" container. Directory records are [len][name][FILETIME][origSize]
[storedSize][offset], payload base at dword 0x0C. A record is stored raw
when storedSize == origSize, otherwise LZW (9->12-bit LSB-first codes,
256=clear, 257=EOF, dict from 258), per Database.cpp:451.
GameModel records are flat /Zp4 structs following the C++ inheritance
chain Entity(0) -> Mover(28) -> MWObject(80) -> Vehicle(664) -> Mech(756),
1636 bytes total. CreateMessage records follow Replicator -> Entity ->
Mover -> MWMover -> MWObject -> Vehicle -> Mech from start=16 (the
undeclared Connection__Message header), ending at 341 and padded to 344.
tools/decompile/
----------------
datamap.py header-driven layout engine; CHAIN + ANCHORS
{Vehicle:664, Mech:756} assert the struct offsets
mw4msg.py CreateMessage reader/walker
data.py .data constants.py define/table symbol resolution
damage.py .damage contents.py .contents
smallmodel.py .torso + .engine instance.py .instance
armature.py / armature_parts.py .armature + armaturedata/armaturevideo
assembly.py joint hierarchy renderer
make_generic_doll.py builds generic MFD/Radar damage dolls
verify_*.py per-type round-trip verifiers
Verified round-trip across all 64 shared chassis:
.armature 2938/2976 pages .subsystems 7579/7585 keys
.data map 6071/6071 values .data trip 8291/8306 keys
.damage 6605/6605 keys .contents 7480/7480 keys
.torso+.engine 1280/1280 keys .instance 896/896 keys, 64/64 pages
armature_parts 1202/1202 .data, 1149/1202 .video
Layout-discovery lessons (documented in DECOMPILING.md)
-------------------------------------------------------
- Never let a field map be discovered by the values that verify it. A
value-matching pass reported 4288/4288 while mis-assigning 34 keys. The
map was rebuilt from header declaration order, anchored on uniquely
resolved fields.
- Read the factory, not the data. 12 .data fields and 5 Torso fields are
declared plain Stuff::Scalar but multiplied by Radians_Per_Degree in
Mech_Tool.cpp:889 / Torso_Tool.cpp.
- Strip typedefs before walking a header. A stray `typedef int AttributeID;`
masked a missing ClassID - two 4-byte errors cancelling out, caught only
by the ANCHORS assertion.
- A verifier that silently narrows its own input reports success. Braced
blocks must be hidden before splitting pages, replacing both CR and LF,
because a `Shadow={...}` block contains a line reading `[shadow]` and
splitlines() also splits on bare CR.
- NSWIZZLE is undefined, so the #else branch is live and orders members
differently. bool is 1 byte; char x[MaxStringLength] is 256.
- V4H carries stale Mech IDs (their Atlas is 5, ours 6), so 64 of 65 shared
chassis are off by one; --retarget-ids emits $(M_<Chassis>)/$(IDS_<Chassis>).
reports/ holds generated diffs. The two ~5 MB manifest-*.tsv intermediates
are gitignored; regenerate everything with run-comparison.sh.
Co-authored-by: Claude Opus 5 (Anthropic) <noreply@anthropic.com>
Co-authored-by: GitHub Copilot <copilot@github.com>
151 lines
5.7 KiB
Python
151 lines
5.7 KiB
Python
#!/usr/bin/env python3
|
|
"""
|
|
instance.py - decompiler for a mech `<chassis>.instance` file.
|
|
|
|
One page named after the chassis, holding the model/armature/subsystem/damage
|
|
references and the mechlab bar ratings. The record is a single
|
|
`Mech__CreateMessage`, so the layout comes from the CreateMessage chain rather
|
|
than the GameModel one:
|
|
|
|
Replicator -> Entity -> Mover -> MWMover -> MWObject -> Vehicle -> Mech
|
|
|
|
`datamap.chain_layout(..., start=16)` computes it -- 16 because the
|
|
`Connection__Message` header (messageLength, priority, flags) sits in front and
|
|
is not declared in any of these classes. The result ends at 341 and pads to
|
|
exactly the 344-byte record, and every offset established independently while
|
|
decoding `.armature` lands on the nose: classID 16, replicatorID 24,
|
|
localToParent 28, dataListID 84, alignment 88, jointName 152.
|
|
|
|
Two parser gaps had to be closed to get there, both fields typed with names the
|
|
member regex did not know: `Stuff::RegisteredClass::ClassID` / `ReplicatorID` in
|
|
the Replicator base, and `Entity__ExecutionStateEngine::FactoryRequest` /
|
|
`ObjectID` in Entity. Missing them silently shifted everything after offset 76
|
|
by 8 bytes while still producing a plausible-looking table.
|
|
|
|
python3 instance.py <chassis-record-dir> [-o out.instance]
|
|
python3 instance.py --verify
|
|
"""
|
|
import argparse, os, re, struct, subprocess, sys
|
|
|
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
|
import datamap, mw4msg, subsystems
|
|
|
|
MANIFEST = "/home/rich/Repositories/FS_Ours_extracted/_manifest.tsv"
|
|
C = datamap.CODE
|
|
CHAIN = [
|
|
("Replicator__CreateMessage", C + "/mw4/Libraries/Adept/Replicator.hpp"),
|
|
("Entity__CreateMessage", C + "/mw4/Libraries/Adept/Entity.hpp"),
|
|
("Mover__CreateMessage", C + "/mw4/Libraries/Adept/Mover.hpp"),
|
|
("MWMover__CreateMessage", C + "/mw4/Code/MW4/MWMover.hpp"),
|
|
("MWObject__CreateMessage", C + "/mw4/Code/MW4/MWObject.hpp"),
|
|
("Vehicle__CreateMessage", C + "/mw4/Code/MW4/Vehicle.hpp"),
|
|
("Mech__CreateMessage", C + "/mw4/Code/MW4/Mech.hpp"),
|
|
]
|
|
CONNECTION_HEADER = 16
|
|
|
|
EXEC_STATE = {1: "NeverExecuteState", 2: "AlwaysExecuteState", 6: "ActiveState"}
|
|
# Entity.hpp:894
|
|
ALIGNMENT = {0: "DefaultAlignment", 1: "Player", 2: "Enemy",
|
|
3: "Team1", 4: "Team2", 5: "Team3", 6: "Team4"}
|
|
|
|
# source key -> message member, in the order 61 of 89 sources use
|
|
KEYS = [
|
|
("Model", "dataListID"),
|
|
("ExecutionState", "executionState"),
|
|
("Armature", "armatureStreamResourceID"),
|
|
("Subsystems", "subsystemStreamResourceID"),
|
|
("DamageObjects", "damageStreamResourceID"),
|
|
("Alignment", "alignment"),
|
|
("CurrentHeat", "currentHeat"),
|
|
("CurrentCoolant", "currentCoolant"),
|
|
("MaxCoolant", "maxCoolant"),
|
|
("DoesHaveInstanceName", "doesHaveInstanceName"),
|
|
("PowerRating", "m_powerBar"),
|
|
("ArmorRating", "m_armorBar"),
|
|
("SpeedRating", "m_speedBar"),
|
|
("HeatRating", "m_heatBar"),
|
|
]
|
|
|
|
|
|
def record(mech_dir):
|
|
for fn in os.listdir(mech_dir):
|
|
if fn.lower().endswith(".instance"):
|
|
return open(os.path.join(mech_dir, fn), "rb").read()
|
|
raise SystemExit(f"no .instance record in {mech_dir}")
|
|
|
|
|
|
def relative(path, chassis):
|
|
"""References are written relative to the mech folder."""
|
|
if not path:
|
|
return path
|
|
return re.sub(rf'^mechs[\\/]{re.escape(chassis or "")}[\\/]', '', path, flags=re.I)
|
|
|
|
|
|
def num(x):
|
|
return f"{int(x)}" if float(x) == int(x) else f"{x:g}"
|
|
|
|
|
|
def decompile(mech_dir, manifest=None):
|
|
"""-> (pageName, [(key, value)])"""
|
|
manifest = manifest if manifest is not None else subsystems.load_manifest(MANIFEST)
|
|
blob = record(mech_dir)
|
|
layout, _ = datamap.chain_layout(CHAIN, {}, start=CONNECTION_HEADER)
|
|
|
|
msgs = list(mw4msg.walk(blob, start=0))
|
|
if len(msgs) != 1:
|
|
raise SystemExit(f"expected one message in {mech_dir}, got {len(msgs)}")
|
|
msg = msgs[0][1]
|
|
|
|
name = datamap.read(msg, *layout["jointName"])
|
|
folder = os.path.basename(mech_dir.rstrip("/"))
|
|
kv = []
|
|
for key, member in KEYS:
|
|
off, typ, size = layout[member]
|
|
val = datamap.read(msg, off, typ, size)
|
|
if member == "executionState":
|
|
kv.append((key, EXEC_STATE.get(val, str(val))))
|
|
elif member == "alignment":
|
|
kv.append((key, ALIGNMENT.get(val, str(val))))
|
|
elif typ == "ResourceID":
|
|
path = manifest.get(val >> 16)
|
|
kv.append((key, relative(path, folder) if path else ""))
|
|
elif typ == "bool":
|
|
kv.append((key, "yes" if val else "no"))
|
|
elif typ in datamap.FLOATS:
|
|
kv.append((key, num(val)))
|
|
else:
|
|
kv.append((key, str(val)))
|
|
return name, kv
|
|
|
|
|
|
def emit(name, kv):
|
|
lines = [f"[{name}]"] + [f"{k}={v}" for k, v in kv]
|
|
return "\r\n".join(lines) + "\r\n"
|
|
|
|
|
|
def main():
|
|
ap = argparse.ArgumentParser()
|
|
ap.add_argument("mech_dir", nargs="?")
|
|
ap.add_argument("-o", "--output")
|
|
ap.add_argument("-m", "--manifest", default=MANIFEST)
|
|
ap.add_argument("--verify", action="store_true")
|
|
args = ap.parse_args()
|
|
if args.verify:
|
|
here = os.path.dirname(os.path.abspath(__file__))
|
|
raise SystemExit(subprocess.call([sys.executable,
|
|
os.path.join(here, "verify_instance.py")]))
|
|
if not args.mech_dir:
|
|
ap.error("mech_dir required")
|
|
name, kv = decompile(args.mech_dir, subsystems.load_manifest(args.manifest))
|
|
text = emit(name, kv)
|
|
if args.output:
|
|
with open(args.output, "wb") as fh:
|
|
fh.write(text.encode("latin-1"))
|
|
print(f"wrote {args.output}")
|
|
else:
|
|
sys.stdout.write(text)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|