83478b7666
Tooling built to recover editable source for six 'Mech chassis that exist
in the parallel FS_Build_V4H build but not in this repo. Reverse-engineers
every compiled record type in the .mw4 package format back to the .data /
.instance / .subsystems / .damage / .contents / .torso / .engine /
.armature sources the content pipeline consumes.
Nothing here is wired into the game build. It is a standalone analysis
harness run from Linux.
Package format
--------------
"#VBD" container. Directory records are [len][name][FILETIME][origSize]
[storedSize][offset], payload base at dword 0x0C. A record is stored raw
when storedSize == origSize, otherwise LZW (9->12-bit LSB-first codes,
256=clear, 257=EOF, dict from 258), per Database.cpp:451.
GameModel records are flat /Zp4 structs following the C++ inheritance
chain Entity(0) -> Mover(28) -> MWObject(80) -> Vehicle(664) -> Mech(756),
1636 bytes total. CreateMessage records follow Replicator -> Entity ->
Mover -> MWMover -> MWObject -> Vehicle -> Mech from start=16 (the
undeclared Connection__Message header), ending at 341 and padded to 344.
tools/decompile/
----------------
datamap.py header-driven layout engine; CHAIN + ANCHORS
{Vehicle:664, Mech:756} assert the struct offsets
mw4msg.py CreateMessage reader/walker
data.py .data constants.py define/table symbol resolution
damage.py .damage contents.py .contents
smallmodel.py .torso + .engine instance.py .instance
armature.py / armature_parts.py .armature + armaturedata/armaturevideo
assembly.py joint hierarchy renderer
make_generic_doll.py builds generic MFD/Radar damage dolls
verify_*.py per-type round-trip verifiers
Verified round-trip across all 64 shared chassis:
.armature 2938/2976 pages .subsystems 7579/7585 keys
.data map 6071/6071 values .data trip 8291/8306 keys
.damage 6605/6605 keys .contents 7480/7480 keys
.torso+.engine 1280/1280 keys .instance 896/896 keys, 64/64 pages
armature_parts 1202/1202 .data, 1149/1202 .video
Layout-discovery lessons (documented in DECOMPILING.md)
-------------------------------------------------------
- Never let a field map be discovered by the values that verify it. A
value-matching pass reported 4288/4288 while mis-assigning 34 keys. The
map was rebuilt from header declaration order, anchored on uniquely
resolved fields.
- Read the factory, not the data. 12 .data fields and 5 Torso fields are
declared plain Stuff::Scalar but multiplied by Radians_Per_Degree in
Mech_Tool.cpp:889 / Torso_Tool.cpp.
- Strip typedefs before walking a header. A stray `typedef int AttributeID;`
masked a missing ClassID - two 4-byte errors cancelling out, caught only
by the ANCHORS assertion.
- A verifier that silently narrows its own input reports success. Braced
blocks must be hidden before splitting pages, replacing both CR and LF,
because a `Shadow={...}` block contains a line reading `[shadow]` and
splitlines() also splits on bare CR.
- NSWIZZLE is undefined, so the #else branch is live and orders members
differently. bool is 1 byte; char x[MaxStringLength] is 256.
- V4H carries stale Mech IDs (their Atlas is 5, ours 6), so 64 of 65 shared
chassis are off by one; --retarget-ids emits $(M_<Chassis>)/$(IDS_<Chassis>).
reports/ holds generated diffs. The two ~5 MB manifest-*.tsv intermediates
are gitignored; regenerate everything with run-comparison.sh.
Co-authored-by: Claude Opus 5 (Anthropic) <noreply@anthropic.com>
Co-authored-by: GitHub Copilot <copilot@github.com>
71 lines
2.8 KiB
Bash
Executable File
71 lines
2.8 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# run-comparison.sh - regenerate every report in MW4COMPARE/reports/.
|
|
#
|
|
# ./run-comparison.sh [V4H_ROOT] [OURS_ROOT]
|
|
#
|
|
# Defaults:
|
|
# V4H_ROOT = /home/rich/Repositories/FS_Build_V4H (deployed 3rd-party build)
|
|
# OURS_ROOT = <repo>/Gameleap/mw4 (our dev/source tree)
|
|
#
|
|
# Takes a few minutes; the decoded props/textures comparisons dominate.
|
|
set -euo pipefail
|
|
|
|
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
T="$HERE/tools"
|
|
R="$HERE/reports"
|
|
V4H="${1:-/home/rich/Repositories/FS_Build_V4H}"
|
|
OURS="${2:-$(cd "$HERE/.." && pwd)/Gameleap/mw4}"
|
|
|
|
mkdir -p "$R"
|
|
echo "V4H = $V4H"
|
|
echo "OURS = $OURS"
|
|
|
|
echo "[1/8] package manifests"
|
|
python3 "$T/mw4index.py" "$V4H/resource" > "$R/manifest-v4h.tsv" 2> "$R/manifest-v4h.err"
|
|
python3 "$T/mw4index.py" "$OURS/Resource" > "$R/manifest-ours.tsv" 2> "$R/manifest-ours.err"
|
|
|
|
echo "[2/8] package + entry name diff"
|
|
python3 "$T/diffindex.py" "$R/manifest-v4h.tsv" "$R/manifest-ours.tsv" V4H OURS \
|
|
> "$R/packages-entrydiff.txt"
|
|
|
|
echo "[3/8] core.mw4 decoded diff"
|
|
python3 "$T/pkgcmp.py" "$V4H/resource/core.mw4" "$OURS/Resource/core.mw4" \
|
|
> "$R/core-decoded-diff.txt"
|
|
|
|
echo "[4/8] props.mw4 decoded diff (~1 min)"
|
|
python3 "$T/pkgcmp.py" "$V4H/resource/props.mw4" "$OURS/Resource/props.mw4" \
|
|
> "$R/props-decoded-diff.txt"
|
|
|
|
echo "[5/8] textures.mw4 decoded diff (~2 min)"
|
|
python3 "$T/pkgcmp.py" "$V4H/resource/textures.mw4" "$OURS/Resource/textures.mw4" \
|
|
> "$R/textures-decoded-diff.txt"
|
|
|
|
echo "[6/8] per-mission and per-map decoded diffs"
|
|
: > "$R/missions-decoded-diff.txt"
|
|
for f in "$V4H"/resource/Missions/*.mw4; do
|
|
b="$(basename "$f")"; o="$OURS/Resource/Missions/$b"
|
|
[ -f "$o" ] || continue
|
|
{ echo "### $b"; python3 "$T/pkgcmp.py" "$f" "$o"; } >> "$R/missions-decoded-diff.txt"
|
|
done
|
|
: > "$R/maps-decoded-diff.txt"
|
|
for f in "$V4H"/resource/maps/*.mw4; do
|
|
b="$(basename "$f")"; o="$OURS/Resource/maps/$b"
|
|
[ -f "$o" ] || continue
|
|
{ echo "### $b"; python3 "$T/pkgcmp.py" "$f" "$o"; } >> "$R/maps-decoded-diff.txt"
|
|
done
|
|
|
|
echo "[7/8] loose file trees"
|
|
python3 "$T/treediff.py" "$V4H/hsh" "$OURS/hsh" V4H OURS > "$R/hsh-diff.txt"
|
|
python3 "$T/treediff.py" "$V4H/hshoriginal" "$OURS/hsh" V4HORIG OURS > "$R/hshoriginal-diff.txt"
|
|
python3 "$T/treediff.py" "$V4H/resource" "$OURS/Resource" V4H OURS > "$R/resource-files-diff.txt"
|
|
# V4H ships only 14 loose Content files against our ~46k-file source tree, so the
|
|
# "-OURS" side is pure noise here; keep only the header, additions and conflicts.
|
|
python3 "$T/treediff.py" "$V4H/content" "$OURS/Content" V4H OURS \
|
|
| grep -v '^-OURS' > "$R/content-diff.txt"
|
|
|
|
echo "[8/8] variant inventory"
|
|
python3 "$T/variants-report.py" "$V4H/resource/Variants" "$R/manifest-v4h.tsv" "$R/manifest-ours.tsv" \
|
|
> "$R/variants-inventory.txt"
|
|
|
|
echo "done -> $R"
|