diff --git a/MECHSPECSHEET/README.md b/MECHSPECSHEET/README.md new file mode 100644 index 00000000..d33216f8 --- /dev/null +++ b/MECHSPECSHEET/README.md @@ -0,0 +1,231 @@ +# MECHSPECSHEET + +Generates a stock-mech specification spreadsheet directly from the FireStorm +source tree, so the numbers come from the data the game actually loads instead +of being transcribed by hand. + + python3 generate-mech-specs.py # writes mech-specifications-5.1.0b.xlsx + python3 verify-mech-specs.py # checks the result, prints PASS/ISSUES + +Both scripts are standard library only. An `.xlsx` is a zip of XML, and only +`xl/worksheets/sheet1.xml` is rewritten, so the template's fonts, column widths +and drawings survive untouched. No `openpyxl` needed. + +## Contents + +| File | What it is | +|---|---| +| `generate-mech-specs.py` | The extractor. Reads the roster from `core.build`, parses each chassis, writes the workbook. | +| `verify-mech-specs.py` | Five independent checks on a generated workbook. | +| `template/mech-spec-template.xlsx` | The original 5.0.7D workbook, kept as the layout template. Its single hand-made Battlemaster row is the calibration reference. | +| `mech-specifications-5.1.0b.xlsx` | Current output: 65 stock chassis, 55 columns. | + +## Current output + +65 chassis (37 Inner Sphere, 28 Clan; 23 Assault, 17 Heavy, 13 Medium, 12 Light). +Every non-weapon column is populated for every chassis. + +The roster is whatever `Content/core.build` registers, via lines of the form +`instance=mechs\\.instance`. The six staged V4H chassis +(Champion, Dasher, Griffin, Jenner IIC, Marauder, Thunderbolt) are **not** +included because they are not registered yet; they will appear automatically +once they are, with no change to this tool. + +## Column reference + +Columns 1-24 and 51-55 come from the template. Columns 25-50 are the weapon +grid, widened from the original (see *Layout changes*). + +| # | Column | Source | +|---|---|---| +| 1 | Mech Chassis | Portrait filename in `hsh/Mechs/` (these are the localized display names), title-cased; falls back to the folder name | +| 2 | Is_Playable | `y` for everything in `core.build` | +| 3 | Release Version | Constant `SOURCE_VERSION` in the script (currently `5.1.0b`) | +| 4 | Technology | `.data` `TechType` -> IS / Clan | +| 5 | Weight (tons) | `.data` `MaxVehicleTonnage` | +| 6 | Class | Derived from tonnage: <=35 Light, <=55 Medium, <=75 Heavy, else Assault | +| 7 | External Armor Type | `.subsystems` Armor block `ArmorType` | +| 8 | Internal Structure Type | `.subsystems` Armor block `InternalType` | +| 9 | Armor Points | **Installed**: sum of the Armor block's per-zone tons x points-per-ton (see *Armour*) | +| 10 | JumpJets | JumpJetSubsystem present in `.subsystems` | +| 11 | Light Amplification | `.instance` `DoesHaveLightAmp` (default 1) | +| 12-15 | ECM / AMS / LAMS / Beagle | Corresponding subsystem present in `.subsystems` | +| 16 | # of Flushes | `.instance` `MaxCoolant` / 5 | +| 17 | Heat Sinks | Count of heatsink subsystems, doubled if they are DoubleHeatSink | +| 18 | Top Speed Normal (kph) | `min(MinMaxSpeed + MPSPerUpgrade * EngineUpgrades, MaxSpeed) * 3.6` | +| 19 | Top Speed Gimped (kph) | `.data` `MaxGimpSpeed * 3.6` | +| 20 | Acceleration | `.data` `Acceleration` | +| 21 | Deceleration | `.data` `Decceleration` (double-c is canonical in the engine) | +| 22 | Turn Rate (rad/sec) | `.data` `TopSpeedTurnRate` (degrees) x pi/180 | +| 23 | Twist Range (degrees) | `.torso` `TwistRadius` x 2, macros resolved through `MechTorso.defines` | +| 24 | Twist Speed | `.torso` `TwistSpeed`, same macro resolution | +| 25-50 | Weapon grid | `.subsystems` weapon blocks, placed by `InternalLocation` | +| 51-54 | Rating bars | `.instance` `PowerRating` / `ArmorRating` / `SpeedRating` / `HeatRating` | +| 55 | (template note) | Left blank | + +Weapon cells read `Weapon Name (ammo)`, with ` (Rear)` appended when the block +sets `WeaponFacing=1`. + +## Data sources + +All under `Gameleap/mw4/Content/`: + +| File | Provides | +|---|---| +| `core.build` | The roster. Presence here is what makes a chassis playable. | +| `Mechs//.data` | Tonnage, tech, heat, movement, `CanLoad*` flags | +| `Mechs//.instance` | Rating bars, coolant, LightAmp fitted | +| `Mechs//.subsystems` | **The stock loadout**: armour type and tonnage, heat sinks, engine upgrades, weapons, electronics | +| `Mechs//.engine` | `MPSPerUpgrade` for the speed calculation | +| `Mechs//.torso` | Twist range and speed (usually macro references) | +| `Subsystems/Armor.data` | Points per ton by armour type | +| `Defines/MechTorso.defines` | Torso macro values | +| `../hsh/Mechs/*.bmp` | Display names | + +`.damage` is deliberately **not** used for armour. See below. + +## Armour: why tons, not multipliers + +The Armor block in `.subsystems` looks like per-zone multipliers: + + ArmorType=Standard + InternalType=EndoSteel + LeftArm=1.45 + CenterFrontTorso=2.05 + +They are **tons of armour**, not multipliers. The proof is in +`MechLab.cpp` `GetCurrentMechArmorData`: + + tonnage_array[i] = armor_array[i] / *points_per; + +so `points = tons x points_per`, with the rate chosen by armour type from +`Content/Subsystems/Armor.data`: + + PointsPerStandardTon=32 PointsPerFerroTon=38 + PointsPerReflectiveTon=30 PointsPerReactiveTon=30 PointsPerSolarianTon=60 + +Battlemaster: 13.0 tons x 32 = **416 points**. + +Cross-check against `MechLab.cpp:796`, which is also the formula written into +the template's own header: + + armor_bar = (TotalArmor / 535) * 100 + +416/535 x 100 = 77.8%, against a stored `ArmorRating` of 79. 50 of the 65 +chassis agree within 5 points, which is good corroboration. + +An earlier attempt summed `MaxArmorValue` from `.damage` scaled by those +numbers. That yields armour **capacity** (1024 for the Battlemaster), not what +is fitted. `.damage` `MaxArmorValue` is the per-zone cap the mechlab will not +let you exceed; it is not the stock allocation. + +## Judgement calls + +**Columns 10-15 mean *installed*, not *can load*.** The Battlemaster's `.data` +says `CanLoadECM=Yes`, `CanLoadAMS=Yes`, `CanLoadLAMS=Yes`, `CanLoadBeagle=Yes`, +yet the template sample marks all four `n`. The template is right: these +describe the stock loadout, so they are read from `.subsystems`, not `.data`. + +**Column 7 follows the data, not the sample.** The template sample says +Ferro-Fibrous for the Battlemaster; `battlemaster.subsystems` says +`ArmorType=Standard`. Internal structure (Endo Steel) matched exactly, so the +sample cell looks like a guess. The data wins. + +**Columns 51-54 are pulled, not recomputed.** These are the authored bar values +from `.instance`. The template header carries formulas for them, but the stored +values are what the game ships. Note that MechLab *recomputes* the bars live +while you edit a mech (`MechLab.cpp:794-796`), so the stored values are the +selection-screen figures, and the two can drift: 15 of 65 chassis diverge from +the `/535` armour formula by more than 5 points, Fafnir worst at 38. That is a +property of the source data, not an extraction error. + +**Column 3 says `5.1.0b`.** This is the current source tree, not the 5.0.7D +drop the template came from. Change `SOURCE_VERSION` if you need a different +stamp. + +## Layout changes + +The template gives each arm three weapon slots. The **Gladiator mounts six Clan +Medium Pulse Lasers in its right arm**, so three were being silently dropped. + +Before widening anything, the maximum was measured per location across all 65 +chassis: + +| Location | Template slots | Max used | Worst case | +|---|---|---|---| +| Head | 1 | 1 | Atlas | +| Left arm | 3 | 3 | Deimos | +| **Right arm** | **3** | **6** | **Gladiator** | +| Left torso | 4 | 4 | Mauler | +| Right torso | 4 | 4 | Chimera | +| Centre torso | 2 | 2 | Annihilator | +| Special 1 | 3 | 2 | Ares | +| Special 2 | 3 | 3 | Ares | + +Only the right arm overflowed. `Right_Arm_4/5/6` were inserted after +`Right_Arm_3`, following the existing naming, and everything from the old +column 32 onward shifted right by three. The workbook is now 55 columns. + +If a future chassis overflows another location, the generator prints a +`overflowed ... -- widen LOC_COLS` warning naming the mech and location. Widen +the entry in `LOC_COLS`, add matching `NEW_HEADERS`, and bump `NCOLS`. + +## Weapon names + +Weapon labels are derived from the subsystem filename, since the `.subsystems` +blocks carry no display string. Three rules cover almost everything: + + camelCase -> spaced ClanMediumPulseLaser -> Clan Medium Pulse Laser + ACRONYM+Word -> split ClanERSmallLaser -> Clan ER Small Laser + letters+digits -> split SRM6 -> SRM 6 + +Two source files are entirely upper case and cannot be split by any general +rule, so they are special-cased: `CLANLRM10` and `ERPPC`. 44 distinct weapon +names are produced; if a new one reads oddly, check the filename first. + +## Pitfalls + +**Folder case.** `core.build` writes `mechs\annihilator\...` but the folder is +`Annihilator`. The generator builds a case-insensitive directory index; file +lookups inside a chassis folder are case-insensitive too. + +**Encoding.** Content files are CRLF and contain legacy high bytes. They are +read as `latin-1`. Do not decode them as UTF-8. + +**Engine upgrades matter.** Top speed is not `MaxSpeed`. It is +`MinMaxSpeed + MPSPerUpgrade x EngineUpgrades`, capped at `MaxSpeed`. The +Battlemaster ships with `EngineUpgrades=5`, which is exactly what turns +65.0 kph into the template's 90.02 kph. Getting this wrong is silently +plausible, which is why the sample row is worth calibrating against. + +**Display names are not folder names.** Folder `Blackhawk` is displayed as +"Black Hawk", `Madcat` as "Mad Cat". The portrait filenames in `hsh/Mechs/` +carry the localized names, so they are used as the name source. + +## Verification + +`verify-mech-specs.py` runs five checks: + +1. **Layout** - weapon and rating headers are where they should be. +2. **Fill rates** - every non-weapon column populated for every chassis. +3. **Calibration** - the generated Battlemaster row against the template's + hand-made sample, weapon cells only. This is the important one: those 11 + cells are an independent statement of the loadout, so if they still match, + `.subsystems` is being read correctly. +4. **Weapon names** - flags camelCase remnants and long capital runs that + suggest an unsplit filename. +5. **Armour cross-check** - installed points against the stored `ArmorRating` + bar via the `/535` formula, reported as a distribution. + +Expected result today: `PASS`, 55 columns x 65 chassis, all 11 sample weapon +cells matching, 44 clean weapon names, 50/65 armour bars within 5 points. + +## Updating + +Re-run the generator. It re-reads the roster each time, so: + +- registering the six new chassis in `core.build` adds six rows automatically +- editing any `.subsystems` loadout is picked up with no code change +- adding a weapon needs no change unless its filename defeats the naming rules + +Then run the verifier and check it still reports `PASS`. diff --git a/MECHSPECSHEET/generate-mech-specs.py b/MECHSPECSHEET/generate-mech-specs.py new file mode 100755 index 00000000..ff93dc72 --- /dev/null +++ b/MECHSPECSHEET/generate-mech-specs.py @@ -0,0 +1,421 @@ +#!/usr/bin/env python3 +"""Generate the stock-mech specification spreadsheet from the FireStorm source tree. + +Reads every chassis registered in Content/core.build and writes one row per mech +into a copy of template/mech-spec-template.xlsx. + + python3 generate-mech-specs.py [-o OUTPUT.xlsx] + +Standard library only -- no openpyxl. An .xlsx is a zip of XML, and the only +part that needs rewriting is xl/worksheets/sheet1.xml, so the template's styling, +column widths and drawings survive untouched. + +See README.md for the column-by-column derivation and the engine-source evidence +behind the armour and rating figures. +""" +import argparse +import math +import os +import re +import sys +import zipfile +from xml.etree import ElementTree as ET + +HERE = os.path.dirname(os.path.abspath(__file__)) +REPO = os.path.abspath(os.path.join(HERE, "..")) +MW4 = os.path.join(REPO, "Gameleap", "mw4") +CONTENT = os.path.join(MW4, "Content") +MECHS = os.path.join(CONTENT, "Mechs") +TEMPLATE = os.path.join(HERE, "template", "mech-spec-template.xlsx") + +NS = "http://schemas.openxmlformats.org/spreadsheetml/2006/main" +Q = "{%s}" % NS +ET.register_namespace("", NS) + +# Source-tree version stamped into the "Release Version" column. +SOURCE_VERSION = "5.1.0b" + + +# --------------------------------------------------------------------- parsing +def read(path): + """Content files are CRLF and latin-1 (some carry legacy high bytes).""" + with open(path, "rb") as f: + return f.read().decode("latin-1") + + +def kv_pages(text): + """Parse a [page] / key=value file into an ordered list of (page, dict).""" + pages, name, cur = [], None, {} + for line in text.splitlines(): + line = line.strip() + if not line or line.startswith("//"): + continue + m = re.match(r"^\[([^\]]+)\]$", line) + if m: + if name is not None: + pages.append((name, cur)) + name, cur = m.group(1), {} + continue + m = re.match(r"^([A-Za-z_0-9]+)\s*=\s*(.*)$", line) + if m and name is not None: + cur.setdefault(m.group(1).lower(), m.group(2).strip()) + if name is not None: + pages.append((name, cur)) + return pages + + +def flat_keys(text): + """Parse a file as a single flat key=value map, ignoring page headers.""" + out = {} + for line in text.splitlines(): + m = re.match(r"^([A-Za-z_0-9]+)\s*=\s*(.*)$", line.strip()) + if m: + out.setdefault(m.group(1).lower(), m.group(2).strip()) + return out + + +def num(d, key, default=None): + v = d.get(key.lower()) + if v is None: + return default + m = re.match(r"^[-+]?[\d.]+", v) + return float(m.group(0)) if m else default + + +# ------------------------------------------------------------------- constants +TORSO_DEFINES = {} +for _line in read(os.path.join(CONTENT, "Defines", "MechTorso.defines")).splitlines(): + _m = re.match(r"^!?([A-Za-z_0-9]+)\s*=\s*([-\d.]+)", _line.strip()) + if _m: + TORSO_DEFINES[_m.group(1)] = float(_m.group(2)) + + +def resolve_torso(val): + """.torso fields are usually $(MACRO) references into MechTorso.defines.""" + if val is None: + return None + m = re.match(r"^\$\(([^)]+)\)$", val.strip()) + if m: + return TORSO_DEFINES.get(m.group(1)) + try: + return float(val) + except ValueError: + return None + + +# Armour points per ton, by type, from Content/Subsystems/Armor.data. +ARMOR_PTS = {} +for _line in read(os.path.join(CONTENT, "Subsystems", "Armor.data")).splitlines(): + _m = re.match(r"^PointsPer([A-Za-z]+)Ton\s*=\s*([\d.]+)", _line.strip()) + if _m: + ARMOR_PTS[_m.group(1).lower()] = float(_m.group(2)) + +# The subsystem spells it FerroFiberus; Armor.data keys that rate as "Ferro". +ARMOR_KEY = {"ferrofiberus": "ferro", "standard": "standard", "reactive": "reactive", + "reflective": "reflective", "solarian": "solarian"} +ARMOR_PRETTY = {"ferrofiberus": "Ferro-Fibrous", "standard": "Standard", + "reactive": "Reactive", "reflective": "Reflective", + "solarian": "Solarian"} +INTERNAL_PRETTY = {"endosteel": "Endo Steel", "standard": "Standard"} +CLASS_BANDS = [(35, "Light"), (55, "Medium"), (75, "Heavy"), (1000, "Assault")] + +# Coolant is spent in fixed 5-point flushes, so MaxCoolant / 5 = flush count. +COOLANT_PER_FLUSH = 5.0 + +# Column layout. Right arm carries 6 slots because the Gladiator mounts six +# Clan Medium Pulse Lasers there; every other location fits the original sheet. +LOC_COLS = { + "head": (25, 1), + "leftarm": (26, 3), "rightarm": (29, 6), + "lefttorso": (35, 4), "righttorso": (39, 4), + "centertorso": (43, 2), + "special1": (45, 3), "special2": (48, 3), +} +COL_POWER, COL_ARMOR_R, COL_SPEED_R, COL_HEAT_R = 51, 52, 53, 54 +NCOLS = 55 +# Old template column -> new column: everything from old col 32 on slides +3. +SHIFT_FROM = 31 +NEW_HEADERS = {32: "Right_Arm_4", 33: "Right_Arm_5", 34: "Right_Arm_6"} + +# Electronics are recognised by their subsystem model path. +ELECTRONICS = {"ecmsubsystem": "ecm", "beaglesubsystem": "bap", "ams": "ams", + "lams": "lams", "jumpjetsubsystem": "jj"} + +ROMAN = {"Ii": "II", "Iic": "IIc", "Iii": "III", "Mkii": "MkII"} + + +def titlecase(s): + words = [w[:1].upper() + w[1:] for w in s.split(" ")] + return " ".join(ROMAN.get(w, w) for w in words) + + +def pretty_weapon(model): + """WeaponSubsystems\\...\\ClanERSmallLaser.data -> 'Clan ER Small Laser'.""" + name = re.sub(r"\.data$", "", os.path.basename(model.replace("\\", "/")), flags=re.I) + name = re.sub(r"(?<=[a-z])(?=[A-Z])", " ", name) # camelCase -> spaced + name = re.sub(r"(?<=[A-Z])(?=[A-Z][a-z])", " ", name) # ERSmall -> ER Small + name = re.sub(r"(?<=[A-Za-z])(?=\d)", " ", name) # SRM6 -> SRM 6 + # a couple of source files are all-caps, which no general rule can split + name = re.sub(r"^CLANLRM\b", "Clan LRM", name) + name = re.sub(r"\bERPPC\b", "ER PPC", name) + return re.sub(r"\s+", " ", name).strip() + + +# ----------------------------------------------------------------- extraction +def build_rows(): + roster = [] + for line in read(os.path.join(CONTENT, "core.build")).splitlines(): + m = re.match(r"^\s*instance\s*=\s*mechs\\([^\\]+)\\([^\\]+)\.instance\s*$", + line, re.I) + if m: + roster.append((m.group(1), m.group(2))) + + # core.build spells folder names lowercase; the tree does not + dirs = {n.lower(): n for n in os.listdir(MECHS) + if os.path.isdir(os.path.join(MECHS, n))} + + # Display names come from the portrait filenames, which are the localized + # names the game shows (e.g. folder "Blackhawk" -> "black hawk.bmp"). + portraits = {} + pdir = os.path.join(MW4, "hsh", "Mechs") + if os.path.isdir(pdir): + for f in os.listdir(pdir): + if f.lower().endswith(".bmp"): + stem = f[:-4] + portraits[stem.lower().replace(" ", "").replace("-", "")] = titlecase(stem) + + rows, warnings = [], [] + for folder, base in roster: + folder = dirs.get(folder.lower(), folder) + d = os.path.join(MECHS, folder) + if not os.path.isdir(d): + warnings.append("%s: folder not found" % folder) + continue + + listing = os.listdir(d) + + def path(ext): + want = (base + ext).lower() + for f in listing: + if f.lower() == want: + return os.path.join(d, f) + return None + + p_data, p_sub = path(".data"), path(".subsystems") + if not (p_data and p_sub): + warnings.append("%s: missing .data or .subsystems" % folder) + continue + + data = flat_keys(read(p_data)) + inst = flat_keys(read(path(".instance"))) if path(".instance") else {} + eng = flat_keys(read(path(".engine"))) if path(".engine") else {} + tor = flat_keys(read(path(".torso"))) if path(".torso") else {} + subs = kv_pages(read(p_sub)) + + disp = portraits.get(folder.lower().replace(" ", "").replace("-", ""), folder) + tech = "Clan" if "clan" in (data.get("techtype") or "").lower() else "IS" + tons = num(data, "MaxVehicleTonnage", 0.0) + klass = next(n for lim, n in CLASS_BANDS if tons <= lim) + + armor_type = internal_type = "" + zone_tons = {} + heatsinks, double_hs, upgrades = 0, False, 0 + installed = dict(jj=False, ecm=False, bap=False, ams=False, lams=False) + weapons = [] + + for page, kv in subs: + model = (kv.get("model") or "").replace("/", "\\") + low = model.lower() + if "subsystems\\armor.data" in low: + armor_type = kv.get("armortype", "") + internal_type = kv.get("internaltype", "") + for k, v in kv.items(): + if k in ("model", "executionstate", "internallocation", + "armortype", "internaltype"): + continue + try: + zone_tons[k] = float(v) + except ValueError: + pass + elif "heatsinksubsystem" in low: + heatsinks += 1 + double_hs = double_hs or ("double" in low) + elif low.endswith(".engine"): + upgrades = int(num(kv, "EngineUpgrades", 0) or 0) + elif "weaponsubsystems" in low: + weapons.append((page, kv, model)) + else: + for token, flag in ELECTRONICS.items(): + if token in low: + installed[flag] = True + + # Installed armour: the Armor block's per-zone values are TONS. + ppt = ARMOR_PTS.get(ARMOR_KEY.get(armor_type.lower(), armor_type.lower()), 0.0) + if armor_type and not ppt: + warnings.append("%s: unknown armour type %r" % (folder, armor_type)) + armor_pts = sum(zone_tons.values()) * ppt + + minmax = num(data, "MinMaxSpeed", 0.0) or 0.0 + maxspd = num(data, "MaxSpeed", 0.0) or 0.0 + mps_up = num(eng, "MPSPerUpgrade", 0.0) or 0.0 + top_mps = min(minmax + mps_up * upgrades, maxspd) if maxspd else minmax + turn_rad = (num(data, "TopSpeedTurnRate", 0.0) or 0.0) * math.pi / 180.0 + twist_radius = resolve_torso(tor.get("twistradius")) + twist_speed = resolve_torso(tor.get("twistspeed")) + + slots = {} + for page, kv, model in weapons: + loc = (kv.get("internallocation") or "").lower().replace(" ", "") + col0, cap = LOC_COLS.get(loc, (None, 0)) + if col0 is None: + warnings.append("%s: weapon %s has unmapped location %r" + % (folder, page, loc)) + continue + label = pretty_weapon(model) + ammo = kv.get("ammocount") + if ammo and re.match(r"^\d+$", ammo) and int(ammo) > 0: + label += " (%s)" % ammo + if (kv.get("weaponfacing") or "0").strip() == "1": + label += " (Rear)" + for i in range(cap): + if (col0 + i) not in slots: + slots[col0 + i] = label + break + else: + warnings.append("%s: %s overflowed %s (cap %d) -- widen LOC_COLS" + % (folder, label, loc, cap)) + + row = { + 1: disp, 2: "y", 3: SOURCE_VERSION, 4: tech, 5: tons, 6: klass, + 7: ARMOR_PRETTY.get(armor_type.lower(), armor_type), + 8: INTERNAL_PRETTY.get(internal_type.lower(), internal_type), + 9: round(armor_pts) if armor_pts else "", + 10: "Y" if installed["jj"] else "n", + 11: "n" if str(inst.get("doeshavelightamp", "1")).strip() == "0" else "Y", + 12: "Y" if installed["ecm"] else "n", + 13: "Y" if installed["ams"] else "n", + 14: "Y" if installed["lams"] else "n", + 15: "Y" if installed["bap"] else "n", + 16: (num(inst, "MaxCoolant", 0.0) or 0.0) / COOLANT_PER_FLUSH, + 17: heatsinks * (2 if double_hs else 1), + 18: round(top_mps * 3.6, 2), + 19: round((num(data, "MaxGimpSpeed", 0.0) or 0.0) * 3.6, 2), + 20: num(data, "Acceleration", ""), + 21: num(data, "Decceleration", ""), # double-c is canonical + 22: round(turn_rad, 3), + 23: twist_radius * 2 if twist_radius is not None else "", + 24: twist_speed if twist_speed is not None else "", + COL_POWER: num(inst, "PowerRating", ""), + COL_ARMOR_R: num(inst, "ArmorRating", ""), + COL_SPEED_R: num(inst, "SpeedRating", ""), + COL_HEAT_R: num(inst, "HeatRating", ""), + } + row.update(slots) + rows.append(row) + + rows.sort(key=lambda r: str(r[1]).lower()) + return rows, warnings + + +# --------------------------------------------------------------------- writing +def colletter(n): + s = "" + while n: + n, rem = divmod(n - 1, 26) + s = chr(65 + rem) + s + return s + + +def esc(s): + return str(s).replace("&", "&").replace("<", "<").replace(">", ">") + + +def cell(ref, val, style=None): + st = ' s="%s"' % style if style else "" + if isinstance(val, (int, float)) and not isinstance(val, bool): + return '%s' % (ref, st, val) + return ('%s' + % (ref, st, esc(val))) + + +def write_xlsx(rows, out_path): + zin = zipfile.ZipFile(TEMPLATE) + sheet = zin.read("xl/worksheets/sheet1.xml").decode("utf-8") + + shared = [] + if "xl/sharedStrings.xml" in zin.namelist(): + for si in ET.fromstring(zin.read("xl/sharedStrings.xml")).findall(Q + "si"): + shared.append("".join(t.text or "" for t in si.iter(Q + "t"))) + + # Recover the template header text and per-cell style so the rebuilt header + # keeps the original wording and formatting. + row1 = re.search(r"]*r=\"1\".*?", sheet, re.S).group(0) + old_hdr, old_style = {}, {} + for m in re.finditer(r']*)>(.*?)', row1, re.S): + col, attrs, body = m.group(1), m.group(2), m.group(3) + n = 0 + for ch in col: + n = n * 26 + (ord(ch) - 64) + sm = re.search(r's="(\d+)"', attrs) + old_style[n] = sm.group(1) if sm else None + v = re.search(r"(\d+)", body) + if v is not None and 't="s"' in attrs: + old_hdr[n] = shared[int(v.group(1))] + else: + t = re.search(r"]*>(.*?)", body, re.S) + old_hdr[n] = t.group(1) if t else "" + + shift = lambda n: n if n <= SHIFT_FROM else n + len(NEW_HEADERS) + new_hdr, new_style = {}, {} + for n, txt in old_hdr.items(): + new_hdr[shift(n)] = txt + new_style[shift(n)] = old_style.get(n) + for n, txt in NEW_HEADERS.items(): + new_hdr[n] = txt + new_style[n] = old_style.get(SHIFT_FROM) + + out_rows = ['%s' % "".join( + cell("%s1" % colletter(c), new_hdr[c], new_style.get(c)) + for c in sorted(new_hdr) if new_hdr[c] != "")] + + for i, row in enumerate(rows, start=2): + cells = [cell("%s%d" % (colletter(c), i), row[c]) + for c in range(1, NCOLS + 1) + if row.get(c, "") not in ("", None)] + out_rows.append('%s' % (i, "".join(cells))) + + body = re.search(r"()(.*?)()", sheet, re.S) + new_sheet = sheet[:body.start(2)] + "".join(out_rows) + sheet[body.end(2):] + new_sheet = re.sub(r"", + '' % (colletter(NCOLS), len(rows) + 1), + new_sheet) + + with zipfile.ZipFile(out_path, "w", zipfile.ZIP_DEFLATED) as zout: + for item in zin.infolist(): + payload = zin.read(item.filename) + if item.filename == "xl/worksheets/sheet1.xml": + payload = new_sheet.encode("utf-8") + zout.writestr(item, payload) + + +def main(): + ap = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("-o", "--output", + default=os.path.join(HERE, "mech-specifications-%s.xlsx" % SOURCE_VERSION)) + args = ap.parse_args() + + rows, warnings = build_rows() + print("chassis parsed: %d" % len(rows)) + if warnings: + print("\nwarnings (%d):" % len(warnings)) + for w in warnings: + print(" %s" % w) + + write_xlsx(rows, args.output) + print("\nwrote %s" % args.output) + return 1 if warnings else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/MECHSPECSHEET/mech-specifications-5.1.0b.xlsx b/MECHSPECSHEET/mech-specifications-5.1.0b.xlsx new file mode 100644 index 00000000..6688eda8 --- /dev/null +++ b/MECHSPECSHEET/mech-specifications-5.1.0b.xlsx @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fe03d39690990d1f9c163a49ec0a4f74d5bee78d507e81e8d24ce51e97b939f3 +size 18972 diff --git a/MECHSPECSHEET/template/mech-spec-template.xlsx b/MECHSPECSHEET/template/mech-spec-template.xlsx new file mode 100644 index 00000000..78a0096e --- /dev/null +++ b/MECHSPECSHEET/template/mech-spec-template.xlsx @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c972607e72fc5b5b793b49dce239b1f11a0023f2ca8743c4c66a7c417c6f4688 +size 24686 diff --git a/MECHSPECSHEET/verify-mech-specs.py b/MECHSPECSHEET/verify-mech-specs.py new file mode 100755 index 00000000..e3a96e87 --- /dev/null +++ b/MECHSPECSHEET/verify-mech-specs.py @@ -0,0 +1,157 @@ +#!/usr/bin/env python3 +"""Verify a generated mech specification workbook. + + python3 verify-mech-specs.py [WORKBOOK.xlsx] + +Checks, in order: + 1. layout -- column count and weapon/rating header positions + 2. fill rates -- how many chassis have a value in each non-weapon column + 3. calibration -- the Battlemaster row against the hand-made template sample + 4. weapon names -- every distinct name, to catch filename-derived artefacts + 5. armour cross-check -- installed points vs the stored ArmorRating bar + +The calibration check is the important one: the template ships a single +hand-filled Battlemaster row, and the 11 weapon cells in it are an independent +statement of what the loadout should be. If those still match, the extraction +is reading .subsystems correctly. +""" +import os +import re +import sys +import zipfile +from collections import Counter +from xml.etree import ElementTree as ET + +HERE = os.path.dirname(os.path.abspath(__file__)) +NS = "{http://schemas.openxmlformats.org/spreadsheetml/2006/main}" +TEMPLATE = os.path.join(HERE, "template", "mech-spec-template.xlsx") +DEFAULT = os.path.join(HERE, "mech-specifications-5.1.0b.xlsx") + + +def load(path): + """Return {(row, col): value} for sheet1, resolving shared + inline strings.""" + z = zipfile.ZipFile(path) + shared = [] + if "xl/sharedStrings.xml" in z.namelist(): + for si in ET.fromstring(z.read("xl/sharedStrings.xml")).findall(NS + "si"): + shared.append("".join(t.text or "" for t in si.iter(NS + "t"))) + cells = {} + for c in ET.fromstring(z.read("xl/worksheets/sheet1.xml")).iter(NS + "c"): + m = re.match(r"([A-Z]+)(\d+)", c.get("r")) + n = 0 + for ch in m.group(1): + n = n * 26 + (ord(ch) - 64) + v, inline = c.find(NS + "v"), c.find(NS + "is") + if v is not None: + val = shared[int(v.text)] if c.get("t") == "s" else v.text + elif inline is not None: + val = "".join(x.text or "" for x in inline.iter(NS + "t")) + else: + continue + cells[(int(m.group(2)), n)] = val + return cells + + +def main(): + path = sys.argv[1] if len(sys.argv) > 1 else DEFAULT + new = load(path) + nrows = max(r for r, _ in new) + ncols = max(c for _, c in new) + hdr = {c: (new.get((1, c)) or "").replace("\n", " ") for c in range(1, ncols + 1)} + problems = [] + + print("workbook: %s" % os.path.basename(path)) + print("layout: %d columns x %d chassis\n" % (ncols, nrows - 1)) + + # 1. layout --------------------------------------------------------------- + expect = {26: "Left_Arm_1", 29: "Right_Arm_1", 34: "Right_Arm_6", + 35: "Left_Torso_1", 39: "Right_Torso_1", 43: "Center_Torso_1", + 45: "Special_1_1", 48: "Special_2_1"} + bad = [(c, w, hdr.get(c)) for c, w in expect.items() if hdr.get(c) != w] + print("1. layout: %s" % ("OK" if not bad else "MISPLACED %s" % bad)) + if bad: + problems.append("layout") + + # 2. fill rates ----------------------------------------------------------- + print("\n2. fill rates (non-weapon columns):") + thin = [] + for c in list(range(1, 25)) + list(range(51, 55)): + filled = sum(1 for r in range(2, nrows + 1) if (new.get((r, c)) or "").strip()) + if filled < nrows - 1: + thin.append((c, hdr[c][:30], filled)) + if thin: + for c, h, f in thin: + print(" col %-2d %-30s only %d/%d" % (c, h, f, nrows - 1)) + problems.append("fill") + else: + print(" all populated for all %d chassis" % (nrows - 1)) + + # 3. calibration against the template sample ------------------------------ + print("\n3. calibration vs template Battlemaster sample:") + if os.path.exists(TEMPLATE): + old = load(TEMPLATE) + bm = next((r for r in range(2, nrows + 1) + if (new.get((r, 1)) or "").lower().startswith("battlemaster")), None) + # weapon cells only: template cols 25..47. Cols 48-51 are the rating + # bars, which the template holds as hand-made guesses ('53?', '98?') + # while we pull the authored values -- see README, Judgement calls. + shift = lambda c: c if c <= 31 else c + 3 + mismatch = [] + for c in range(25, 48): + a = (old.get((2, c)) or "").strip() + b = (new.get((bm, shift(c))) or "").strip() + if a and a != b: + mismatch.append((c, a, b)) + if mismatch: + for c, a, b in mismatch: + print(" col %-2d sample %-24r generated %r" % (c, a, b)) + problems.append("calibration") + else: + print(" all 11 sample weapon cells match") + else: + print(" template missing, skipped") + + # 4. weapon names --------------------------------------------------------- + names = set() + for r in range(2, nrows + 1): + for c in range(25, 51): + v = new.get((r, c)) + if v: + names.add(re.sub(r"\s*\(.*\)$", "", v)) + # SSRM/LRM/PPC are legitimately all-caps; only flag camelCase remnants or + # runs long enough to be an unsplit filename such as CLANLRM. + odd = [n for n in names if re.search(r"[a-z][A-Z]|[A-Z]{5,}", n)] + print("\n4. weapon names: %d distinct" % len(names)) + if odd: + print(" possible un-split filenames: %s" % sorted(odd)) + problems.append("names") + else: + print(" all names look cleanly separated") + + # 5. armour cross-check --------------------------------------------------- + # MechLab.cpp:796 armor_bar = (TotalArmor / 535) * 100 + # The stored bars are authored per chassis, so they are a sanity signal, not + # ground truth. Most sit close to the computed value; a handful do not. + print("\n5. armour points vs stored ArmorRating bar:") + deltas = [] + for r in range(2, nrows + 1): + try: + pts = float(new.get((r, 9))) + bar = float(new.get((r, 52))) + except (TypeError, ValueError): + continue + deltas.append((abs(pts / 535.0 * 100.0 - bar), new.get((r, 1)), pts, bar)) + deltas.sort(reverse=True) + close = sum(1 for d, *_ in deltas if d <= 5) + print(" within 5 points: %d/%d chassis" % (close, len(deltas))) + print(" largest divergences (authored bars, not recomputed):") + for d, nm, pts, bar in deltas[:5]: + print(" %-18s %6.0f pts -> %5.1f%% stored %5.1f%% (delta %.1f)" + % (nm, pts, pts / 535.0 * 100.0, bar, d)) + + print("\nresult: %s" % ("PASS" if not problems else "ISSUES: %s" % ", ".join(problems))) + return 1 if problems else 0 + + +if __name__ == "__main__": + sys.exit(main())