Vendor dpl3-revive: the Division/DPL3 renderer, now ours

Bring the graphics-dev collaborator's dpl3-revive into the repo as first-class
project code (they've handed it off; it's ours now). This is the proven
Division renderer that our in-process rt_draw has been trying to be.

What's here:
- parser/  B2Z/V2Z/SVT/SCN/SPL/BGF/BMF/BSL decoders (pure Python).
- spec/    reverse-engineered format + the definitive VelociRender wire
           protocol (from the original DIVISION source, matches our live
           VPX node/action tables exactly).
- source-ref/  read-only copies of the original DIVISION C (BIZREAD.C,
           DPLTYPES.H, DPL.H) that define the formats.
- patha/   the "virtual VelociRender board": vrboard.py (24-action protocol
           server), vrview.py (numpy software rasterizer, the reference),
           vrview_gl.py (moderngl GPU backend, 832x512@60Hz), plus the
           run/replay/regress tooling and evidence renders. Drives FLYK/BLADE/
           Star Trek demos AND our btl4opt/rpl4opt game binaries.
- viewer/  WebGL archive generators (.py); prebuilt HTML/data regeneratable.
- samples/ small test models/textures.
- bt*.raw.bin  real BTL4OPT arena wire captures (kept for offline renderer
           testing/regression against OUR game).

.gitignore keeps the multi-hundred-MB demo capture dumps + debug logs +
regeneratable viewer bundles out of history (they stay on disk).

Phase 0 of the integration is validated: their board decodes our bt8 capture
with zero errors (3748 nodes, 507 instances, 4 mechs) and renders our arena
(terrain/dome/sky, correct Division DAC gamma). Plan + status in memory;
integration continues in emulator/RENDERER-COLLAB.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Cyd
2026-07-05 22:06:25 -05:00
co-authored by Claude Fable 5
parent 9536d4cbc0
commit afc3fd839e
247 changed files with 13656 additions and 0 deletions
+23
View File
@@ -0,0 +1,23 @@
# dpl3-revive is now part of this repo (the friend's Division renderer, ours).
# Vendor the CODE + specs + parsers + source-ref + samples + Path-A board and
# tools + evidence PNGs. Keep the huge regeneratable debug bulk out of git
# (it stays on disk, but multi-hundred-MB wire dumps don't belong in history).
__pycache__/
*.pyc
# Wire-capture debug logs (regeneratable text output) — do not track.
*.log
# Wire captures: large demo/other-product dumps stay untracked; OUR BattleTech
# (game) captures ARE tracked for offline renderer testing / regression.
*.raw.bin
!bt*.raw.bin
!btnub*.raw.bin
# Prebuilt / regeneratable viewer bulk. Rebuild with viewer/archive.py (data/)
# and viewer/bundle.py (dpl3-viewer.html). Keep the small archive.html shell +
# the .py generators tracked.
viewer/data/
viewer/dpl3-viewer.html
viewer/vwe-archive.html
+135
View File
@@ -0,0 +1,135 @@
# dpl3-revive
Porting the Virtual World Entertainment **DPL3** renderer (BattleTech / Red Planet
arcade pods, c. 199497) to modern hardware. The original ran on an Intel **i860**
graphics coprocessor driven by an x86 DOS host; the i860 and its PixelPlanes-5-style
board are long gone, so the plan is a **reimplementation** on a modern GPU, not a
recompile.
The source drive is treated as a **read-only archive**. Nothing here writes back to
it; the reference source and sample models below were *copied out* of it.
## Status
**Milestones 1 & 2a — asset formats decoded.** The `.B2Z` / `.V2Z` binary geometry
format and the `.SVT` texture format are both fully reverse-engineered, with
working, validated parsers. Models export to OBJ; textures export to PNG. The art
is now recoverable independent of the dead engine.
Roadmap:
1.**B2Z/V2Z geometry format** — spec + parser + OBJ export.
2a. ✅ **SVT texture format** — spec + parser + PNG export (`0BGR`, size-inferred dims).
3.**GPU viewer** — self-contained WebGL page renders textured models on modern
hardware (`viewer/`). Proves the whole art pipeline end-to-end.
2b. ✅ **SCN scene format** — plain-text scenes parsed + assembled into world space
(models placed via the exact DPL transform), textured, fogged, and rendered.
`CANAL.SCN` (47 objects) explorable in the viewer.
4.**SPL camera splines** — cubic-Hermite fly-through paths evaluated and animated.
`RAPTOR.SCN` + `CAMERA.SPL` play back the original 1994 flyover in the viewer.
4b. ✅ **Scene gallery** — the viewer now hosts several recovered worlds (Raptor
flyover, Red Planet canal, underwater reef). Fish are textured by the
**model-name** convention (`shark``SHARK.SVT`) when the material carries none.
4c. ✅ **Historical archive** — every resolvable scene (~60) **and every binary model
(~264)** batch-built and browsable in a searchable, lazy-loading viewer
(`archive.html` + `serve.py`) with Scenes/Models tabs. Camera has mouse orbit +
**WASD fly movement** (Q/E for down/up). A large array to judge render/colour.
5.**Material libraries + output gamma** — authoritative per-material colour+texture
from each model's `.V2Z` (`DIFFUSE × texture`), detail textures as luminance, and
the Division card's 10-bit-DAC **gamma-1.7** curve (`GAMMA.C`) for the authentic
pastel look. Red rock / grey concrete columns / red pipes now match the original.
5b. ✅ **Texture V orientation** — the original renderer samples v=0 at SVT row 0
(no flip). Fixed a wrong global V-flip that inverted mech cockpits/pods and fish
shading. Also decoded GLOMM bit-plane texture packs (`*ALL.SVT`) — see
`spec/SVT_FORMAT.md`.
5d. ✅ **Game maps: MAP / MOD** — the real playable worlds (BattleTech arenas, Red
Planet tracks) from the game trees' `MAPS/` dirs: INI-style maps placing `.MOD`
game objects (→ BGF video geometry) with nesting, includes, and pod dropzones
(drawn as red markers). Also fixed the 1996 **LOD-container** BGF layout, which
unlocked ~850 game models that previously parsed empty. Archive: **142 scenes +
41 maps + 1,462 models**, with a Maps tab in the viewer.
5c. ✅ **Game-side formats: BGF / BMF / VGF / VMF / BSL** — the game pipeline turns
out to be the same BIZ2/VIZ2 family: `.BGF` geometry, `.BMF`/`.VMF` material
libraries (namespaced refs like `stships:eblu4_mtl`), plus `DIV-BSL2` bit-slice
texture packs (texture tag `0x018` = BITSLICE). This unlocked **Star Trek** (the
"Tesla" Galaxy-class starship + Klingon cruiser, hull textures from `STEH.BSL`)
and the BattleTech / Red Planet / Hull Pressure game content. Archive now holds
**142 scenes + 608 models** across all product lines.
6. ⬜ Port the DPL3 geometry pipeline (transform / light / cull / LOD) from the C.
7. ⬜ Replace the PXPL5 rasterizer backend with the GPU.
### Path A — run the original binaries unmodified (alternative track)
The full original DPL3 engine source is on the archive drive (`sda4/DPL3/`), both the
DOS host side and the i860 board side. That means the **VelociRender wire protocol** the
host uses to drive the (dead) i860 board is fully documented in C — no disassembly. Path
A stands up a *virtual VelociRender board*: emulate the INMOS C012 link adapter (six I/O
ports at `0x150`) as a **DOSBox-X device**, answer the 24-command protocol, and use the
renderer this project already built as the board's guts — so `FLYK.EXE` and the other
HPDAVE/DPL3 binaries run **unmodified**, renderer transparently swapped underneath.
Spec + architecture: `spec/VELOCIRENDER_PROTOCOL.md`.
## Layout
```
spec/ format documentation
B2Z_FORMAT.md the reverse-engineered DIV-BIZ2 geometry spec
SVT_FORMAT.md the reverse-engineered SVT texture spec
SCN_FORMAT.md the reverse-engineered SCN scene-format spec
SPL_FORMAT.md the reverse-engineered SPL camera-spline spec
parser/ the decoders (pure Python 3, no deps)
b2z.py geometry reader + triangulating OBJ exporter
svt.py texture reader + PNG exporter
scn.py scene-file reader + the DPL instance-transform math
spl.py camera-spline evaluator (cubic Hermite) -> camera frames
samples/ small test models/textures from the archive (+ exported .obj/.png)
source-ref/ read-only copies of the original DIVISION source that define the format
viewer/ the WebGL viewers
bundle.py packs a few curated scenes into one self-contained HTML
dpl3-viewer.html the curated gallery (open in any browser; shareable)
archive.py batch-builds EVERY resolvable scene -> data/*.json
archive.html the historical archive viewer (searchable, lazy-loads scenes)
serve.py local launcher for the archive (needed for fetch())
data/ one JSON per scene + manifest.json (~60 scenes)
render_preview.py CPU rasterizer used to verify the pipeline
preview_*.png verification renders
```
## Usage
```sh
python parser/b2z.py dump samples/TESTBOX.B2Z # structural dump
python parser/b2z.py obj samples/BALL.B2Z out.obj # export triangulated OBJ
python parser/svt.py info samples/SHARK.SVT # texture dimensions/stats
python parser/svt.py png samples/SHARK.SVT out.png # export texture to PNG
python parser/scn.py "<archive>/DPL3RLS/SCENES/CANAL.SCN" # dump a scene's contents
python viewer/bundle.py # (re)build viewer/dpl3-viewer.html
```
Then open `viewer/dpl3-viewer.html` in any browser — drag to orbit, scroll to zoom,
switch models. It's fully self-contained (geometry inlined, textures as base64).
### The historical archive (all ~60 scenes)
```sh
python viewer/archive.py # (re)build data/*.json for every resolvable scene
python viewer/serve.py # serve locally + open the archive viewer
```
`serve.py` opens `http://localhost:8765/archive.html`**Scenes** and **Models**
tabs, each a searchable list **grouped by product** (Red Planet, BattleTech, Hull
Pressure, Star Trek, Milestone, Engine/Demo). Camera: **drag** to look, **scroll**
to zoom, **WASD** to fly, **Q/E** down/up. A live **gamma slider** (default 1.25)
tunes the Division-DAC brightness. A local server is required because browsers
block `fetch()` of local files over `file://`.
Rebuild subsets with `python viewer/archive.py --scenes-only` / `--models-only`;
re-tag products (no data rebuild) with `--reclassify`.
The exported `.obj` files open in any modern 3D tool (Blender, MeshLab, etc.);
the `.png` textures open anywhere.
## Provenance
Format decoded from `source-ref/BIZREAD.C` — "Copyright DIVISION Limited (c) 1994,
author PJA" (Phil Atkin). Data model in `source-ref/DPLTYPES.H`. See
`spec/B2Z_FORMAT.md` §10 for the validation evidence.
+523
View File
@@ -0,0 +1,523 @@
#!/usr/bin/env python3
"""
b2z.py -- reader for DIVISION / Virtual World Entertainment "DIV-BIZ2" (.B2Z / .V2Z)
binary geometry objects, used by the DPL3 renderer (c. 1994).
Faithful transcription of DPL3/BIZREAD.C (Copyright DIVISION Ltd 1994, author PJA).
The original read the file in 8 KB blocks with endian correction; here we load the
whole file into memory and walk an absolute cursor -- identical results, less code.
Little-endian throughout (the files were authored on x86 / i860 hosts).
Usage:
python b2z.py dump <file.b2z> # structural dump
python b2z.py obj <file.b2z> [out.obj] # export triangulated Wavefront OBJ
"""
import struct
import sys
from dataclasses import dataclass, field
# ----------------------------------------------------------------------------
# low-level stream (mirrors get_char / get_int* / get_float32 / get_struct32)
# ----------------------------------------------------------------------------
class Stream:
def __init__(self, data: bytes):
self.d = data
self.pos = 0
def eof(self) -> bool:
return self.pos >= len(self.d)
def u8(self) -> int:
v = self.d[self.pos]
self.pos += 1
return v
def i8(self) -> int:
v = self.u8()
return v - 256 if v >= 128 else v
def u16(self) -> int:
v = struct.unpack_from("<H", self.d, self.pos)[0]
self.pos += 2
return v
def i32(self) -> int:
v = struct.unpack_from("<i", self.d, self.pos)[0]
self.pos += 4
return v
def f32(self) -> float:
v = struct.unpack_from("<f", self.d, self.pos)[0]
self.pos += 4
return v
def floats(self, n: int):
v = struct.unpack_from("<%df" % n, self.d, self.pos)
self.pos += 4 * n
return list(v)
def cstr(self) -> str:
"""null-terminated string (the `while (c=get_char())` idiom)."""
start = self.pos
while self.d[self.pos] != 0:
self.pos += 1
s = self.d[start:self.pos].decode("latin-1")
self.pos += 1 # skip the NUL
return s
def fixed_str(self, n: int) -> str:
s = self.d[self.pos:self.pos + n].decode("latin-1")
self.pos += n
return s
def skip(self, n: int):
self.pos += n
# ----------------------------------------------------------------------------
# block header: 16-bit type; top nibble selects the width of the length field.
# (hdr >> 12) & 0xc == 0x8 -> int32 len, 0x4 -> int16 len, 0x0 -> int8 len
# The record TYPE is (hdr & 0xfff); the top nibble is only a length-encoding flag,
# which is why every record type appears as 0x0NNN / 0x4NNN / 0x8NNN in the source.
# ----------------------------------------------------------------------------
def read_block(s: Stream):
hdr = s.u16()
sel = (hdr >> 12) & 0xc
if sel == 0x8:
length = s.i32() & 0xffffffff
elif sel == 0x4:
length = s.u16()
elif sel == 0x0:
length = s.u8()
else:
raise ValueError("Unrecognised block header 0x%04x" % hdr)
return hdr, length
# ----------------------------------------------------------------------------
# data model (subset of dpltypes.h that a .b2z can express)
# ----------------------------------------------------------------------------
@dataclass
class Vertex:
pos: tuple # (x, y, z)
normal: tuple = None # (nx, ny, nz)
rgba: tuple = None # (r, g, b, a)
lum: tuple = None # (l, a)
uv: tuple = None # (u, v) or (u, v, w)
# geometry primitive types (dpl_geo_type)
TRISTRIP, POLYSTRIP, PMESH, POLYGON = "tristrip", "polystrip", "pmesh", "polygon"
@dataclass
class Geometry:
geotype: str
verts: list = field(default_factory=list)
conns: list = field(default_factory=list) # explicit faces (pmesh); each = list[int]
@dataclass
class Geogroup:
name: str = ""
draw_mode: int = 0
f_material: str = None
b_material: str = None
geoms: list = field(default_factory=list)
@dataclass
class Material:
name: str
ambient: tuple = (1, 1, 1)
diffuse: tuple = (1, 1, 1)
specular: tuple = (0, 0, 0, 0)
emissive: tuple = (1, 1, 1)
opacity: tuple = (1, 1, 1)
texture: str = None
ramp: str = None
@dataclass
class Texture:
name: str
mapfile: str = None
minify: int = 0
magnify: int = 0
alpha: int = 0
wrap_u: int = 0
wrap_v: int = 0
bitslice: int = 0 # tag 0x018 (1996+): 4-bit plane index into a .BSL pack
special: str = None # tag 0x037: free-text hook, e.g. " SCROLL u0 v0 du dv"
@dataclass
class Model:
object_name: str = ""
scale: float = 1.0
units: float = 1.0
geogroups: list = field(default_factory=list)
materials: dict = field(default_factory=dict)
textures: dict = field(default_factory=dict)
ramps: dict = field(default_factory=dict)
# vertex-format table: header type (12-bit) -> (floats_per_vertex, offsets dict)
# offsets are indices into the per-vertex float array. Mirrors parse_vertices().
VFMT = {
0x080: (3, {}), # flat x y z
0x081: (6, {"n": 3}), # + normal
0x082: (7, {"c": 3}), # + rgba (cooked)
0x083: (10, {"n": 3, "c": 6}), # normal + rgba
0x084: (5, {"l": 3}), # + luminance/alpha
0x085: (8, {"n": 3, "l": 6}), # normal + la
0x088: (5, {"t2": 3}), # + 2d texture uv
0x089: (8, {"n": 3, "t2": 6}), # normal + uv
0x08a: (9, {"c": 3, "t2": 7}), # rgba + uv
0x08c: (7, {"l": 3, "t2": 5}), # la + uv
0x090: (6, {"t3": 3}), # + 3d texture uvw
0x091: (9, {"n": 3, "t3": 6}), # normal + uvw
0x092: (10, {"c": 3, "t3": 7}), # rgba + uvw
0x094: (8, {"l": 3, "t3": 5}), # la + uvw
}
PRIM = {0x043: POLYGON, 0x044: TRISTRIP, 0x045: POLYSTRIP, 0x046: PMESH}
class Reader:
def __init__(self, data: bytes):
self.s = Stream(data)
self.m = Model()
# -- header ------------------------------------------------------------
def parse_header(self, length):
end = self.s.pos + length
while self.s.pos < end:
hdr, ln = read_block(self.s)
body = self.s.pos
t = hdr & 0xfff
if t == 0x005: # scale (0x2005)
self.m.scale = self.s.f32()
elif t == 0x006: # precision (0x2006): 0 = single precision
if self.s.u8() != 0:
raise ValueError("double-precision files unsupported (as in original)")
elif t == 0x009: # unit (0x2009)
u = self.s.u8()
self.m.units = 1.0 if u == 0 else (1.0 / 25.4)
# version/date/time/filetype -> skipped
self.s.pos = body + ln
# -- vertices ----------------------------------------------------------
def read_vertices(self, fpv, offs, n):
out = []
for _ in range(n):
f = self.s.floats(fpv)
v = Vertex(pos=(f[0], f[1], f[2]))
if "n" in offs:
o = offs["n"]; v.normal = (f[o], f[o+1], f[o+2])
if "c" in offs:
o = offs["c"]; v.rgba = (f[o], f[o+1], f[o+2], f[o+3])
if "l" in offs:
o = offs["l"]; v.lum = (f[o], f[o+1])
if "t2" in offs:
o = offs["t2"]; v.uv = (f[o], f[o+1])
if "t3" in offs:
o = offs["t3"]; v.uv = (f[o], f[o+1], f[o+2])
out.append(v)
return out
def parse_vertices(self, length, geotype, group):
end = self.s.pos + length
geom = Geometry(geotype)
pre_conns = [] # pmesh connectivity accumulates across sub-blocks
while self.s.pos < end:
hdr, ln = read_block(self.s)
body = self.s.pos
t = hdr & 0xfff
nfloats = ln >> 2
if t == 0x047: # pmesh triangle connectivity
for _ in range(ln // 12):
pre_conns.append([self.s.i32(), self.s.i32(), self.s.i32()])
elif t == 0x04d: # pmesh polygon connectivity
vpp = self.s.u8()
for _ in range((ln - 1) // (vpp * 4)):
pre_conns.append([self.s.i32() for _ in range(vpp)])
elif t in VFMT: # a vertex block
fpv, offs = VFMT[t]
geom.verts = self.read_vertices(fpv, offs, nfloats // fpv)
# sphere/line/text (0x048/0x04a/0x04b) not implemented in original
self.s.pos = body + ln
if geotype == PMESH:
geom.conns = pre_conns
group.geoms.append(geom)
# -- patch (geogroup) --------------------------------------------------
def parse_patch(self, length, draw_mode):
end = self.s.pos + length
g = Geogroup(draw_mode=draw_mode)
while self.s.pos < end:
hdr, ln = read_block(self.s)
body = self.s.pos
t = hdr & 0xfff
if t == 0x008: # name (0x2008)
g.name = self.s.cstr()
elif t == 0x030: # front material (0x2030)
g.f_material = self._material_ref()
elif t == 0x031: # back material (0x2031)
g.b_material = self._material_ref(back=True, front=g.f_material)
elif t == 0x036: # vertex format / draw mode
g.draw_mode = self.s.u8()
elif t in PRIM: # a geometry primitive
self.s.pos = body # parse_vertices re-reads from body
# length already known (ln); rewind not needed -- body is content start
self.parse_vertices(ln, PRIM[t], g)
# plane/decal/facet/voodoo/sphere/line/text -> skipped
self.s.pos = body + ln
self.m.geogroups.append(g)
def _material_ref(self, back=False, front=None):
typ = self.s.u8()
if typ == 0:
return None
if typ == 2:
return "DEFAULT"
if typ == 3 and back:
return front
if typ == 1:
return self.s.cstr()
return None
# -- object ------------------------------------------------------------
def parse_object(self, length):
self._lod_taken = False
self._object_body(self.s.pos + length, 0)
def _object_body(self, end, draw_mode, in_lod=False):
while self.s.pos < end:
hdr, ln = read_block(self.s)
body = self.s.pos
t = hdr & 0xfff
if t == 0x008: # object name
self.m.object_name = self.s.cstr()
elif t == 0x036: # vertex format -> draw_mode
draw_mode = self.s.u8()
elif t == 0x042: # patch
self.s.pos = body
self.parse_patch(ln, draw_mode)
elif t == 0x041 and not in_lod:
# LOD. In the 1994 format its payload was empty (a marker the
# original loader skipped); the 1996 game format nests the
# patches INSIDE it, with 0x046(len 8) = switch in/out floats.
# Parse only the FIRST (highest-detail) LOD -- later ones are
# lower-poly alternates and would double-draw.
if not self._lod_taken:
self._lod_taken = True
self._object_body(body + ln, draw_mode, in_lod=True)
# materials-on-object / comments -> skipped
self.s.pos = body + ln
# -- texture / material / ramp ----------------------------------------
def parse_texture(self, length):
end = self.s.pos + length
tx = Texture(name="")
while self.s.pos < end:
hdr, ln = read_block(self.s)
body = self.s.pos
t = hdr & 0xfff
if t == 0x008:
tx.name = self.s.cstr()
elif t == 0x011:
tx.mapfile = self.s.cstr()
elif t == 0x012:
tx.minify = self.s.u8()
elif t == 0x013:
tx.magnify = self.s.u8()
elif t == 0x014:
tx.alpha = self.s.u8()
elif t == 0x015:
tx.wrap_u = self.s.u8()
elif t == 0x016:
tx.wrap_v = self.s.u8()
elif t == 0x018:
tx.bitslice = self.s.u8()
elif t == 0x037:
tx.special = self.s.cstr()
self.s.pos = body + ln
self.m.textures[tx.name] = tx
def parse_material(self, length):
end = self.s.pos + length
mt = Material(name="")
while self.s.pos < end:
hdr, ln = read_block(self.s)
body = self.s.pos
t = hdr & 0xfff
if t == 0x008:
mt.name = self.s.cstr()
elif t == 0x021: # texture ref
mode = self.s.u8()
if mode == 2:
mt.texture = self.s.cstr()
elif t == 0x023:
mt.ambient = tuple(self.s.floats(3))
elif t == 0x024:
mt.diffuse = tuple(self.s.floats(3))
elif t == 0x025:
mt.specular = tuple(self.s.floats(4))
elif t == 0x026:
mt.emissive = tuple(self.s.floats(3))
elif t == 0x027:
mt.opacity = tuple(self.s.floats(3))
elif t == 0x028:
mt.ramp = self.s.cstr()
self.s.pos = body + ln
self.m.materials[mt.name] = mt
def parse_ramp(self, length):
end = self.s.pos + length
name, c0, c1 = "", (0, 0, 0), (0, 0, 0)
while self.s.pos < end:
hdr, ln = read_block(self.s)
body = self.s.pos
t = hdr & 0xfff
if t == 0x008:
name = self.s.cstr()
elif t == 0x031:
c0 = tuple(self.s.floats(3))
c1 = tuple(self.s.floats(3))
self.s.pos = body + ln
self.m.ramps[name] = (c0, c1)
# -- top level ---------------------------------------------------------
def read(self):
magic = self.s.fixed_str(8)
if magic != "DIV-BIZ2":
raise ValueError("Not a DIV-BIZ2 file (magic=%r)" % magic)
while not self.s.eof():
hdr, length = read_block(self.s)
t = hdr & 0xfff
if t == 0x005: # trailer
break
elif t == 0x003: # header
self.parse_header(length)
elif t == 0x010: # texture
self.parse_texture(length)
elif t == 0x020: # material
self.parse_material(length)
elif t == 0x030: # ramp
self.parse_ramp(length)
elif t == 0x040: # object
self.parse_object(length)
elif t == 0x004: # comment
self.s.skip(length)
else:
self.s.skip(length)
return self.m
def load(path: str) -> Model:
with open(path, "rb") as fp:
return Reader(fp.read()).read()
# ----------------------------------------------------------------------------
# triangulation for export
# ----------------------------------------------------------------------------
def triangles(geom: Geometry):
"""Yield (i0, i1, i2) index triples in the geometry's own vertex space."""
n = len(geom.verts)
if geom.geotype == TRISTRIP:
for i in range(n - 2):
yield (i, i + 1, i + 2) if i % 2 == 0 else (i + 1, i, i + 2)
elif geom.geotype == PMESH:
for face in geom.conns:
for k in range(1, len(face) - 1):
yield (face[0], face[k], face[k + 1])
elif geom.geotype in (POLYGON, POLYSTRIP):
# a strip of polygons stored as a fan of the block's vertices
for k in range(1, n - 1):
yield (0, k, k + 1)
def export_obj(model: Model, out) -> tuple:
base = 1 # OBJ indices are 1-based
nv = nf = 0
out.write("# exported from DIV-BIZ2 by b2z.py\n")
out.write("# object: %s\n" % model.object_name)
for gi, g in enumerate(model.geogroups):
out.write("g %s\n" % (g.name or ("geogroup_%d" % gi)))
for geom in g.geoms:
for v in geom.verts:
out.write("v %.6g %.6g %.6g\n" % v.pos)
has_uv = any(v.uv for v in geom.verts)
if has_uv:
for v in geom.verts:
u, w = (v.uv[0], v.uv[1]) if v.uv else (0.0, 0.0)
out.write("vt %.6g %.6g\n" % (u, w))
for (a, b, c) in triangles(geom):
if has_uv:
out.write("f %d/%d %d/%d %d/%d\n" %
(base+a, base+a, base+b, base+b, base+c, base+c))
else:
out.write("f %d %d %d\n" % (base+a, base+b, base+c))
nf += 1
base += len(geom.verts)
nv += len(geom.verts)
return nv, nf
# ----------------------------------------------------------------------------
# CLI
# ----------------------------------------------------------------------------
def cmd_dump(path):
m = load(path)
print("file :", path)
print("object :", m.object_name or "(unnamed)")
print("scale/units: %g / %g" % (m.scale, m.units))
print("materials : %d %s" % (len(m.materials), list(m.materials)))
print("textures : %d %s" % (len(m.textures), list(m.textures)))
print("ramps : %d %s" % (len(m.ramps), list(m.ramps)))
print("geogroups : %d" % len(m.geogroups))
tot_v = tot_t = 0
for gi, g in enumerate(m.geogroups):
print(" [%d] name=%r draw_mode=0x%x f_mtl=%r b_mtl=%r geoms=%d"
% (gi, g.name, g.draw_mode, g.f_material, g.b_material, len(g.geoms)))
for geom in g.geoms:
tris = list(triangles(geom))
tot_v += len(geom.verts); tot_t += len(tris)
attrs = []
if geom.verts:
v0 = geom.verts[0]
for a in ("normal", "rgba", "lum", "uv"):
if getattr(v0, a) is not None:
attrs.append(a)
print(" - %-9s verts=%-4d faces=%-4d tris=%-4d attrs=%s"
% (geom.geotype, len(geom.verts), len(geom.conns), len(tris),
",".join(attrs) or "pos"))
print("TOTAL : %d verts, %d triangles" % (tot_v, tot_t))
for name, mt in m.materials.items():
print(" material %-16s diffuse=%s tex=%s" %
(name, tuple(round(x, 3) for x in mt.diffuse), mt.texture))
def cmd_obj(path, outpath=None):
m = load(path)
outpath = outpath or (path.rsplit(".", 1)[0] + ".obj")
with open(outpath, "w") as fp:
nv, nf = export_obj(m, fp)
print("wrote %s : %d vertices, %d triangles" % (outpath, nv, nf))
if __name__ == "__main__":
if len(sys.argv) < 3:
print(__doc__)
sys.exit(1)
cmd, path = sys.argv[1], sys.argv[2]
if cmd == "dump":
cmd_dump(path)
elif cmd == "obj":
cmd_obj(path, sys.argv[3] if len(sys.argv) > 3 else None)
else:
print("unknown command", cmd); sys.exit(1)
+115
View File
@@ -0,0 +1,115 @@
#!/usr/bin/env python3
"""
bsl.py -- reader for DIV-BSL2 ".BSL" bit-slice texture packs.
A BSL packs several 4-bit mono textures ("slices" A..F) into the nibbles of one
32-bit texel map -- the formalised successor of GLOMM's /m: packing (GLOMM.C).
Materials reference a slice via `MAP {"name"} + BITSLICE {k}` in .VMF/.BMF
libraries; the material's DIFFUSE colours the mono plane.
Layout (reversed from STEH.BSL / F15_GLOM.BSL):
"DIV-BSL2" 8-byte magic
u32 version, u32 ? (0x100, 1)
u32 bits_per_slice (4)
u32 dir_bytes directory size hint
u32 n_slices
n x { u32 type, u32 slice_index, cstr name }
... texel words follow; data size = edge*edge*4, edge
inferred from remaining bytes (64/128/256), same
convention as .SVT
Slice k occupies a nibble of each 32-bit word; the exact nibble order was
determined empirically (see spec): slice k = bits [28-4k .. 31-4k].
python bsl.py info <file.bsl>
python bsl.py png <file.bsl> <slice|nibble:N> [out.png]
python bsl.py probe <file.bsl> # dump ALL 8 nibble planes for eyeballing
"""
import struct
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import svt # PNG writer
SIZE_TO_EDGE = {16384 * 4: 256, 65536: 128, 16384: 64, 262144: 256}
def load(path):
with open(path, "rb") as fp:
data = fp.read()
if data[:8] != b"DIV-BSL2":
raise ValueError("Not a DIV-BSL2 file")
pos = 8
version, one, bits, dirbytes, nslices = struct.unpack_from("<5i", data, pos)
pos += 20
entries = []
for _ in range(nslices):
typ, idx = struct.unpack_from("<2i", data, pos)
pos += 8
end = data.index(b"\0", pos)
name = data[pos:end].decode("latin-1")
pos = end + 1
entries.append({"name": name, "type": typ, "slice": idx})
body = data[pos:]
edge = {16384: 64, 65536: 128, 262144: 256}.get(len(body))
if edge is None:
# tolerate padding: use largest fitting square
for sz, e in ((262144, 256), (65536, 128), (16384, 64)):
if len(body) >= sz:
body = body[:sz]
edge = e
break
words = struct.unpack("<%dI" % (edge * edge), body)
return {"entries": entries, "edge": edge, "words": words,
"bits": bits, "version": version}
def nibble_plane(b, nib):
"""Extract nibble `nib` (0 = bits 0-3 ... 7 = bits 28-31) as 8-bit luminance."""
edge, words = b["edge"], b["words"]
shift = nib * 4
out = bytearray(edge * edge * 4)
for i, w in enumerate(words):
v = (w >> shift) & 0xF
L = v * 17 # 0..15 -> 0..255
o = i * 4
out[o] = out[o + 1] = out[o + 2] = L
out[o + 3] = 255
return bytes(out)
def slice_nibble(slice_index):
"""slice k -> nibble (k+2)^1.
Slices occupy bits 8..31 in ascending order, pair-swapped within each byte by
the writer's 32-bit byte reversal (as in GLOMM's save_svt); the low byte
(nibbles 0-1) is reserved for an 8-bit colour plane. Verified by correlating
all 12 source TGAs (STEH1-6, KLNG1-6) against every nibble plane: r >= 0.98
for this mapping on every slice of both packs."""
return (slice_index + 2) ^ 1
def slice_plane(b, slice_index):
return nibble_plane(b, slice_nibble(slice_index))
if __name__ == "__main__":
cmd, path = sys.argv[1], sys.argv[2]
b = load(path)
if cmd == "info":
print("edge %dx%d bits/slice=%d slices:" % (b["edge"], b["edge"], b["bits"]))
for e in b["entries"]:
print(" slice %d type=%d %s" % (e["slice"], e["type"], e["name"]))
elif cmd == "probe":
base = path.rsplit(".", 1)[0]
for nib in range(8):
out = "%s_nib%d.png" % (base, nib)
svt.write_png(out, b["edge"], b["edge"], nibble_plane(b, nib))
print("wrote", out)
elif cmd == "png":
sel = sys.argv[3]
nib = int(sel.split(":")[1]) if sel.startswith("nibble:") else slice_nibble(int(sel))
out = sys.argv[4] if len(sys.argv) > 4 else path.rsplit(".", 1)[0] + "_s%s.png" % sel
svt.write_png(out, b["edge"], b["edge"], nibble_plane(b, nib))
print("wrote", out)
+215
View File
@@ -0,0 +1,215 @@
#!/usr/bin/env python3
"""
gamemap.py -- reader for the game-side ".MAP" arena/track format and its ".MOD"
game-object definitions (BTDAVE / BTLIVE / BTRAVINE / RPDAVE / RPLIVE MAPS dirs).
A .MAP is INI-style text:
[hall2] ; declare an asset section
type=model
modelfile=hall2.mod
[one]
type=dropzone ; pod spawn ring
dropzone= x y z rx ry rz ; rotations in RADIANS
[arena1] ; the map itself
type=map
instance=hall2 x y z rx ry rz
instancedropzone=one x y z rx ry rz
[inc]
type=include ; pull in another .MAP's sections
file=other.map
A .MOD is the game object: `[video] object=X.bgf` (render geometry, DIV-BIZ2),
`[collision] name=X.sld`, `[gamedata]` physics/damage/animations. We only need
the video geometry.
expand(path) resolves everything to a flat list of world-space placements:
[(bgf_name, matrix4x4), ...], plus dropzone positions.
Maps may instance other maps; expansion is recursive with matrix composition
(row-vector convention, as everywhere in DPL -- see SCN_FORMAT.md).
"""
import math
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import scn
def _parse_ini(path):
"""-> ordered {section: {'type':..,'modelfile':..,'file':..,
'instances':[(name,[6f])], 'dropzones':[[6f]],
'instancedropzones':[(name,[6f])]}}"""
sections = {}
cur = None
try:
lines = open(path, encoding="latin-1").read().splitlines()
except OSError:
return sections
for raw in lines:
line = raw.split("//")[0].strip()
if not line:
continue
if line.startswith("[") and line.endswith("]"):
name = line[1:-1].strip().lower()
cur = sections.setdefault(name, {"type": None, "instances": [],
"dropzones": [], "instancedropzones": []})
continue
if cur is None or "=" not in line:
continue
key, _, val = line.partition("=")
key = key.strip().lower()
val = val.strip()
if key == "type":
cur["type"] = val.lower()
elif key == "instance":
p = val.split()
cur["instances"].append((p[0].lower(), [float(x) for x in p[1:7]]))
elif key == "instancedropzone":
p = val.split()
cur["instancedropzones"].append((p[0].lower(), [float(x) for x in p[1:7]]))
elif key == "dropzone":
cur["dropzones"].append([float(x) for x in val.split()[:6]])
else:
cur[key] = val.lower() # generic key (modelfile, MotionExtent, ...)
return sections
def load_sections(path, _seen=None):
"""Parse a .MAP plus its type=include files (same directory)."""
_seen = _seen or set()
key = os.path.normcase(os.path.abspath(path))
if key in _seen:
return {}
_seen.add(key)
sections = _parse_ini(path)
merged = dict(sections)
for name, sec in sections.items():
if sec.get("type") == "include" and sec.get("file"):
inc = os.path.join(os.path.dirname(path), sec["file"])
for n, s in load_sections(inc, _seen).items():
merged.setdefault(n, s)
return merged
def _matrix(vals):
"""instance 6-tuple: x y z rx ry rz (RADIANS) -> row-vector matrix,
same composition as the SCN STATIC transform (unit scale)."""
x, y, z, rx, ry, rz = (vals + [0.0] * 6)[:6]
return scn.instance_matrix(1.0, x, y, z,
math.degrees(rx), math.degrees(ry), math.degrees(rz))
def pick_root(sections, path):
"""The map section to expand: prefer one named like the file, else the
map-typed section with the most instances."""
stem = os.path.splitext(os.path.basename(path))[0].lower()
maps = {n: s for n, s in sections.items() if s.get("type") in ("map",)}
if stem in maps:
return stem
if maps:
return max(maps, key=lambda n: len(maps[n]["instances"]))
return None
def expand(path, mod_resolver, max_depth=6):
"""Flatten a .MAP to world space.
mod_resolver(modfile_basename) -> bgf_name or None (looks up the .MOD's
[video] object=). Returns (placements, dropzones):
placements = [(bgf_name, matrix)]
dropzones = [[x,y,z,rx,ry,rz]] (world space)"""
sections = load_sections(path)
root = pick_root(sections, path)
placements, dropzones = [], []
if root is None:
return placements, dropzones
def emit(sec_name, parent_mtx, depth):
sec = sections.get(sec_name)
if sec is None or depth > max_depth:
return
t = sec.get("type")
if t == "model":
# mod_resolver -> list of (bgf_name, anim_or_None); a MOD may carry
# several video objects (door frame + sliding leaves, laser drill...)
for item in (mod_resolver(sec.get("modelfile", "")) or []):
placements.append((item, parent_mtx))
elif t == "map":
for name, vals in sec["instances"]:
emit(name, scn.matmul(_matrix(vals), parent_mtx), depth + 1)
for name, vals in sec["instancedropzones"]:
dz = sections.get(name)
if dz:
m = scn.matmul(_matrix(vals), parent_mtx)
for d in dz["dropzones"]:
wx, wy, wz = scn.xform_point(m, d[0], d[1], d[2])
dropzones.append([wx, wy, wz, d[3], d[4], d[5]])
elif t == "dropzone":
for d in sec["dropzones"]:
wx, wy, wz = scn.xform_point(parent_mtx, d[0], d[1], d[2])
dropzones.append([wx, wy, wz, d[3], d[4], d[5]])
emit(root, scn.ident(), 0)
return placements, dropzones
def parse_mod_objects(path):
""".MOD -> list of [video] object= geometry basenames (no extension).
A MOD may carry several objects ("number and order is significant!"), e.g.
doors: frame + leaves (+ low-poly LOD twins, which we drop: any name equal to
an earlier name + 'l'). Also returns the [gamedata] class and Subsystems file.
-> (objects, gameclass, subsystems_file)"""
sec = _parse_ini(path)
vid = sec.get("video") or {}
raw = vid.get("object") or vid.get("name") or ""
if not raw:
for s in sec.values():
if s.get("object"):
raw = s["object"]
break
names = []
for tok in raw.split():
base = os.path.splitext(os.path.basename(tok))[0].lower()
if not tok.lower().endswith((".bgf", ".b2z", ".biz")):
continue # skip node-name tokens
if base[:-1] in names and base.endswith("l"):
continue # low-poly LOD twin of an earlier object
names.append(base)
game = sec.get("gamedata") or {}
return names, (game.get("class") or "").lower(), game.get("subsystems")
def parse_mod(path):
"""Back-compat: the first video object basename, or None."""
names, _, _ = parse_mod_objects(path)
return names[0] if names else None
def parse_sub(path):
""".SUB subsystems -> {collision_stem: {'extent':[x,y,z], 'travel':s, 'dead':s}}
for DoorClassID sections. The collision stem (dr1_cv -> dr1) links a subsystem
to its video object by name."""
out = {}
for name, sec in _parse_ini(path).items():
if "motionextent" not in sec:
continue
ext = [float(x) for x in sec["motionextent"].split()[:3]]
coll = sec.get("collision", name)
stem = os.path.splitext(os.path.basename(coll))[0].lower()
if stem.endswith("_cv"):
stem = stem[:-3]
out[stem] = {"extent": ext,
"travel": float(sec.get("traveltime", 10.0)),
"dead": float(sec.get("deadtime", 3.0))}
return out
if __name__ == "__main__":
path = sys.argv[1]
secs = load_sections(path)
types = {}
for n, s in secs.items():
types.setdefault(s.get("type"), []).append(n)
print("root:", pick_root(secs, path))
for t, names in types.items():
print(" %-10s x%-3d %s" % (t, len(names), names[:8]))
+171
View File
@@ -0,0 +1,171 @@
#!/usr/bin/env python3
"""
scn.py -- reader for DPL3 ".SCN" scene files (the FLYK "cheesy scene format").
.SCN is plain text, one command per line. This module implements the core subset
that FLYK.C parses -- environment + object placement -- which covers the vast
majority of every scene on the disk (STATIC alone is ~5000 of ~9000 lines):
FOG z0 z1 r g b fog start/end distance + colour
BACKGND r g b background clear colour
AMBIENT r g b ambient light
LIGHT r g b ax ay az directional light: colour + orientation angles
STATIC name scale x y z ax ay az place a model instance
DYNAMIC name scale splinefile t0 dt model that follows a camera spline
SCROLL texname u0 v0 du dv scroll rate for a TEXTURE entity ("lib:name"
or a model-own name); overrides the SPECIAL
" SCROLL ..." baked into the material source
Comments start with // or #. Other keywords seen in game-tool scenes (SPECIALFX,
MORPH, TEXTURE/GEOMETRY/MATERIAL, ZONE/START/END, CLIP, VIEWANGLE, ...) are
recognised and skipped by this loader -- see spec/SCN_FORMAT.md.
Object placement transform (from FLYK.C + MATRIX.C), row-vector convention
(a point is a row, p' = p . M), angles in DEGREES:
M = Rz(az) . Rx(ax) . Ry(ay) . Scale(s) . Translate(x,y,z)
"""
import math
import sys
# ---- row-vector 4x4 matrices (DPL convention: translation in the bottom row) --
def ident():
return [[1.0 if i == j else 0.0 for j in range(4)] for i in range(4)]
def matmul(a, b):
return [[sum(a[i][k] * b[k][j] for k in range(4)) for j in range(4)] for i in range(4)]
def m_translate(x, y, z):
m = ident(); m[3][0], m[3][1], m[3][2] = x, y, z; return m
def m_scale(s):
m = ident(); m[0][0] = m[1][1] = m[2][2] = s; return m
def m_rotZ(deg):
c, s = math.cos(math.radians(deg)), math.sin(math.radians(deg))
return [[c, s, 0, 0], [-s, c, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1]]
def m_rotX(deg):
c, s = math.cos(math.radians(deg)), math.sin(math.radians(deg))
return [[1, 0, 0, 0], [0, c, s, 0], [0, -s, c, 0], [0, 0, 0, 1]]
def m_rotY(deg):
c, s = math.cos(math.radians(deg)), math.sin(math.radians(deg))
return [[c, 0, -s, 0], [0, 1, 0, 0], [s, 0, c, 0], [0, 0, 0, 1]]
def instance_matrix(scale, tx, ty, tz, ax, ay, az):
"""Exactly the sequence FLYK applies: Rz . Rx . Ry . Scale . Translate."""
m = m_rotZ(az)
m = matmul(m, m_rotX(ax))
m = matmul(m, m_rotY(ay))
m = matmul(m, m_scale(scale))
m = matmul(m, m_translate(tx, ty, tz))
return m
def xform_point(m, x, y, z):
return (x*m[0][0] + y*m[1][0] + z*m[2][0] + m[3][0],
x*m[0][1] + y*m[1][1] + z*m[2][1] + m[3][1],
x*m[0][2] + y*m[1][2] + z*m[2][2] + m[3][2])
def xform_dir(m, x, y, z):
"""rotate a direction/normal by the upper-left 3x3 (no translation)."""
return (x*m[0][0] + y*m[1][0] + z*m[2][0],
x*m[0][1] + y*m[1][1] + z*m[2][1],
x*m[0][2] + y*m[1][2] + z*m[2][2])
# ---- scene model -----------------------------------------------------------
class Instance:
__slots__ = ("name", "scale", "pos", "rot", "dynamic", "spline", "matrix")
def __init__(self, name, scale, pos, rot, dynamic=False, spline=None):
self.name = name
self.scale = scale
self.pos = pos
self.rot = rot
self.dynamic = dynamic
self.spline = spline
self.matrix = instance_matrix(scale, pos[0], pos[1], pos[2], rot[0], rot[1], rot[2])
class Scene:
def __init__(self):
self.fog = None # (z0, z1, r, g, b)
self.background = (0, 0, 0)
self.ambient = (0, 0, 0)
self.lights = [] # list of (r,g,b, ax,ay,az) (ambient excluded)
self.instances = []
self.start = None # START x y z ax ay az -- player/camera spawn
self.scrolls = {} # texture entity name -> [u0, v0, du, dv]
self.unhandled = {} # keyword -> count (recognised but skipped)
def _nums(parts, n):
return [float(x) for x in parts[:n]]
def load(path) -> Scene:
sc = Scene()
with open(path, "r", encoding="latin-1") as fp:
for raw in fp:
line = raw.strip()
if not line or line.startswith("//") or line.startswith("#"):
continue
p = line.split()
kw = p[0].upper()
a = p[1:]
try:
if kw == "FOG":
z0, z1, r, g, b = _nums(a, 5); sc.fog = (z0, z1, r, g, b)
elif kw == "BACKGND":
sc.background = tuple(_nums(a, 3))
elif kw == "AMBIENT":
sc.ambient = tuple(_nums(a, 3))
elif kw == "LIGHT":
r, g, b, ax, ay, az = _nums(a, 6)
sc.lights.append((r, g, b, ax, ay, az))
elif kw in ("STATIC", "SSTATIC"):
name = a[0]; s, x, y, z, ax, ay, az = _nums(a[1:], 7)
sc.instances.append(Instance(name, s, (x, y, z), (ax, ay, az)))
elif kw == "DYNAMIC":
name = a[0]; s = float(a[1]); spline = a[2]
sc.instances.append(Instance(name, s, (0, 0, 0), (0, 0, 0),
dynamic=True, spline=spline))
elif kw == "START":
sc.start = _nums(a, 6) # x y z ax ay az
elif kw == "SCROLL":
sc.scrolls[a[0].lower()] = _nums(a[1:], 4)
elif kw == "SCHILD":
pass # LOD child of the preceding STATIC -- skip, don't double-draw
elif kw in ("GEOMETRY", "MATERIAL", "TEXTURE"):
pass # search-path hints; the archive-wide index covers these
else:
sc.unhandled[kw] = sc.unhandled.get(kw, 0) + 1
except (ValueError, IndexError):
sc.unhandled[kw + "?"] = sc.unhandled.get(kw + "?", 0) + 1
return sc
if __name__ == "__main__":
sc = load(sys.argv[1])
print("fog :", sc.fog)
print("background:", sc.background)
print("ambient :", sc.ambient)
print("lights :", len(sc.lights), sc.lights[:2])
print("instances :", len(sc.instances))
from collections import Counter
c = Counter(i.name for i in sc.instances)
for name, n in c.most_common():
print(" %-14s x%d" % (name, n))
if sc.unhandled:
print("skipped :", dict(sc.unhandled))
+119
View File
@@ -0,0 +1,119 @@
#!/usr/bin/env python3
"""
spl.py -- reader/evaluator for DPL3 ".SPL" camera-path splines.
Faithful port of DPL3/EXAMPLES/SPLINE.C. A .SPL is plain text:
N number of control points
x y z ax ay az x N -- position + euler angles (degrees)
The path is a CLOSED LOOP of cubic-Hermite segments with Catmull-Rom tangents
(vel = (next - prev) / 2). Position and the three euler angles are each splined.
We sample the loop densely and bake, per sample, a camera basis (eye / center /
up) so downstream code just needs lookAt(). Camera looks down local -Z (proved by
CAMERA.SPL starting at ay=180 and facing into the +Z scene).
"""
import math
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import scn # reuse the DPL row-vector rotation matrices
def _solve_cubic(v0, v1, d0, d1):
# v = a t^3 + b t^2 + c t + d ; endpoints v0,v1 and tangents d0,d1
a = d1 + d0 - 2.0 * v1 + 2.0 * v0
b = v1 - v0 - d0 - a
return (a, b, d0, v0) # (c0=a, c1=b, c2=c, c3=d) matching eval order
def _solve_rot_cubic(v0, v1, d0, d1):
if d0 > 360.0:
d0 -= 360.0
if d1 > 360.0:
d1 -= 360.0
a = d1 + d0 - 2.0 * v1 + 2.0 * v0
b = v1 - v0 - d0 - a
return (a, b, d0, v0)
def _eval(c, t):
return c[0]*t*t*t + c[1]*t*t + c[2]*t + c[3]
def load_points(path):
with open(path, "r", encoding="latin-1") as fp:
toks = fp.read().split()
n = int(toks[0])
vals = [float(x) for x in toks[1:]]
pts = []
for i in range(n):
base = i * 6
if base + 6 > len(vals):
break
pts.append({"pos": vals[base:base+3], "ang": vals[base+3:base+6]})
return pts
def build_segments(pts):
"""Compute per-knot tangents (closed loop) and per-segment cubic coeffs."""
n = len(pts)
for i in range(n):
p, c, nx = pts[(i-1) % n], pts[i], pts[(i+1) % n]
c["vel"] = [(nx["pos"][k] - p["pos"][k]) / 2.0 for k in range(3)]
rot = []
for k in range(3):
delta = nx["ang"][k] - c["ang"][k]
while delta > 180:
delta -= 180
while delta < -180:
delta += 180
rot.append(delta / 2.0)
while c["ang"][k] > 360.0:
c["ang"][k] -= 360.0
c["rot"] = rot
segs = []
for i in range(n):
a, b = pts[i], pts[(i+1) % n]
seg = {"pos": [], "rot": []}
for k in range(3):
seg["pos"].append(_solve_cubic(a["pos"][k], b["pos"][k], a["vel"][k], b["vel"][k]))
seg["rot"].append(_solve_rot_cubic(a["ang"][k], b["ang"][k], a["rot"][k], b["rot"][k]))
segs.append(seg)
return segs
def _basis(pos, ang):
"""eye/center/up from position + euler (ax,ay,az) via DPL rotation order."""
R = scn.matmul(scn.matmul(scn.m_rotZ(ang[2]), scn.m_rotX(ang[0])), scn.m_rotY(ang[1]))
fwd = scn.xform_dir(R, 0.0, 0.0, -1.0) # camera looks down local -Z
up = scn.xform_dir(R, 0.0, 1.0, 0.0)
return {"eye": list(pos),
"center": [pos[i] + fwd[i] for i in range(3)],
"up": list(up)}
def sample_flythrough(path, per_seg=12):
"""Return a list of camera frames {eye,center,up} around the closed loop."""
pts = load_points(path)
segs = build_segments(pts)
frames = []
for seg in segs:
for s in range(per_seg):
t = s / float(per_seg)
pos = [_eval(seg["pos"][k], t) for k in range(3)]
ang = [_eval(seg["rot"][k], t) for k in range(3)]
frames.append(_basis(pos, ang))
return frames
if __name__ == "__main__":
pts = load_points(sys.argv[1])
frames = sample_flythrough(sys.argv[1])
print("%d control points -> %d camera frames" % (len(pts), len(frames)))
xs = [f["eye"][0] for f in frames]; ys = [f["eye"][1] for f in frames]; zs = [f["eye"][2] for f in frames]
print("eye path X[%.0f,%.0f] Y[%.0f,%.0f] Z[%.0f,%.0f]" %
(min(xs), max(xs), min(ys), max(ys), min(zs), max(zs)))
print("frame0:", {k: [round(x, 1) for x in v] for k, v in frames[0].items()})
+106
View File
@@ -0,0 +1,106 @@
#!/usr/bin/env python3
"""
svt.py -- reader for DIVISION / VWE ".SVT" textures used by the DPL3 renderer.
Format (from DPL3/DPL_LOAD.C : load_svt): there is NO header. The file is a raw
array of 32-bit texels, always square. The edge length is inferred from the file
size alone:
16384 bytes -> 64 x 64
65536 bytes -> 128 x 128
262144 bytes -> 256 x 256
Each texel is 4 bytes. The channel order within the 32-bit word is not stated in
the loader (the hardware consumed it directly), so this tool lets you pick it and
writes a standard PNG for viewing. Determined empirically: see spec/SVT_FORMAT.md.
Usage:
python svt.py info <file.svt>
python svt.py png <file.svt> [out.png] [order] # order default = rgba
"""
import struct
import sys
import zlib
SIZE_TO_EDGE = {16384: 64, 65536: 128, 262144: 256}
def load(path):
with open(path, "rb") as fp:
data = fp.read()
edge = SIZE_TO_EDGE.get(len(data))
if edge is None:
raise ValueError("Bad texture file size %d (not 64/128/256 square RGBA32)" % len(data))
return data, edge
def _png_chunk(tag, payload):
return (struct.pack(">I", len(payload)) + tag + payload +
struct.pack(">I", zlib.crc32(tag + payload) & 0xffffffff))
def write_png(path, width, height, rgba: bytes):
"""rgba = width*height*4 bytes, row-major, top-to-bottom."""
raw = bytearray()
stride = width * 4
for y in range(height):
raw.append(0) # filter: none
raw += rgba[y * stride:(y + 1) * stride]
out = b"\x89PNG\r\n\x1a\n"
out += _png_chunk(b"IHDR", struct.pack(">IIBBBBB", width, height, 8, 6, 0, 0, 0))
out += _png_chunk(b"IDAT", zlib.compress(bytes(raw), 9))
out += _png_chunk(b"IEND", b"")
with open(path, "wb") as fp:
fp.write(out)
def to_rgba(data: bytes, order: str) -> bytes:
"""Reorder 4-byte texels (file byte positions) into R,G,B,A for PNG.
`order` names which file byte maps to each channel, e.g. 'rgba','bgra'.
Use 'x' for an unused/pad byte; alpha is then forced opaque (255).
"""
idx = {c: i for i, c in enumerate(order)}
r, g, b = idx["r"], idx["g"], idx["b"]
a = idx.get("a") # may be None when a pad byte ('x') is present
mv = memoryview(data)
out = bytearray(len(data))
for p in range(0, len(data), 4):
out[p + 0] = mv[p + r]
out[p + 1] = mv[p + g]
out[p + 2] = mv[p + b]
out[p + 3] = mv[p + a] if a is not None else 255
return bytes(out)
def cmd_info(path):
data, edge = load(path)
print("file :", path)
print("size :", len(data), "bytes")
print("dims : %d x %d, 32bpp (%d texels)" % (edge, edge, edge * edge))
# quick stats on the raw bytes to hint at channel usage
b = memoryview(data)
for ch in range(4):
vals = b[ch::4]
lo = min(vals); hi = max(vals)
print(" byte[%d]: min=%3d max=%3d" % (ch, lo, hi))
def cmd_png(path, out=None, order="rgba"):
data, edge = load(path)
out = out or (path.rsplit(".", 1)[0] + ".png")
write_png(out, edge, edge, to_rgba(data, order))
print("wrote %s : %dx%d order=%s" % (out, edge, edge, order))
if __name__ == "__main__":
if len(sys.argv) < 3:
print(__doc__); sys.exit(1)
cmd = sys.argv[1]
if cmd == "info":
cmd_info(sys.argv[2])
elif cmd == "png":
cmd_png(sys.argv[2],
sys.argv[3] if len(sys.argv) > 3 else None,
sys.argv[4] if len(sys.argv) > 4 else "xbgr")
else:
print("unknown command", cmd); sys.exit(1)
+123
View File
@@ -0,0 +1,123 @@
# Capturing real FLYK.EXE link traffic (Path A validation)
Goal: run the genuine `FLYK.EXE` under DOSBox-X, have it talk to our Python
`vrboard` in place of the dead i860 board, and log every byte — to validate the
message framing against the *shipped* binary and pin the two open items
(`vr_readpixels` reply layout, and the bit30 `0x40000000` receive-side meaning).
## Why a responder, not a passive log
`start_velocirender()` streams the transputer `.BTL`, then blocks in
`startup_handshake()` waiting for **3 iserver transactions from the board**
(`VR_COMMS.C`). No framed `vr_*` message is sent until that handshake completes. So a
passive port logger stalls at boot and never sees the interesting scene traffic. The
link must *respond*. We keep all protocol logic in the tested Python `vrboard` and make
the emulator side a dumb byte-pipe.
## Architecture — socket bridge
```
FLYK.EXE (unmodified, in DOSBox-X)
│ IN/OUT ports 0x150-0x161 (polled C012, LINKIO.C)
DOSBox-X I/O handler (host-level C++, ~40 lines) ── TCP 127.0.0.1:8620 ──► vrserver.py
▲ │
└─────────────────────── board→host reply bytes ◄─────────────────────────┘
(VirtualBoard + full logging → capture.log)
```
The I/O handler runs at **host** level (it's DOSBox-X code, not guest code), so it can
open a normal host socket. No guest-side driver, no changes to `FLYK.EXE`.
## Port map (mirror of LINKIO.C `setLA`)
| Port | Dir | Handler action |
|------|-----|----------------|
| 0x150 | R (Input Data) | return next board→host byte (from socket recv buffer) |
| 0x151 | W (Output Data) | send byte to socket (host→board) |
| 0x152 | R (Input Status) | pump socket; bit0=1 if a byte is available |
| 0x153 | R (Output Status)| bit0=1 (always ready to accept) |
| 0x160 | W reset / R fifo | on write send the RESET marker; on read return 1 |
| 0x161 | W (Analyse) | ignore |
## Drop-in DOSBox-X I/O handler (sketch)
Add to a new `src/hardware/vrlink.cpp` and call `VRLINK_Init()` from hardware init.
Adapt the handler signatures to your DOSBox-X version; init Winsock on Windows.
```cpp
// vrlink.cpp -- bridge the C012 link ports (0x150-0x161) to vrserver.py
#include "inout.h"
#include <winsock2.h> // Windows; use <sys/socket.h> on *nix
static SOCKET s = INVALID_SOCKET;
static unsigned char rxbuf[4096]; static int rxlen=0, rxpos=0;
static void connect_board() {
WSADATA w; WSAStartup(MAKEWORD(2,2), &w);
s = socket(AF_INET, SOCK_STREAM, 0);
int one=1; setsockopt(s, IPPROTO_TCP, TCP_NODELAY,(char*)&one,sizeof one);
sockaddr_in a{}; a.sin_family=AF_INET; a.sin_port=htons(8620);
a.sin_addr.s_addr=inet_addr("127.0.0.1");
connect(s,(sockaddr*)&a,sizeof a);
u_long nb=1; ioctlsocket(s, FIONBIO, &nb); // non-blocking reads
}
static void pump() { // fill rxbuf if empty
if (rxpos<rxlen) return;
int n=recv(s,(char*)rxbuf,sizeof rxbuf,0);
if (n>0){ rxlen=n; rxpos=0; } else { rxlen=rxpos=0; }
}
static Bitu rd(Bitu port, Bitu){
switch(port){
case 0x150: pump(); return rxpos<rxlen ? rxbuf[rxpos++] : 0xFF; // Input Data
case 0x152: pump(); return rxpos<rxlen ? 1 : 0; // Input Status
case 0x153: return 1; // Output Status
case 0x160: return 1; // fifo-ok
} return 0xFF;
}
static void wr(Bitu port, Bitu val, Bitu){
unsigned char b=(unsigned char)val;
switch(port){
case 0x151: send(s,(char*)&b,1,0); break; // Output Data
case 0x160: send(s,"\x00RSET",5,0); break; // reset marker
case 0x161: break; // analyse: ignore
}
}
void VRLINK_Init(){
connect_board();
for (Bitu p=0x150; p<=0x161; ++p){ IO_RegisterReadHandler(p,rd,IO_MB); IO_RegisterWriteHandler(p,wr,IO_MB); }
}
```
## Run
1. `python vrserver.py` (listens on 127.0.0.1:8620, writes `capture.log`)
2. Launch DOSBox-X (with the handler built in). Mount the HPDAVE tree, set `DOS4GPATH`
to `sda4/BIN`, run the app the way `SETSVGA.BAT` + `HLOOP.BAT` do:
```
set DOS4GPATH=C:\SDA4\BIN
call SETSVGA.BAT REM sets DPLARG=/tranny~...~/i860~...~/device~0x150~/video~svga~...
FLYK.EXE <scene args> REM (see HFLY/HLOOP for the scene/run pair)
```
3. Watch `capture.log` — it annotates every reassembled message and reply. Boot should
walk: RESET → 3 iserver txns → `hspcode`/`860code`/`860data`/`860bss`/`860args`
(discarded) → `init` → scene `create`/`flush`/`dcs_*`/`set_geom_verts`/
`set_texmap_texels` → `draw_scene` → `readpixels`.
## What the capture proves / pins
- **Assembler validation:** every message reassembles with the source-derived framing
(incl. the mirrored bit30). Any mismatch shows up immediately as a bad length/action.
- **`vr_readpixels` layout (open item):** the request args and the reply chunking/pixel
word order the shipped lib expects — the one payload the source snapshot stubs.
- **bit30 semantics (open item):** whether FLYK's receive path requires `0x40000000` on
replies (we already set it) or ignores it.
Once the capture is clean, swap the stub responder for the real render path
(`vrboard` → dpl3-revive pipeline) and `vr_readpixels` returns actual frames.
## No build environment?
If DOSBox-X + a C++ toolchain isn't set up, the alternative is to compile the original
host comms (`VR_COMMS.C` + `LINKIO.C` + `DPL_HOST.C`, port I/O teed to a file) natively
and drive it with a `STARTDPL.C`-style scene — same framed output, no emulator. Heavier
(needs the DPL types to build); the socket bridge above is the lighter path.
+94
View File
@@ -0,0 +1,94 @@
#!/usr/bin/env python3
"""
analyze_scene.py -- decode the live scene graph FLYK streams to the board.
Parses the framed region of capture.raw.bin (skipping the raw .BTL boot), tracks
create/flush per handle, and dumps the DPL node structs (VIEW camera, DCS matrices,
INSTANCE placements, MATERIAL colours) plus the mystery 0x1c/0x1d ops -- so we can
feed the dpl3-revive renderer.
python analyze_scene.py [capture.raw.bin]
"""
import sys, struct
from vrboard import Assembler, A
TYPE = {2:'zone',3:'view',4:'instance',5:'dcs',6:'lmodel',7:'light',8:'object',
9:'lod',10:'geogroup',11:'geometry',12:'material',13:'texmap',14:'texture',15:'ramp'}
def floats(b, n=None):
n = n if n else len(b)//4
return list(struct.unpack_from('<%df' % n, b, 0))
def main():
path = sys.argv[1] if len(sys.argv) > 1 else "capture.raw.bin"
data = open(path, "rb").read()
# find framed start: first 0x40ff00NN length word past the BTL
start = None
for i in range(80000, len(data)-8):
w = struct.unpack_from('<I', data, i)[0]
if (w & 0xffff0000) == 0x40ff0000 and 8 <= (w & 0xffff) <= 1024:
act = struct.unpack_from('<I', data, i+4)[0]
if act in (A.init, A.code860, A.args860, A.data860, A.bss860):
start = i; break
print(f"framed region starts @ {start}")
asm = Assembler(); asm.feed(data[start:])
nodes = {} # handle -> {'type':.., 'body':..}
creates = []
op1c, op1d = [], []
for m in asm:
if m.iserver: continue
a, p = m.action, m.payload
if a == A.create:
typ = struct.unpack_from('<I', p, 0)[0]
h = struct.unpack_from('<I', p, 4)[0] if len(p) >= 8 else 0
nodes.setdefault(h, {})['type'] = typ
creates.append((h, typ))
elif a == A.flush:
h = struct.unpack_from('<I', p, 0)[0]
nodes.setdefault(h, {})['body'] = p
elif a == 0x1c: op1c.append(p)
elif a == 0x1d: op1d.append(p)
print(f"\nnodes: {len(nodes)} creates: {len(creates)} 0x1c: {len(op1c)} 0x1d: {len(op1d)}\n")
by_type = {}
for h, n in nodes.items():
by_type.setdefault(TYPE.get(n.get('type'), n.get('type')), []).append(h)
print("by type:", {k: len(v) for k, v in by_type.items()})
def show(h):
n = nodes[h]; t = TYPE.get(n.get('type'), n.get('type'))
body = n.get('body', b'')
print(f"\n--- handle {h:#x} type={t} body={len(body)}B ---")
# body = [remote:4][type_check:4][struct-specific...]
if len(body) >= 8:
rem, tc = struct.unpack_from('<II', body, 0)
print(f" remote={rem:#x} type_check={tc}")
rest = body[8:]
if t == 'dcs' and len(rest) >= 68: # node(4) + matrix(64)
mat = floats(rest[4:68], 16)
for r in range(4):
print(" [" + " ".join(f"{mat[r*4+c]:8.3f}" for c in range(4)) + "]")
elif t == 'view' and len(rest) >= 64:
mat = floats(rest[0:64], 16)
print(" camera matrix:")
for r in range(4):
print(" [" + " ".join(f"{mat[r*4+c]:8.3f}" for c in range(4)) + "]")
tail = floats(rest[64:64+13*4]) if len(rest) >= 64+52 else []
print(" view params (enable,x0,y0,x1,y1,zeye,xs,ys,hither,yon,bg3...):")
print(" ", [round(v,3) for v in tail])
else:
print(" floats:", [round(v,3) for v in floats(rest[:min(64,len(rest))])])
for t in ('view', 'dcs', 'instance', 'material'):
hs = by_type.get(t, [])
if hs:
show(hs[0])
if op1c:
print(f"\n0x1c sample ({len(op1c[0])}B):", op1c[0].hex())
if op1d:
p = op1d[0]; print(f"\n0x1d sample ({len(p)}B) as floats:", [round(v,3) for v in floats(p)])
if __name__ == '__main__':
main()
Binary file not shown.

After

Width:  |  Height:  |  Size: 101 KiB

+76
View File
@@ -0,0 +1,76 @@
#!/usr/bin/env python3
"""
bsl_pair.py -- decode the wire BSL slice-selector encoding by pairing:
wire side: mode-0 texmap pages + the texture nodes' selector words
file side: which .BSL file each page is (byte compare) + the bitslice values
(b2z tag 0x018) of every texture entry that references that .BSL
in the scene's .BGF/.BMF models.
python bsl_pair.py sdemo2.raw.bin c:\\temp\\flykc\\HPDAVE\\VIDEO
"""
import os, sys, struct, glob
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'parser'))
from vrboard import VirtualBoard, Assembler
from decode_anim import find_framed_start
import b2z
cap = sys.argv[1] if len(sys.argv) > 1 else 'sdemo2.raw.bin'
vid = sys.argv[2] if len(sys.argv) > 2 else r'c:\temp\flykc\HPDAVE\VIDEO'
data = open(cap, 'rb').read()
board = VirtualBoard(); asm = Assembler(); asm.feed(data[find_framed_start(data):])
for m in asm:
try: board.handle(m)
except Exception: pass
if board.frames >= 50: break
types = {h: nd.get('type') for h, nd in board.nodes.items()}
# wire: selectors per texmap
sel_by_texmap = {}
for h, nd in board.nodes.items():
if types.get(h) != 0xc:
continue
b = nd.get('body') or b''
if len(b) < 56:
continue
texm = struct.unpack_from('<I', b, 4)[0]
sel = struct.unpack_from('<I', b, 52)[0]
sel_by_texmap.setdefault(texm, []).append((h, sel))
# file: read every .BSL in the search dir
bsls = {}
for f in glob.glob(os.path.join(vid, '*.BSL')):
bsls[os.path.basename(f)] = open(f, 'rb').read()
# file: texture entries (name, mapfile, bitslice) from every .BGF/.BMF
filetex = {}
for f in glob.glob(os.path.join(vid, '*.BGF')) + glob.glob(os.path.join(vid, '*.BMF')):
try:
mdl = b2z.load(f)
except Exception:
continue
for name, t in mdl.textures.items():
if t.mapfile:
filetex.setdefault(t.mapfile.upper(), []).append(
(name, t.bitslice, os.path.basename(f)))
for texm, sels in sorted(sel_by_texmap.items()):
entry = board.tex.get(texm)
if entry is None or entry['mode'] != 0:
continue
page = bytes(entry['data'])
match = None
for bn, bd in bsls.items():
if bd[:len(page)] == page or page[:min(len(page), len(bd))] == bd[:min(len(page), len(bd))]:
match = bn
break
print(f"texmap {texm:#5x} {entry['u']}x{entry['v']} mode0 -> file {match}")
print(f" wire selectors: {sorted(set(s for _, s in sels))} "
f"({[hex(s) for _, s in sorted(sels)]})")
if match:
keys = [k for k in filetex if match.split('.')[0] in k]
for k in keys:
for name, bs, src in sorted(set(filetex[k])):
print(f" file: {name:24} bitslice={bs:3} (in {src}, map {k})")
Binary file not shown.
View File
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 9.8 KiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 5.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.0 KiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 KiB

Binary file not shown.
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 114 KiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 195 KiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 4.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.9 KiB

View File
View File
View File
View File
Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 556 B

+71
View File
@@ -0,0 +1,71 @@
#!/usr/bin/env python3
"""
census.py -- full inventory of a staged-run capture: create types, list_add tree,
instance/geogroup/material flush refs. Establishes the shipped node-type ids and
the object->lod->geogroup->geometry linkage the renderer walks.
python census.py cap5.raw.bin
"""
import sys, struct
from collections import defaultdict
sys.path.insert(0, '.')
from vrboard import Assembler, A
from decode_anim import find_framed_start
# shipped type ids (established cap3-cap5; 1994 ids differed from 7 up)
TYPE = {2:'zone',3:'view',4:'instance',5:'dcs',6:'lmodel',7:'object',8:'lod',
9:'geogroup',0xa:'geometry',0xb:'material',0xc:'texmap',0xd:'texture',
0xe:'light',0xf:'ramp'}
def main():
path = sys.argv[1] if len(sys.argv) > 1 else 'cap5.raw.bin'
data = open(path, 'rb').read()
asm = Assembler(); asm.feed(data[find_framed_start(data):])
types, flushes, edges = {}, defaultdict(list), []
extra = defaultdict(int)
for m in asm:
if m.iserver: continue
a, p = m.action, m.payload
if a == A.create and len(p) >= 8:
t, h = struct.unpack_from('<II', p, 0); types[h] = t
elif a == A.flush:
h = struct.unpack_from('<I', p, 0)[0]; flushes[h].append(p)
elif a in (A.dcs_nest, A.dcs_link, A.dcs_prune, A.list_add, A.list_remove):
x, y = struct.unpack_from('<II', p, 0); edges.append((A(a).name, x, y))
elif a in (A.draw_scene,) or a in (0x1d, 0x17, 0x19, 0x2a, 0x1c):
if a not in (9,): extra[a] += 0
else:
try: A(a)
except ValueError: extra[a] += 1
tn = lambda h: TYPE.get(types.get(h), f'?{types.get(h)}') + f':{h:#x}'
print('type census:', {TYPE.get(t, hex(t)): sum(1 for x in types.values() if x == t)
for t in sorted(set(types.values()))})
print('unknown extra actions:', {hex(k): v for k, v in extra.items()})
print('\n=== list_add / dcs edges ===')
for op, x, y in edges:
lhs = 'scene:0x0' if x == 0 else tn(x)
print(f' {op:9} {lhs:16} <- {tn(y)}')
print('\n=== per-type last-flush word dump (refs annotated) ===')
for h in sorted(flushes):
t = TYPE.get(types.get(h))
if t not in ('instance', 'geogroup', 'material', 'lod', 'object', 'texmap',
'texture', 'geometry'):
continue
body = flushes[h][-1]
n = len(body) // 4
wi = struct.unpack_from('<%dI' % n, body, 0)
wf = struct.unpack_from('<%df' % n, body, 0)
out = []
for i in range(2, n):
w, f = wi[i], wf[i]
if w in types: out.append(f'[{i}]={tn(w)}')
elif w == 0xffffffff: out.append(f'[{i}]=-1')
elif w and (w < 0x1000): out.append(f'[{i}]={w}')
elif w and abs(f) > 1e-6 and abs(f) < 1e6: out.append(f'[{i}]={f:.3f}')
print(f' {t:9} {h:#04x} ({len(body)}B x{len(flushes[h])}): ' + ' '.join(out))
if __name__ == '__main__':
main()
+69
View File
@@ -0,0 +1,69 @@
#!/usr/bin/env python3
"""debug_ident.py -- render a capture frame with each instance in a unique flat
color (no materials/textures) to identify what occupies the screen."""
import os, sys, struct
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
os.environ.setdefault('SDL_VIDEODRIVER', 'dummy')
import numpy as np
from vrboard import VirtualBoard, Assembler
from decode_anim import find_framed_start
import vrview
path = sys.argv[1] if len(sys.argv) > 1 else 'klng.raw.bin'
upto = int(sys.argv[2]) if len(sys.argv) > 2 else 900
out = sys.argv[3] if len(sys.argv) > 3 else 'ident.png'
data = open(path, 'rb').read()
board = VirtualBoard(); asm = Assembler(); asm.feed(data[find_framed_start(data):])
for m in asm:
try: board.handle(m)
except Exception: pass
if board.frames >= upto: break
r = vrview.Renderer()
c = r.cache; c.maybe_rebuild(board)
V = np.linalg.inv(r.chain_matrix(board, c.cam_chain))
W, H = r.w, r.h
img = np.zeros((H, W, 3), np.float32)
zbuf = np.full((H, W), np.inf, np.float32)
vp = {'x0': -1, 'y0': -0.625, 'x1': 1, 'y1': 0.625, 'zeye': 1.0, 'hither': 1.0}
import colorsys
legend = []
for i, inst in enumerate(c.instances):
M = r.chain_matrix(board, inst['chain']) @ V
R, T = M[:3, :3], M[3, :3]
col3 = np.array(colorsys.hsv_to_rgb((i * 0.37) % 1.0, 0.9, 1.0)) * 255
drew = False
for gh in inst['geoms']:
mesh = c._mesh(board, gh)
if mesh is None: continue
pe = mesh['pos'] @ R + T
z = -pe[:, 2]
ok = z > vp['hither']
if not ok.any(): continue
zs = np.where(ok, z, 1.0)
px = (pe[:, 0] * vp['zeye'] / zs - vp['x0']) / 2 * W
py = (1 - (pe[:, 1] * vp['zeye'] / zs - vp['y0']) / 1.25) * H
col = np.repeat(col3[None, :], len(pe), 0)
r._raster(mesh['tri'], px, py, z, ok, col, None, None, None, None,
np.zeros(3), img, zbuf, W, H)
drew = True
if drew:
legend.append((i, inst['chain'][0], [int(x) for x in col3]))
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'parser'))
import svt
rgba = np.zeros((H, W, 4), np.uint8); rgba[..., 3] = 255
rgba[..., :3] = np.clip(img, 0, 255).astype(np.uint8)
svt.write_png(out, W, H, rgba.tobytes())
print('wrote', out)
# which instances actually own visible pixels?
vis = {}
flat = rgba[..., :3].reshape(-1, 3)
for i, dcs, col in legend:
n = int(np.all(np.abs(flat.astype(int) - col) < 12, axis=1).sum())
if n > 200:
vis[i] = (hex(dcs), col, n)
for i, (dcs, col, n) in sorted(vis.items(), key=lambda kv: -kv[1][2]):
print(f'instance {i:3} dcs {dcs:>6} color={col} pixels={n}')
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
"""debug_view.py -- numeric check of vrview transforms against a capture replay."""
import os, sys, struct
import numpy as np
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
os.environ.setdefault('SDL_VIDEODRIVER', 'dummy')
from vrboard import VirtualBoard, Assembler
from decode_anim import find_framed_start
from vrview import Renderer, _mat_from_dcs
path = sys.argv[1] if len(sys.argv) > 1 else 'cap5.raw.bin'
data = open(path, 'rb').read()
board = VirtualBoard()
asm = Assembler(); asm.feed(data[find_framed_start(data):])
for m in asm:
try: board.handle(m)
except Exception: pass
if board.frames >= 600: break
r = Renderer(); r.cache.maybe_rebuild(board)
c = r.cache
print('cam chain', [hex(h) for h in c.cam_chain])
for h in (0x1, 0x2, 0x3):
b = board.nodes.get(h, {}).get('body') or b''
if len(b) >= 76:
M = _mat_from_dcs(b)
print(f'dcs {h:#x} diag={np.diag(M)[:3]} row3={M[3,:3]} anim={h in board.anim}')
V = np.linalg.inv(r.chain_matrix(board, c.cam_chain))
print('V row3 (eye offset):', V[3, :3])
for inst in c.instances:
M = r.chain_matrix(board, inst['chain'])
for gh in inst['geoms']:
mesh = c.meshes[gh]
pw = mesh['pos'] @ M[:3, :3] + M[3, :3]
pe = pw @ V[:3, :3] + V[3, :3]
z = -pe[:, 2]
vis = z > 8
if vis.any():
xp = pe[vis, 0] * 1.3 / z[vis]; yp = pe[vis, 1] * 1.3 / z[vis]
scr = f"proj x[{xp.min():7.2f},{xp.max():7.2f}] y[{yp.min():7.2f},{yp.max():7.2f}]"
else:
scr = "behind camera"
print(f"dcs={inst['chain'][0]:#04x} geom={gh:#04x} v={len(pw):4d} "
f"world y[{pw[:,1].min():8.1f},{pw[:,1].max():8.1f}] "
f"size={np.ptp(pw,0).round(0)} {scr}")
+125
View File
@@ -0,0 +1,125 @@
#!/usr/bin/env python3
"""
decode_anim.py -- full decode of the live animation stream (action 0x1d) plus the
scene-graph edges, from a framed capture of a FLYK.EXE run.
Answers, from capture.raw.bin:
1. exact 0x1d record layout + which handles it animates
2. the scene-graph edges (dcs_link/list_add/dcs_nest) -> who is the VIEW's DCS
3. creates in wire order (to map handles -> SHARKS.SCN entities by order)
4. per-frame trajectories of the animated handles (draw_scene = frame boundary)
python decode_anim.py [capture.raw.bin]
"""
import sys, struct
from collections import Counter, defaultdict
from vrboard import Assembler, A
TYPE = {2:'zone',3:'view',4:'instance',5:'dcs',6:'lmodel',7:'light',8:'object',
9:'lod',10:'geogroup',11:'geometry',12:'material',13:'texmap',14:'texture',15:'ramp'}
def find_framed_start(data):
for i in range(80000, len(data)-8):
w = struct.unpack_from('<I', data, i)[0]
if (w & 0xffff0000) == 0x40ff0000 and 8 <= (w & 0xffff) <= 1024:
act = struct.unpack_from('<I', data, i+4)[0]
if act in (A.init, A.code860, A.args860, A.data860, A.bss860):
return i
raise SystemExit("no framed start found")
def main():
path = sys.argv[1] if len(sys.argv) > 1 else "capture.raw.bin"
data = open(path, "rb").read()
start = find_framed_start(data)
asm = Assembler(); asm.feed(data[start:])
creates = [] # (handle, type) in wire order
flushes = defaultdict(list) # handle -> [body bytes]
edges = [] # (op, a, b) in wire order
anim = [] # (frame, handle, w1, 12 floats) for 0x1d
op1c = []
frame = 0
for m in asm:
if m.iserver: continue
a, p = m.action, m.payload
if a == A.create and len(p) >= 8:
typ, h = struct.unpack_from('<II', p, 0)
creates.append((h, typ))
elif a == A.flush:
h = struct.unpack_from('<I', p, 0)[0]
flushes[h].append(p)
elif a in (A.dcs_nest, A.dcs_link, A.dcs_prune, A.list_add, A.list_remove):
x, y = struct.unpack_from('<II', p, 0)
edges.append((A(a).name, x, y))
elif a == A.draw_scene:
frame += 1
elif a == 0x1d and len(p) >= 56:
h, w1 = struct.unpack_from('<II', p, 0)
fl = struct.unpack_from('<12f', p, 8)
anim.append((frame, h, w1, fl))
elif a == 0x1c:
op1c.append((frame, p))
types = {h: t for h, t in creates}
tname = lambda h: TYPE.get(types.get(h), f'?{types.get(h)}')
print(f"frames (draw_scene count): {frame}")
print(f"creates: {len(creates)} edges: {len(edges)} 0x1d: {len(anim)} 0x1c: {len(op1c)}")
print("\n=== creates in wire order ===")
for i, (h, t) in enumerate(creates):
print(f" {i:3} handle {h:#06x} {TYPE.get(t, t)}")
print("\n=== edges in wire order ===")
for op, x, y in edges:
print(f" {op:12} {x:#06x}({tname(x)}) {y:#06x}({tname(y)})")
print("\n=== instance bodies (as int words) ===")
for h, t in creates:
if TYPE.get(t) != 'instance': continue
body = flushes[h][-1] if flushes[h] else b''
words = struct.unpack_from('<%dI' % (len(body)//4), body, 0)
annot = []
for w in words[2:]:
if w in types: annot.append(f"{w:#x}={tname(w)}")
elif w == 0xffffffff: annot.append("-1")
elif w == 0: annot.append("0")
else: annot.append(f"{w:#x}")
print(f" inst {h:#06x}: [{', '.join(annot)}]")
print("\n=== 0x1d records ===")
hist = Counter(h for _, h, _, _ in anim)
w1s = Counter(w1 for _, _, w1, _ in anim)
print(" word1 values:", dict(w1s))
print(" targets:")
for h, n in hist.most_common():
print(f" handle {h:#06x} ({tname(h)}): {n} updates")
# per-target first/mid/last translation (assuming [3x3 rows][tx ty tz])
print("\n per-target trajectory (frame: t=(tx,ty,tz), mat row0):")
per = defaultdict(list)
for f, h, w1, fl in anim:
per[h].append((f, fl))
for h, lst in per.items():
print(f" handle {h:#06x} ({tname(h)}):")
for f, fl in (lst[0], lst[len(lst)//2], lst[-1]):
m = fl[:9]; t = fl[9:12]
print(f" frame {f:6}: t=({t[0]:9.3f},{t[1]:9.3f},{t[2]:9.3f}) "
f"m0=({m[0]:6.3f},{m[1]:6.3f},{m[2]:6.3f}) m1=({m[3]:6.3f},{m[4]:6.3f},{m[5]:6.3f}) "
f"m2=({m[6]:6.3f},{m[7]:6.3f},{m[8]:6.3f})")
# how many 0x1d per frame, and which handles per frame (sample a few frames)
byframe = defaultdict(list)
for f, h, w1, fl in anim:
byframe[f].append(h)
counts = Counter(len(v) for v in byframe.values())
print("\n 0x1d msgs per frame histogram:", dict(counts))
fr = sorted(byframe)[len(byframe)//2]
print(f" handles updated in frame {fr}: {[hex(h) for h in byframe[fr]]}")
print("\n=== flush counts per handle (re-flushed nodes = animated via flush?) ===")
for h, lst in sorted(flushes.items()):
if len(lst) > 1:
print(f" handle {h:#06x} ({tname(h)}): {len(lst)} flushes")
if __name__ == '__main__':
main()
+118
View File
@@ -0,0 +1,118 @@
#!/usr/bin/env python3
"""
decode_anim2.py -- second-pass decode: 0x1d split by word1 (the real handle),
full DCS body dump (matrix + parent/sibling/child tree), zone/view bodies,
init argv, 0x1c hex, and ASCII strings on the wire.
"""
import sys, struct
from collections import defaultdict
from vrboard import Assembler, A
from decode_anim import find_framed_start, TYPE
def main():
path = sys.argv[1] if len(sys.argv) > 1 else "capture.raw.bin"
data = open(path, "rb").read()
asm = Assembler(); asm.feed(data[find_framed_start(data):])
types = {}
flushes = defaultdict(list)
anim = defaultdict(list) # w1 -> [(frame, w0, floats)]
op1c = []
init_argv = None
frame = 0
for m in asm:
if m.iserver: continue
a, p = m.action, m.payload
if a == A.init:
init_argv = p
elif a == A.create and len(p) >= 8:
typ, h = struct.unpack_from('<II', p, 0)
types[h] = typ
elif a == A.flush:
h = struct.unpack_from('<I', p, 0)[0]
flushes[h].append(p)
elif a == A.draw_scene:
frame += 1
elif a == 0x1d and len(p) >= 56:
w0, w1 = struct.unpack_from('<II', p, 0)
fl = struct.unpack_from('<12f', p, 8)
anim[w1].append((frame, w0, fl))
elif a == 0x1c:
op1c.append((frame, p))
tname = lambda h: TYPE.get(types.get(h), f'?{types.get(h)}')
print("=== init argv ===")
print(repr(init_argv))
print("\n=== 0x1d split by word1 (candidate handle) ===")
for w1, lst in sorted(anim.items()):
print(f" w1={w1:#x} ({tname(w1)}): {len(lst)} records")
# sample across the run
n = len(lst)
for i in (0, n//4, n//2, 3*n//4, n-1):
f, w0, fl = lst[i]
m = fl[:9]; t = fl[9:12]
print(f" frame {f:6} w0={w0}: t=({t[0]:9.3f},{t[1]:9.3f},{t[2]:9.3f}) "
f"m=({m[0]:5.2f},{m[1]:5.2f},{m[2]:5.2f} | {m[3]:5.2f},{m[4]:5.2f},{m[5]:5.2f} | "
f"{m[6]:5.2f},{m[7]:5.2f},{m[8]:5.2f})")
# does it ever change?
first = lst[0][2]; changed = sum(1 for _, _, fl in lst if fl != first)
print(f" records differing from first: {changed}/{n}")
print("\n=== DCS bodies (last flush) as annotated words ===")
for h in sorted(flushes):
if tname(h) != 'dcs': continue
body = flushes[h][-1]
nw = len(body)//4
wi = struct.unpack_from('<%dI' % nw, body, 0)
wf = struct.unpack_from('<%df' % nw, body, 0)
# find plausible 4x4 matrix start: scan for identity-ish diag or just dump both
print(f" dcs {h:#04x} ({len(body)}B, {nw}w):")
ints = []
for i, w in enumerate(wi):
if w in types: ints.append(f"[{i}]{w:#x}={tname(w)}")
elif 0 < w < 0x10000: ints.append(f"[{i}]{w:#x}")
print(" int-ish:", " ".join(ints))
print(" floats:", " ".join(f"{v:.2f}" if abs(v) < 1e6 and v == v else "." for v in wf))
print("\n=== zone bodies ===")
for h in sorted(flushes):
if tname(h) != 'zone': continue
body = flushes[h][-1]
wi = struct.unpack_from('<%dI' % (len(body)//4), body, 0)
annot = [f"{w:#x}" + (f"={tname(w)}" if w in types else "") for w in wi]
print(f" zone {h:#04x}: [{', '.join(annot)}]")
print("\n=== view body (last flush) ===")
for h in sorted(flushes):
if tname(h) != 'view': continue
for k, body in enumerate(flushes[h]):
wf = struct.unpack_from('<%df' % (len(body)//4), body, 0)
wi = struct.unpack_from('<%dI' % (len(body)//4), body, 0)
print(f" view {h:#04x} flush {k} ({len(body)}B):")
print(" floats:", [round(v, 3) if abs(v) < 1e7 and v == v else hex(wi[i]) for i, v in enumerate(wf)])
print("\n=== lmodel/material bodies ===")
for h in sorted(flushes):
if tname(h) not in ('lmodel', 'material'): continue
body = flushes[h][-1]
wf = struct.unpack_from('<%df' % (len(body)//4), body, 0)
wi = struct.unpack_from('<%dI' % (len(body)//4), body, 0)
print(f" {tname(h)} {h:#04x} ({len(body)}B):",
[round(v, 3) if abs(v) < 1e7 and v == v else hex(wi[i]) for i, v in enumerate(wf)])
print("\n=== 0x1c messages ===")
for f, p in op1c:
print(f" frame {f}: {len(p)}B")
wf = struct.unpack_from('<%df' % (len(p)//4), p, 0)
wi = struct.unpack_from('<%dI' % (len(p)//4), p, 0)
print(" ", [round(v, 4) if abs(v) < 1e7 and v == v else hex(wi[i]) for i, v in enumerate(wf)])
print("\n=== ASCII strings (>=6 chars) in framed region ===")
import re
for mt in re.finditer(rb'[ -~]{6,}', data[find_framed_start(data):]):
print(" ", mt.group().decode('latin1'))
if __name__ == '__main__':
main()
Binary file not shown.

After

Width:  |  Height:  |  Size: 50 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 146 KiB

+48
View File
@@ -0,0 +1,48 @@
# flyk_vr.conf -- run the original FLYK.EXE against the virtual VelociRender board.
#
# 1. python patha\vrrun.py [prefix] (board, 127.0.0.1:8620)
# 2. $env:VRLINK='1'; G:\DOSBox-X\dosbox-x-vrlink.exe -conf patha\flyk_vr.conf
#
# C: is the staged asset tree (patha notes / stage step):
# c:\temp\flykc\HPDAVE = sda4\HPDAVE (FLYK.EXE, SHARKS.SCN, BTL, MNG)
# c:\temp\flykc\GEOMETRY = sda4\DPL3\GEOMETRY (satisfies SCN "GEOMETRY ..\geometry")
# c:\temp\flykc\TEXTURE = sda4\DPL3\TEXTURE (satisfies SCN "TEXTURE ..\texture")
# c:\temp\flykc\SHARKS = empty (no source dir survives on the drive)
# Staging exists because sda4 has no GEOMETRY/SHARKS siblings for HPDAVE: FLYK ran
# with dead search paths and built an empty world (object=NULL on every instance).
[sdl]
autolock=false
# PHYSICAL STICK MODE (default): native joystick mapping, as validated on
# BLADE. For KEYBOARD flight uncomment the two blocks below -- arrows = stick
# X/Y, A/Z = throttle, Space = trigger (the shipped RPL4.RES has ONLY a
# Thrustmaster VTV mapper; KEYBOARD-only L4CONTROLS crashes the game, so
# keyboard flight = DOSBox synthesizing the game-port stick from keys).
# NOTE: joysticktype=fcs remaps a physical stick's axes (resting throttle
# reads as constant deflection = steady rotation) -- don't mix modes, and
# delete the staged JOYSTICK.CAL after switching.
#mapperfile=c:\temp\flykc\keyjoy.map
#[joystick]
#joysticktype=fcs
[dosbox]
memsize=32
# pod network: NETNUB (wattcp over a packet driver) hard-requires a NIC --
# NE2000 emulation + the drive's ODI stack (LSL+NE2000+ODIPKT, loaded by
# run_game --netnub) recreate the 1996 BattleTech-center LAN inside DOSBox
[ne2000]
ne2000=true
nicbase=300
nicirq=3
backend=auto
[cpu]
core=dynamic
cycles=max
[autoexec]
mount c c:\temp\flykc
c:
call C:\RUNSCN.BAT
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.0 KiB

+69
View File
@@ -0,0 +1,69 @@
#!/usr/bin/env python3
"""
gen_launchers.py -- generate one Windows .bat per runnable staged scene
(patha\\launch\\RUN_<PRODUCT>_<SCENE>.BAT), each starting the virtual board +
window + DOSBox via run_demo.py. Also writes launch\\MENU.BAT (a picker).
Skips scenes in the MUNGA key=value dialect (they hang FLYK's parser) and the
known cwd-only loaders (HPDAVE BDEMO/SAND/TEMP).
python gen_launchers.py
"""
import os, re, glob
HERE = os.path.dirname(os.path.abspath(__file__))
STAGE = r"c:\temp\flykc"
OUT = os.path.join(HERE, "launch")
SKIP = {("HPDAVE", "BDEMO"), ("HPDAVE", "SAND"), ("HPDAVE", "TEMP")}
def munga_dialect(path):
"""MUNGA map-scenes use key=value lines (viewangle=60.0) FLYK can't read."""
try:
head = open(path, encoding='latin1').read(600)
except OSError:
return True
return bool(re.search(r'(?m)^\s*(viewangle|clip|fog|backgnd)\s*=', head))
def main():
os.makedirs(OUT, exist_ok=True)
entries = []
for prod in ("HPDAVE", "STDAVE", "RPDAVE", "BTDAVE", "FX"):
pdir = os.path.join(STAGE, prod)
if not os.path.isdir(pdir):
continue
for scn in sorted(glob.glob(os.path.join(pdir, "*.SCN"))):
name = os.path.splitext(os.path.basename(scn))[0].upper()
if (prod, name) in SKIP or munga_dialect(scn):
continue
# ambient particle sim only where it belongs (underwater HPDAVE)
sfx = " --sfx" if (prod, name) in {
("HPDAVE", "SHARKS"), ("HPDAVE", "SDEMO"), ("HPDAVE", "BDDEMO"),
("HPDAVE", "FISHSPLS"), ("HPDAVE", "BDPAL")} else ""
bat = os.path.join(OUT, f"RUN_{prod}_{name}.BAT")
with open(bat, "w") as fp:
fp.write(
"@echo off\r\n"
f"rem {prod}\\{name}.SCN on the virtual VelociRender board\r\n"
f"cd /d \"{HERE}\"\r\n"
f"python run_demo.py {name} --dir {prod} --prefix {name.lower()}{sfx}\r\n"
"pause\r\n")
entries.append((prod, name))
print(f" RUN_{prod}_{name}.BAT")
with open(os.path.join(OUT, "MENU.BAT"), "w") as fp:
fp.write("@echo off\r\nsetlocal enabledelayedexpansion\r\n:menu\r\ncls\r\n"
"echo === DPL3 virtual VelociRender scene launcher ===\r\necho.\r\n")
for i, (prod, name) in enumerate(entries, 1):
fp.write(f"echo {i:3}. {prod:8} {name}\r\n")
fp.write("echo.\r\nset /p pick=scene number (or Q): \r\n"
"if /i \"%pick%\"==\"Q\" exit /b\r\n")
for i, (prod, name) in enumerate(entries, 1):
fp.write(f"if \"%pick%\"==\"{i}\" call \"%~dp0RUN_{prod}_{name}.BAT\"\r\n")
fp.write("goto menu\r\n")
print(f"\n{len(entries)} launchers + MENU.BAT -> {OUT}")
if __name__ == "__main__":
main()
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
"""
heading_test.py -- identify the 0x1d matrix convention + shark nose axis by
correlating every candidate basis vector against the swim velocity across the
whole capture. The right candidate tracks velocity with a consistently high dot.
python heading_test.py [capture]
"""
import sys, struct
import numpy as np
sys.path.insert(0, '.')
from vrboard import Assembler
from decode_anim import find_framed_start
path = sys.argv[1] if len(sys.argv) > 1 else 'cap7.raw.bin'
data = open(path, 'rb').read()
asm = Assembler(); asm.feed(data[find_framed_start(data):])
recs = []
for m in asm:
if m.iserver: continue
if m.action == 0x1d and len(m.payload) >= 56:
h = struct.unpack_from('<I', m.payload, 4)[0]
if h != 1:
f = struct.unpack_from('<12f', m.payload, 8)
recs.append((np.array(f[:9]).reshape(3, 3), np.array(f[9:12])))
print(f'{len(recs)} shark 0x1d records')
cands = {}
for name, vec in (('row0', lambda M: M[0]), ('row2', lambda M: M[2]),
('col0', lambda M: M[:, 0]), ('col2', lambda M: M[:, 2])):
dots = []
for i in range(10, len(recs) - 10, 25):
M, t = recs[i]
_, t2 = recs[i + 5]
v = t2 - t
n = np.linalg.norm(v)
if n < 0.5: # skip near-stationary spans
continue
v = v / n
b = vec(M); b = b / np.linalg.norm(b)
dots.append(float(np.dot(v, b)))
dots = np.array(dots)
cands[name] = dots
print(f'{name}: mean={dots.mean():6.3f} median={np.median(dots):6.3f} '
f' frac>0.9={np.mean(dots > 0.9):.2f} frac<-0.9={np.mean(dots < -0.9):.2f}'
f' frac|.|<0.5={np.mean(np.abs(dots) < 0.5):.2f}')
+128
View File
@@ -0,0 +1,128 @@
#!/usr/bin/env python3
"""
hunt_actions.py -- static hunt for undecoded vr_action ids in shipped FLYK.EXE.
Method: parse the LE fixup tables to find code sites that reference a given
data-segment string (raw LE pages hold placeholders at fixup sites, so naive
scanning cannot work), then disassemble the surrounding function with capstone
and report the immediate constants loaded near velocirender_transmit calls --
the action id is among them.
python hunt_actions.py "get_geom_vertices" [path-to-exe]
"""
import os, sys, struct
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from leimage import LE, rd
EXE = sys.argv[2] if len(sys.argv) > 2 else \
r"s:\OneDrive\Tesla III\DaveMcCoy\sda4\HPDAVE\FLYK.EXE"
def parse_fixups(le):
"""Yield (src_linear, target_obj, target_off) for 32-bit offset fixups."""
with open(le.path, 'rb') as f:
# page fixup offset table: n_pages+1 dwords
pgtab = rd(f, le.fixup_pg_off, 4 * (le.n_pages + 1))
offs = struct.unpack('<%dI' % (le.n_pages + 1), pgtab)
recs_base = le.fixup_rec_off
# page -> linear address mapping: need object bases + page ranges
objs = rd(f, le.obj_tab_off, 24 * le.n_objects)
page_lin = {}
for i in range(le.n_objects):
vsize, base, flags, pgidx, pgcnt, _ = struct.unpack_from('<6I', objs, i * 24)
for k in range(pgcnt):
page_lin[pgidx + k] = base + k * le.page_size
out = []
for pg in range(1, le.n_pages + 1):
lo, hi = offs[pg - 1], offs[pg]
if hi <= lo:
continue
blob = rd(f, recs_base + lo, hi - lo)
p = 0
while p < len(blob) - 3:
src = blob[p]; flags = blob[p + 1]; p += 2
srcoffs = []
if src & 0x20: # source list
cnt = blob[p]; p += 1
else:
srcoffs.append(struct.unpack_from('<h', blob, p)[0]); p += 2
cnt = 0
tobj_big = flags & 0x40
if tobj_big:
tobj = struct.unpack_from('<H', blob, p)[0]; p += 2
else:
tobj = blob[p]; p += 1
stype = src & 0x0F
toff = 0
if stype != 0x02: # not 16-bit selector fixup
if flags & 0x10:
toff = struct.unpack_from('<I', blob, p)[0]; p += 4
else:
toff = struct.unpack_from('<H', blob, p)[0]; p += 2
if src & 0x20:
for _ in range(cnt):
so = struct.unpack_from('<h', blob, p)[0]; p += 2
srcoffs.append(so)
if stype == 0x07: # 32-bit offset fixup
for so in srcoffs:
if pg in page_lin and 0 <= so < le.page_size:
out.append((page_lin[pg] + so, tobj, toff))
return out
def main():
pat = sys.argv[1].encode('latin1') if len(sys.argv) > 1 else b"get_geom_vertices"
le = LE(EXE)
base1, code = le.image(1)
base2, data = le.image(2)
print(f"code obj1 @ {base1:#x} ({len(code)} bytes), data obj2 @ {base2:#x}")
# all occurrences of the pattern in the data object
targets = []
i = 0
while True:
i = data.find(pat, i)
if i < 0:
break
s = max(0, i - 60)
start = data.rfind(b'\x00', s, i) + 1
targets.append((start, data[start:data.find(b'\x00', i)]))
i += 1
if not targets:
print("string not found"); return
for off, s in targets:
print(f"data+{off:#x} (lin {base2+off:#x}): {s[:70]!r}")
fixups = parse_fixups(le)
print(f"{len(fixups)} 32-bit fixups parsed")
import capstone
md = capstone.Cs(capstone.CS_ARCH_X86, capstone.CS_MODE_32)
md.detail = False
for off, s in targets:
want = {base2 + off}
sites = [src for src, tobj, toff in fixups
if tobj == 2 and (base2 + toff) in want]
print(f"\n=== refs to {s[:50]!r}: {len(sites)} site(s)")
for src in sites:
# disassemble a window before the reference site
lo = max(base1, src - 0x90)
blob = code[lo - base1: src - base1 + 0x60]
print(f"-- code around {src:#x}:")
for ins in md.disasm(blob, lo):
mark = ' <== strref' if ins.address <= src < ins.address + ins.size else ''
if ins.mnemonic in ('mov', 'push', 'call', 'cmp') or mark:
txt = f" {ins.address:#x}: {ins.mnemonic} {ins.op_str}{mark}"
if ins.mnemonic in ('mov', 'push') and ',' in ins.op_str:
try:
imm = int(ins.op_str.split(',')[1].strip(), 0)
if 0 < imm < 0x60:
txt += f" ** small imm {imm:#x}"
except ValueError:
pass
print(txt)
if __name__ == '__main__':
main()
+97
View File
@@ -0,0 +1,97 @@
#!/usr/bin/env python3
"""
hunt_pe.py -- static protocol hunt for the Borland PE game binaries
(RPL4OPT/BTL4OPT under 32RTM), the PE counterpart of hunt_actions.py (LE).
Find an error-string's data VA, locate the code xref (32-bit immediate),
and disassemble the surrounding function to read off the protocol constants
(expected reply actions, request sizes) that the DOS-side source never had.
python hunt_pe.py <exe> <needle-string> [--back N] [--fwd N]
"""
import struct
import sys
import capstone
def pe_sections(data):
e_lfanew = struct.unpack_from('<I', data, 0x3c)[0]
machine, nsec = struct.unpack_from('<HH', data, e_lfanew + 4)
opt_size = struct.unpack_from('<H', data, e_lfanew + 20)[0]
image_base = struct.unpack_from('<I', data, e_lfanew + 24 + 28)[0]
secs = []
off = e_lfanew + 24 + opt_size
for i in range(nsec):
name = data[off:off + 8].rstrip(b'\0').decode('latin1')
vsize, va, rsize, roff = struct.unpack_from('<4I', data, off + 8)
flags = struct.unpack_from('<I', data, off + 36)[0]
secs.append({'name': name, 'va': va, 'vsize': vsize,
'roff': roff, 'rsize': rsize, 'flags': flags})
off += 40
return image_base, secs
def file_to_va(secs, base, foff):
for s in secs:
if s['roff'] <= foff < s['roff'] + s['rsize']:
return base + s['va'] + (foff - s['roff'])
return None
def va_to_file(secs, base, va):
rva = va - base
for s in secs:
if s['va'] <= rva < s['va'] + max(s['vsize'], s['rsize']):
return s['roff'] + (rva - s['va'])
return None
def hunt(path, needle, back=0x120, fwd=0x60):
data = open(path, 'rb').read()
base, secs = pe_sections(data)
print(f"{path}: base {base:#x}, sections "
+ " ".join(f"{s['name']}@{s['va']:#x}" for s in secs))
hits = []
i = data.find(needle.encode('latin1'))
while i != -1:
hits.append(i); i = data.find(needle.encode('latin1'), i + 1)
if not hits:
raise SystemExit(f"string {needle!r} not found")
md = capstone.Cs(capstone.CS_ARCH_X86, capstone.CS_MODE_32)
md.detail = False
for h in hits:
# xrefs point at the start of the full string literal, which may begin
# before the needle -- scan back to the previous NUL
s0 = h
while s0 > 0 and data[s0 - 1] != 0:
s0 -= 1
va = file_to_va(secs, base, s0)
print(f"\nstring @ file {s0:#x} va {va:#x}: "
f"{data[s0:s0+70].rstrip(bytes(1))!r}")
if va is None:
continue
pat = struct.pack('<I', va)
j = data.find(pat)
while j != -1:
code_va = file_to_va(secs, base, j)
sec = next((s for s in secs if s['roff'] <= j < s['roff'] + s['rsize']), None)
if sec and code_va and (sec['flags'] & 0x20000000 or sec['name'] in ('.text', 'CODE')):
start = max(j - back, sec['roff'])
code = data[start:j + fwd]
print(f"--- xref imm @ file {j:#x} va {code_va:#x} ({sec['name']}) ---")
for ins in md.disasm(code, file_to_va(secs, base, start)):
mark = " <-- xref" if ins.address <= code_va < ins.address + ins.size else ""
print(f" {ins.address:#010x} {ins.mnemonic:8} {ins.op_str}{mark}")
j = data.find(pat, j + 1)
if __name__ == '__main__':
argv = [a for a in sys.argv[1:] if not a.startswith('--')]
kw = {}
for k in ('back', 'fwd'):
if f'--{k}' in sys.argv:
kw[k] = int(sys.argv[sys.argv.index(f'--{k}') + 1], 0)
hunt(argv[0], argv[1], **kw)
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env python3
"""
hybrid_render.py -- Path A rendering (hybrid): render the scene FLYK.EXE is running,
from an in-scene camera, using the dpl3-revive pipeline.
FLYK streams the scene GRAPH (camera projection, DCS transforms, instance placements,
materials) over the VelociRender link, and animates it per-frame via action 0x1d
(decoded by analyze_scene.py). The GEOMETRY is referenced by name and loaded from files
-- exactly what dpl3-revive already parses. So we assemble the scene from files and view
it from the camera FLYK uses (a diver inside the reef), producing the picture FLYK's dead
i860 board would have drawn.
python hybrid_render.py [showcase_index] [out.png]
Default: SHARKS.SCN (the Hull-Pressure aquarium reef), interior camera on the shark.
"""
import sys, os
HERE = os.path.dirname(os.path.abspath(__file__))
VIEWER = os.path.join(os.path.dirname(HERE), "viewer")
sys.path.insert(0, VIEWER)
sys.path.insert(0, os.path.join(os.path.dirname(HERE), "parser"))
import bundle, render_preview
# Interior cameras (eye, center, up) framing the animated content of each showcase scene.
# For SHARKS the fish/sharks cluster near (-220, 225, -350) -- matches FLYK's live 0x1d
# translations (y~230, z~-200), confirming the camera looks where FLYK animates.
CAMS = {
2: {"eye": [450, 300, 250], "center": [-220, 225, -350], "up": [0, 1, 0]}, # SHARKS
}
def main():
idx = int(sys.argv[1]) if len(sys.argv) > 1 else 2
out = sys.argv[2] if len(sys.argv) > 2 else os.path.join(HERE, "flyk_render.png")
md = bundle.build_model(bundle.SHOWCASE[idx])
render_preview.render(md, out, cam=CAMS.get(idx))
if __name__ == "__main__":
main()
Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 KiB

+126
View File
@@ -0,0 +1,126 @@
@echo off
setlocal enabledelayedexpansion
:menu
cls
echo === DPL3 virtual VelociRender scene launcher ===
echo.
echo 1. HPDAVE BATEST
echo 2. HPDAVE BDDEMO
echo 3. HPDAVE BDPAL
echo 4. HPDAVE BIGDEMO
echo 5. HPDAVE DEMOREC
echo 6. HPDAVE DUANE
echo 7. HPDAVE FISHSPLS
echo 8. HPDAVE NEWLIGHT
echo 9. HPDAVE NL1
echo 10. HPDAVE NL2
echo 11. HPDAVE NL3
echo 12. HPDAVE PHIL1
echo 13. HPDAVE PHIL2
echo 14. HPDAVE PHIL3
echo 15. HPDAVE PT2
echo 16. HPDAVE SANDTEST
echo 17. HPDAVE SDEMO
echo 18. HPDAVE SHARKS
echo 19. HPDAVE TMPRIGS
echo 20. STDAVE BLACK
echo 21. STDAVE DIVCAL
echo 22. STDAVE KLNGVID
echo 23. STDAVE TREK
echo 24. STDAVE TREKVID
echo 25. RPDAVE BLADE
echo 26. RPDAVE DESTEST
echo 27. RPDAVE ONETHING
echo 28. RPDAVE RAVTEST
echo 29. BTDAVE DTEST
echo 30. BTDAVE FXTEST
echo 31. BTDAVE POLAR4
echo 32. FX AFC
echo 33. FX AFCDAY
echo 34. FX AFCNITE
echo 35. FX BIGBOOM
echo 36. FX BOOMDAY
echo 37. FX BOOMNITE
echo 38. FX BTFX
echo 39. FX BTFX2
echo 40. FX CALASP
echo 41. FX DETHDAY
echo 42. FX DETHNITE
echo 43. FX DTEST
echo 44. FX FIREDAY
echo 45. FX FIRENITE
echo 46. FX FLUSH
echo 47. FX FOGTEST
echo 48. FX FXTEST
echo 49. FX LITTEST
echo 50. FX MACHDAY
echo 51. FX MACHNITE
echo 52. FX MISBMDAY
echo 53. FX MISDAY
echo 54. FX MISNITE
echo 55. FX NBOOM
echo 56. FX RAIN
echo 57. FX TEST
@@ -0,0 +1,5 @@
@echo off
rem BTDAVE\DTEST.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
@@ -0,0 +1,5 @@
@echo off
rem BTDAVE\FXTEST.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
@@ -0,0 +1,5 @@
@echo off
rem BTDAVE\POLAR4.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
+5
View File
@@ -0,0 +1,5 @@
@echo off
rem FX\AFC.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
@@ -0,0 +1,5 @@
@echo off
rem FX\AFCDAY.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
@@ -0,0 +1,5 @@
@echo off
rem FX\AFCNITE.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
@@ -0,0 +1,5 @@
@echo off
rem FX\BIGBOOM.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
@@ -0,0 +1,5 @@
@echo off
rem FX\BOOMDAY.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
@@ -0,0 +1,5 @@
@echo off
rem FX\BOOMNITE.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
+5
View File
@@ -0,0 +1,5 @@
@echo off
rem FX\BTFX.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
@@ -0,0 +1,5 @@
@echo off
rem FX\BTFX2.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py BTFX2 --dir FX --prefix btfx2
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\CALASP.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py CALASP --dir FX --prefix calasp
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\DETHDAY.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py DETHDAY --dir FX --prefix dethday
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\DETHNITE.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py DETHNITE --dir FX --prefix dethnite
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\DTEST.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py DTEST --dir FX --prefix dtest
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\FIREDAY.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py FIREDAY --dir FX --prefix fireday
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\FIRENITE.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py FIRENITE --dir FX --prefix firenite
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\FLUSH.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py FLUSH --dir FX --prefix flush
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\FOGTEST.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py FOGTEST --dir FX --prefix fogtest
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\FXTEST.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py FXTEST --dir FX --prefix fxtest
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\LITTEST.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py LITTEST --dir FX --prefix littest
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\MACHDAY.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py MACHDAY --dir FX --prefix machday
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\MACHNITE.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py MACHNITE --dir FX --prefix machnite
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\MISBMDAY.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py MISBMDAY --dir FX --prefix misbmday
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\MISDAY.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py MISDAY --dir FX --prefix misday
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\MISNITE.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py MISNITE --dir FX --prefix misnite
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\NBOOM.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py NBOOM --dir FX --prefix nboom
pause
+5
View File
@@ -0,0 +1,5 @@
@echo off
rem FX\RAIN.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py RAIN --dir FX --prefix rain
pause
+5
View File
@@ -0,0 +1,5 @@
@echo off
rem FX\TEST.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py TEST --dir FX --prefix test
pause
@@ -0,0 +1,5 @@
@echo off
rem FX\TORCHFIR.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py TORCHFIR --dir FX --prefix torchfir
pause
@@ -0,0 +1,5 @@
@echo off
rem HPDAVE\BATEST.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py BATEST --dir HPDAVE --prefix batest
pause
@@ -0,0 +1,5 @@
@echo off
rem HPDAVE\BDDEMO.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py BDDEMO --dir HPDAVE --prefix bddemo --sfx
pause
@@ -0,0 +1,5 @@
@echo off
rem HPDAVE\BDPAL.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py BDPAL --dir HPDAVE --prefix bdpal --sfx
pause
@@ -0,0 +1,5 @@
@echo off
rem HPDAVE\BIGDEMO.SCN on the virtual VelociRender board
cd /d "S:\OneDrive\Tesla III\DaveMcCoy\dpl3-revive\patha"
python run_demo.py BIGDEMO --dir HPDAVE --prefix bigdemo
pause

Some files were not shown because too many files have changed in this diff Show More