CydandClaude Fable 5
afc3fd839e
Bring the graphics-dev collaborator's dpl3-revive into the repo as first-class
project code (they've handed it off; it's ours now). This is the proven
Division renderer that our in-process rt_draw has been trying to be.
What's here:
- parser/ B2Z/V2Z/SVT/SCN/SPL/BGF/BMF/BSL decoders (pure Python).
- spec/ reverse-engineered format + the definitive VelociRender wire
protocol (from the original DIVISION source, matches our live
VPX node/action tables exactly).
- source-ref/ read-only copies of the original DIVISION C (BIZREAD.C,
DPLTYPES.H, DPL.H) that define the formats.
- patha/ the "virtual VelociRender board": vrboard.py (24-action protocol
server), vrview.py (numpy software rasterizer, the reference),
vrview_gl.py (moderngl GPU backend, 832x512@60Hz), plus the
run/replay/regress tooling and evidence renders. Drives FLYK/BLADE/
Star Trek demos AND our btl4opt/rpl4opt game binaries.
- viewer/ WebGL archive generators (.py); prebuilt HTML/data regeneratable.
- samples/ small test models/textures.
- bt*.raw.bin real BTL4OPT arena wire captures (kept for offline renderer
testing/regression against OUR game).
.gitignore keeps the multi-hundred-MB demo capture dumps + debug logs +
regeneratable viewer bundles out of history (they stay on disk).
Phase 0 of the integration is validated: their board decodes our bt8 capture
with zero errors (3748 nodes, 507 instances, 4 mechs) and renders our arena
(terrain/dome/sky, correct Division DAC gamma). Plan + status in memory;
integration continues in emulator/RENDERER-COLLAB.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
136 lines
7.6 KiB
Markdown
136 lines
7.6 KiB
Markdown
# dpl3-revive
|
||
|
||
Porting the Virtual World Entertainment **DPL3** renderer (BattleTech / Red Planet
|
||
arcade pods, c. 1994–97) to modern hardware. The original ran on an Intel **i860**
|
||
graphics coprocessor driven by an x86 DOS host; the i860 and its PixelPlanes-5-style
|
||
board are long gone, so the plan is a **reimplementation** on a modern GPU, not a
|
||
recompile.
|
||
|
||
The source drive is treated as a **read-only archive**. Nothing here writes back to
|
||
it; the reference source and sample models below were *copied out* of it.
|
||
|
||
## Status
|
||
|
||
**Milestones 1 & 2a — asset formats decoded.** The `.B2Z` / `.V2Z` binary geometry
|
||
format and the `.SVT` texture format are both fully reverse-engineered, with
|
||
working, validated parsers. Models export to OBJ; textures export to PNG. The art
|
||
is now recoverable independent of the dead engine.
|
||
|
||
Roadmap:
|
||
1. ✅ **B2Z/V2Z geometry format** — spec + parser + OBJ export.
|
||
2a. ✅ **SVT texture format** — spec + parser + PNG export (`0BGR`, size-inferred dims).
|
||
3. ✅ **GPU viewer** — self-contained WebGL page renders textured models on modern
|
||
hardware (`viewer/`). Proves the whole art pipeline end-to-end.
|
||
2b. ✅ **SCN scene format** — plain-text scenes parsed + assembled into world space
|
||
(models placed via the exact DPL transform), textured, fogged, and rendered.
|
||
`CANAL.SCN` (47 objects) explorable in the viewer.
|
||
4. ✅ **SPL camera splines** — cubic-Hermite fly-through paths evaluated and animated.
|
||
`RAPTOR.SCN` + `CAMERA.SPL` play back the original 1994 flyover in the viewer.
|
||
4b. ✅ **Scene gallery** — the viewer now hosts several recovered worlds (Raptor
|
||
flyover, Red Planet canal, underwater reef). Fish are textured by the
|
||
**model-name** convention (`shark` → `SHARK.SVT`) when the material carries none.
|
||
4c. ✅ **Historical archive** — every resolvable scene (~60) **and every binary model
|
||
(~264)** batch-built and browsable in a searchable, lazy-loading viewer
|
||
(`archive.html` + `serve.py`) with Scenes/Models tabs. Camera has mouse orbit +
|
||
**WASD fly movement** (Q/E for down/up). A large array to judge render/colour.
|
||
5. ✅ **Material libraries + output gamma** — authoritative per-material colour+texture
|
||
from each model's `.V2Z` (`DIFFUSE × texture`), detail textures as luminance, and
|
||
the Division card's 10-bit-DAC **gamma-1.7** curve (`GAMMA.C`) for the authentic
|
||
pastel look. Red rock / grey concrete columns / red pipes now match the original.
|
||
5b. ✅ **Texture V orientation** — the original renderer samples v=0 at SVT row 0
|
||
(no flip). Fixed a wrong global V-flip that inverted mech cockpits/pods and fish
|
||
shading. Also decoded GLOMM bit-plane texture packs (`*ALL.SVT`) — see
|
||
`spec/SVT_FORMAT.md`.
|
||
5d. ✅ **Game maps: MAP / MOD** — the real playable worlds (BattleTech arenas, Red
|
||
Planet tracks) from the game trees' `MAPS/` dirs: INI-style maps placing `.MOD`
|
||
game objects (→ BGF video geometry) with nesting, includes, and pod dropzones
|
||
(drawn as red markers). Also fixed the 1996 **LOD-container** BGF layout, which
|
||
unlocked ~850 game models that previously parsed empty. Archive: **142 scenes +
|
||
41 maps + 1,462 models**, with a Maps tab in the viewer.
|
||
5c. ✅ **Game-side formats: BGF / BMF / VGF / VMF / BSL** — the game pipeline turns
|
||
out to be the same BIZ2/VIZ2 family: `.BGF` geometry, `.BMF`/`.VMF` material
|
||
libraries (namespaced refs like `stships:eblu4_mtl`), plus `DIV-BSL2` bit-slice
|
||
texture packs (texture tag `0x018` = BITSLICE). This unlocked **Star Trek** (the
|
||
"Tesla" Galaxy-class starship + Klingon cruiser, hull textures from `STEH.BSL`)
|
||
and the BattleTech / Red Planet / Hull Pressure game content. Archive now holds
|
||
**142 scenes + 608 models** across all product lines.
|
||
6. ⬜ Port the DPL3 geometry pipeline (transform / light / cull / LOD) from the C.
|
||
7. ⬜ Replace the PXPL5 rasterizer backend with the GPU.
|
||
|
||
### Path A — run the original binaries unmodified (alternative track)
|
||
|
||
The full original DPL3 engine source is on the archive drive (`sda4/DPL3/`), both the
|
||
DOS host side and the i860 board side. That means the **VelociRender wire protocol** the
|
||
host uses to drive the (dead) i860 board is fully documented in C — no disassembly. Path
|
||
A stands up a *virtual VelociRender board*: emulate the INMOS C012 link adapter (six I/O
|
||
ports at `0x150`) as a **DOSBox-X device**, answer the 24-command protocol, and use the
|
||
renderer this project already built as the board's guts — so `FLYK.EXE` and the other
|
||
HPDAVE/DPL3 binaries run **unmodified**, renderer transparently swapped underneath.
|
||
Spec + architecture: `spec/VELOCIRENDER_PROTOCOL.md`.
|
||
|
||
## Layout
|
||
|
||
```
|
||
spec/ format documentation
|
||
B2Z_FORMAT.md the reverse-engineered DIV-BIZ2 geometry spec
|
||
SVT_FORMAT.md the reverse-engineered SVT texture spec
|
||
SCN_FORMAT.md the reverse-engineered SCN scene-format spec
|
||
SPL_FORMAT.md the reverse-engineered SPL camera-spline spec
|
||
parser/ the decoders (pure Python 3, no deps)
|
||
b2z.py geometry reader + triangulating OBJ exporter
|
||
svt.py texture reader + PNG exporter
|
||
scn.py scene-file reader + the DPL instance-transform math
|
||
spl.py camera-spline evaluator (cubic Hermite) -> camera frames
|
||
samples/ small test models/textures from the archive (+ exported .obj/.png)
|
||
source-ref/ read-only copies of the original DIVISION source that define the format
|
||
viewer/ the WebGL viewers
|
||
bundle.py packs a few curated scenes into one self-contained HTML
|
||
dpl3-viewer.html the curated gallery (open in any browser; shareable)
|
||
archive.py batch-builds EVERY resolvable scene -> data/*.json
|
||
archive.html the historical archive viewer (searchable, lazy-loads scenes)
|
||
serve.py local launcher for the archive (needed for fetch())
|
||
data/ one JSON per scene + manifest.json (~60 scenes)
|
||
render_preview.py CPU rasterizer used to verify the pipeline
|
||
preview_*.png verification renders
|
||
```
|
||
|
||
## Usage
|
||
|
||
```sh
|
||
python parser/b2z.py dump samples/TESTBOX.B2Z # structural dump
|
||
python parser/b2z.py obj samples/BALL.B2Z out.obj # export triangulated OBJ
|
||
python parser/svt.py info samples/SHARK.SVT # texture dimensions/stats
|
||
python parser/svt.py png samples/SHARK.SVT out.png # export texture to PNG
|
||
python parser/scn.py "<archive>/DPL3RLS/SCENES/CANAL.SCN" # dump a scene's contents
|
||
python viewer/bundle.py # (re)build viewer/dpl3-viewer.html
|
||
```
|
||
|
||
Then open `viewer/dpl3-viewer.html` in any browser — drag to orbit, scroll to zoom,
|
||
switch models. It's fully self-contained (geometry inlined, textures as base64).
|
||
|
||
### The historical archive (all ~60 scenes)
|
||
|
||
```sh
|
||
python viewer/archive.py # (re)build data/*.json for every resolvable scene
|
||
python viewer/serve.py # serve locally + open the archive viewer
|
||
```
|
||
|
||
`serve.py` opens `http://localhost:8765/archive.html` — **Scenes** and **Models**
|
||
tabs, each a searchable list **grouped by product** (Red Planet, BattleTech, Hull
|
||
Pressure, Star Trek, Milestone, Engine/Demo). Camera: **drag** to look, **scroll**
|
||
to zoom, **WASD** to fly, **Q/E** down/up. A live **gamma slider** (default 1.25)
|
||
tunes the Division-DAC brightness. A local server is required because browsers
|
||
block `fetch()` of local files over `file://`.
|
||
|
||
Rebuild subsets with `python viewer/archive.py --scenes-only` / `--models-only`;
|
||
re-tag products (no data rebuild) with `--reclassify`.
|
||
|
||
The exported `.obj` files open in any modern 3D tool (Blender, MeshLab, etc.);
|
||
the `.png` textures open anywhere.
|
||
|
||
## Provenance
|
||
|
||
Format decoded from `source-ref/BIZREAD.C` — "Copyright DIVISION Limited (c) 1994,
|
||
author PJA" (Phil Atkin). Data model in `source-ref/DPLTYPES.H`. See
|
||
`spec/B2Z_FORMAT.md` §10 for the validation evidence.
|