CydandClaude Opus 4.8
99652d894b
The full DPL hierarchy the game uses (vs flyk's flat scene) is now decoded and rendered: - stride-aware set_geom_verts (header word 3 = floats/vertex: 3/4/5/8/9; mech meshes carry normals + UVs) - instances are list_add children of DCS nodes; instance flush field 4 -> object; object->lod->geogroup->geometry; dcs_link builds the articulation tree of 4x4s (payload floats 4..19, row-major, row 3 = translation) - game world is y-down (DCS matrices carry a reflection); projection flips x (Division mirror) and y render_game.py reconstructs a captured game stream offline: the mission arena (10km, 246 instances, 330 geometries), the player's Thor at the camera, six enemy mechs 1.5km north -- game-mech-decoded.png shows one with real hull/armor/glass materials; game-cockpit-decoded.png the cockpit view. The live backend (vpxlog.cpp) gained the same traversal and now draws the game's out-the-window view in real time (game-live-gl.png): sky, arena floor to the horizon, own gun barrels at frame bottom. Next: texturing (action-26 texel maps + UVs), lighting from wire normals, per-frame articulation once the RIO drives the sim. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
217 lines
11 KiB
Markdown
217 lines
11 KiB
Markdown
# Phase 3 — Render Backend: Progress
|
||
|
||
**Status (2026-07-03): Phase 3a + 3b + 3c complete. 3a — the render command
|
||
stream is fully captured and decoded, and a captured DIVRGB frame
|
||
reconstructed to pixels offline. 3b — a live OpenGL window built into the
|
||
emulated board draws each frame in real time inside DOSBox-X. 3c — the
|
||
production `vr_sync` abort is fixed and the full BattleTech v4.10 game now runs
|
||
indefinitely through the emulated board, drawing 1000s of frames (was hard-
|
||
capped at 200). First images ever produced from the Rel 4.10 VPX protocol
|
||
without a real board. Next (3d): traverse the game's DPL hierarchy + DCS
|
||
transforms so its mechs actually appear.**
|
||
|
||
|
||
|
||
That is `flyk divrgb.scn` — Division's SMPTE color-bar calibration scene —
|
||
rendered *entirely from the FIFO command stream captured by the emulated VPX
|
||
board*: geometry, connectivity, materials, camera, viewport and background all
|
||
come from the wire, none from the scene files. `divrgb-frame0.png` is the same
|
||
capture drawn with the actual frame-0 spline camera the app sent.
|
||
|
||
## 3a. Full FIFO capture
|
||
|
||
`vpxlog.cpp` now records every FIFO burst when `VPX_FIFODUMP=<path>` is set
|
||
(records: `'VPXM'` magic, u32 length, raw bytes; one record per burst between
|
||
`outputData` tag writes). The capture used here: `divrgb.fifodump`
|
||
(1297 messages) from `divrgb.conf`.
|
||
|
||
Tools:
|
||
|
||
- `decode_fifodump.py <dump> [--hex N] [--action A]` — action census + payload
|
||
hexdumps.
|
||
- `render_capture.py <dump> [-o out.png] [--frame N] [--eye x,y,z]` —
|
||
reconstructs the scene graph and software-renders the frame each
|
||
`draw_scene` commits.
|
||
|
||
## The Rel 4.10 wire protocol (established from this capture)
|
||
|
||
One burst per message: `[action:4][payload]`, packetized at 508 bytes.
|
||
The i860 image download itself rides the FIFO in this build (actions 18–20,
|
||
`VREND.MNG`). The DPL3 `vr_action` enum holds for 0–23 but Rel 4.10 extends
|
||
and re-purposes the tail:
|
||
|
||
| action | meaning | payload |
|
||
|---|---|---|
|
||
| 0 | init | args string (`/device~0x150~/video~svga~…`) |
|
||
| 1 | create | `[type][name]` — **the host assigns node names** (1,2,3…); the board's create reply node value is ignored |
|
||
| 3 | flush | `[name][type][node struct]` (see node types below) |
|
||
| 7 | dcs_link | `[parent][child]` |
|
||
| 9 | draw_scene | `[view?][0][0][1][1.0f]` — commits the frame |
|
||
| 11 | list_add | `[parent][child]` |
|
||
| 18–21 | 860code/data/bss/args | i860 download (FIFO in this build) |
|
||
| 23 | **set_geom_verts** (Rel4.10; DPL3 had 22) | hdr `[name][0][n_verts][3][n_blocks][1][5][n_verts][1.0f]`, then float32 x,y,z per vertex |
|
||
| 25 | **set_geom_conns** (new) | hdr `[name][n_polys][loop_len][0]`, then indices; each poly a closed loop (last=first) |
|
||
| 31 | **camera** (new) | `[?][view][3×3 rotation, row-major][eye x,y,z]` per frame |
|
||
| 45 (0x2D) | sync | token ping (see PHASE2) |
|
||
|
||
Node types (from create/flush pairs in this capture):
|
||
|
||
| type | node | flush payload highlights |
|
||
|---|---|---|
|
||
| 2 | texture/ramp? | one word, `0xFFFFFFFF` / `0x0FFFFFFF` |
|
||
| 3 | view | window l,b,r,t (±1 × ±0.6154), window-plane distance 1.3, viewport 832×512, near 2, far 12000, background RGB |
|
||
| 4 | light | |
|
||
| 5 | dcs | 4×4 matrix |
|
||
| 6 | material (old-style) | RGB floats |
|
||
| 7 | object | |
|
||
| 8 | lod | 2×4 bounds + lists |
|
||
| 9 | geogroup | bounds; **payload int 14 = material node name** |
|
||
| 10 | geometry | `[geo_type][n_polys][n_verts]…[radius]` |
|
||
| 11 | material | **floats 10–12 (payload) = diffuse RGB** |
|
||
|
||
Graph: `list_add` links object→lod→geogroup→geometry (and zone-level nodes to
|
||
node 0); geogroups bind materials by name. The 13 DIVRGB bars decode to the
|
||
textbook SMPTE pattern — 7 color bars, the reverse strip, and the −I/white/+Q
|
||
PLUGE row — which validates vertices, connectivity, materials, camera and
|
||
window mapping in one image. (Division screen x runs opposite to a GL-style
|
||
eye space; without negating x the pattern comes out mirrored.)
|
||
|
||
## 3b. Live OpenGL backend (DONE)
|
||
|
||
|
||
|
||
`VPX_RENDER=1` turns on a live render backend built into `vpxlog.cpp`
|
||
(Windows/WGL). The same burst assembler that feeds `VPX_FIFODUMP` now also
|
||
feeds an in-process scene store (`scene_burst()` → `struct VScene`), and each
|
||
`vr_draw_scene` publishes a frame snapshot to a dedicated OpenGL window thread
|
||
(`rt_main`) that draws it with fixed-function GL. No build-system change was
|
||
needed — `opengl32` was already linked, and `vpxlog.cpp` is already in
|
||
`hardware/`.
|
||
|
||
Validated: `flyk divrgb.scn` (`divrgb.conf` with `VPX_RENDER=1`) opens the
|
||
"VPX VelociRender (emulated)" window and draws the SMPTE bars live, framed by
|
||
the real `camera.spl` spline camera the app sends — matching the offline
|
||
`render_capture.py` decode. The window runs on its own thread, so it keeps
|
||
displaying the last frame after the DOS app exits.
|
||
|
||
Design notes:
|
||
- Multi-burst payloads (`set_geom_verts`/`conns` continuations) are tracked
|
||
with `geom_active` / `conn_active` state, same as the offline decoder.
|
||
- Projection: `glFrustum` from the view-node window rect scaled by
|
||
`near/window_distance`; a `glScalef(-1,1,1)` handles Division's mirrored
|
||
screen-x. Camera is the row-major 3×3 from action 31 loaded as the
|
||
modelview rotation, then `glTranslatef(-eye)`.
|
||
- Frame handoff is a critical-section-guarded `VFrame` + auto-reset event;
|
||
the emulator thread never blocks on GL.
|
||
|
||
Game path (`alpha1.conf`, full BattleTech v4.10): the window opens and draws
|
||
the view background, but the run hits the **pre-existing production-path sync
|
||
timeout** (`velocirender_receive timed out — sends_wo_rcv=3`, then
|
||
`failed in vr_sync`) — the `btdpl.ini`/netnub launch bursts several sends
|
||
before its first receive, which the POLL_THRESHOLD gating stalls. This is not
|
||
a 3b issue (the `flyk` clean-launch path renders fine); it is the same
|
||
production-sync item still open from Phase 2.
|
||
|
||
## 3d. The game's own world decodes and renders — LIVE
|
||
|
||
|
||
|
||
The live backend now draws the game's actual out-the-window view in real
|
||
time: sky, the arena floor receding to the horizon, and the player's own gun
|
||
barrels at frame bottom (static scene — without a RIO the sim doesn't
|
||
advance). Same frame decoded offline below.
|
||
|
||
|
||
|
||
That is an enemy mech (object 1048, 488 verts) standing in the mission arena —
|
||
reconstructed **entirely from the game's captured FIFO stream** by
|
||
`render_game.py` (real hull/armor/cockpit-glass materials from the wire; the
|
||
half-buried look is the offline painter's-algorithm artifact, which the live
|
||
GL depth buffer doesn't have). `game-cockpit-decoded.png` is the actual
|
||
cockpit camera: the arena floor to the horizon with the player's own gun
|
||
barrels rising at frame bottom.
|
||
|
||
What the game adds over flyk's flat scene (all now handled, offline +
|
||
live backend):
|
||
|
||
- **Stride-aware vertices.** `set_geom_verts` header word 3 = floats per
|
||
vertex: 3 (xyz), 4, 5 (xyz+uv), 8 (xyz+normal+uv), 9. Mech meshes carry
|
||
normals and texture coordinates — lighting/texturing data is on the wire.
|
||
- **Full DPL hierarchy.** Instances are `list_add` children of DCS nodes
|
||
(dcs→instance); instance flush field 4 references the object (type 7);
|
||
object→lod→geogroup→geometry via list_add. `dcs_link` (action 7) builds the
|
||
articulation tree (mech torso/arms/legs), each DCS a 4×4 at payload floats
|
||
4–19 (row-major, row 3 = translation).
|
||
- **The world is y-down** (the DCS matrices carry a y reflection; game coords
|
||
vs Division's). The renderer flips both x (Division mirror) and y.
|
||
- **Mission scene scale**: 10 km arena of 1000-unit ground tiles, 246 placed
|
||
instances, 330 geometries, 280 materials; the player's Thor (644 verts,
|
||
articulated sub-parts as sibling instances) sits exactly at the camera;
|
||
six enemy mechs stand ~1.5 km north.
|
||
- Camera: action 31 (rotation + eye at the cockpit position); with the sim
|
||
stalled (no RIO) it arrives once and the scene is static.
|
||
|
||
Offline tool: `render_game.py <fifodump>` (near-plane clipping, hierarchy
|
||
traversal). Fixture: a 954-frame capture. The live backend (`vpxlog.cpp`)
|
||
gained the same traversal: stride-aware geometry, dcs/instance/object link
|
||
decode, cached world transforms per DCS, y-down projection.
|
||
|
||
Still to come: texturing (stride-8 UVs + action-26 texel maps), lighting from
|
||
the wire normals, per-frame articulation once the RIO drives the sim, LOD
|
||
selection by distance.
|
||
|
||
## 3c. The full game runs through the live renderer (sync abort fixed)
|
||
|
||
**The production `vr_sync` abort is fixed, and BattleTech v4.10 now runs
|
||
indefinitely through the emulated board + live renderer.** Root cause was a
|
||
Phase-2 bring-up guard left in the device: `vpx_max_postboot_acks = 200`
|
||
capped how many post-boot replies the device would ever feed. A real game
|
||
session issues an unbounded stream of sync/frame/render replies; the game
|
||
aborted with `velocirender_receive timed out — sends_wo_rcv` the instant the
|
||
device fell silent — at *exactly* the 200th ack (measured: 143 sync + 55
|
||
frame-ack + 2 render = 200). The cap is now effectively unlimited
|
||
(`0x7fffffff`, override with `VPX_MAX_ACKS`). Same run after the fix: 2500+
|
||
syncs, 1264+ frame-acks, no abort. The game progresses past sync into
|
||
`L4VIDEO.cpp` content/entity setup (loading the Thor mech and terrain) and
|
||
draws continuously.
|
||
|
||
Contrary to the earlier note, the content **is** on the production image: 841
|
||
`.bgf` models under `ALPHA_1/REL410/BT/VIDEO/GEO` (incl. `BUTTEE.BGF`,
|
||
`MSLR.BGF`, the `THR*` Thor mech). `test.egg` is only mission parameters (as
|
||
the pod owner said); the geometry loads from `VIDEO/GEO` regardless.
|
||
|
||
Game render config: `game.conf` (RIO disabled so the missing cockpit board
|
||
doesn't stall the control manager; `cycles=max`).
|
||
|
||
### What the game's wire stream looks like (vs. flyk)
|
||
|
||
The game geometry path is heavier and different from flyk's flat DIVRGB:
|
||
- **action 26 (0x1A) dominates** — texture texel upload. 32-byte headers
|
||
`[node][size][w][h]` (`0x1000`=64×64, `0x4000`=128×128 tiles) then 256-byte
|
||
texel rows. 2880+ of these = the mission texture set streaming in.
|
||
- action 28 (0x1C): material/light parameter blocks.
|
||
- actions 23/25 (verts/conns) still carry mesh geometry but are a small
|
||
fraction of the stream.
|
||
- The scene graph is the **full DPL hierarchy** — zone → dcs → instance →
|
||
object → lod → geogroup → geometry — with **DCS transform matrices**
|
||
positioning each mech/prop. flyk's DIVRGB put geometry directly in
|
||
geogroups at world coordinates.
|
||
|
||
## Remaining (Phase 3d: game geometry on screen)
|
||
|
||
The window currently shows only the view background for the game because the
|
||
live backend's `scene_publish_frame()` walks the *flat* flyk layout
|
||
(geogroup→geometry at world coords). To draw the game's mechs it must:
|
||
|
||
1. **Traverse the full hierarchy** (instance/object/lod/geogroup) and **apply
|
||
DCS transform matrices** (accumulate the 4×4s down the graph) so each mech
|
||
lands at its world position; pick a LOD.
|
||
2. **Arena content search path**: the game logs `couldn't load object
|
||
thr_tshd.bgf` — that object lives in `VIDEO/GEO/ARENA/` and `…/POLAR/`
|
||
(arena-specific), not the top `GEO` dir; the geometry search path needs the
|
||
selected arena subdir (a `setenv`/config detail, not protocol).
|
||
3. **Texturing** (action 26 texel maps + SVT), lighting, depth/material.
|
||
Current backend is flat-shaded untextured polys.
|
||
4. **DOS/4GW sync variant** for the CYCLE `flyk yip.scn` fixture (Red Planet
|
||
geometry) — the older action-check sync.
|