215 lines
13 KiB
Markdown
215 lines
13 KiB
Markdown
# Division Renderer Collaboration Log
|
||
|
||
Shared working doc for the two renderer efforts against the VPX/Division
|
||
hardware interface:
|
||
|
||
- **In-process renderer** — GL HLE inside the DOSBox-X fork's VPX device
|
||
(`src/src/hardware/vpxlog.cpp`, commit-copy `vpx-device/vpxlog.cpp`).
|
||
Maintained by Cyd + Claude (this repo's sessions).
|
||
- **External renderer** — separate approach by the graphics developer +
|
||
their Claude instance. *(Fill in your section below.)*
|
||
|
||
**How to use this file:** each session (human or Claude) reads it at start,
|
||
appends to its own section and the Log, and commits. Don't rewrite the other
|
||
team's sections — append questions/answers instead. Commit messages prefixed
|
||
`collab:` so both sides can `git log --grep=collab:` for changes.
|
||
|
||
---
|
||
|
||
## Ground-truth sources (in priority order)
|
||
|
||
1. `sda4/DPL3/LIBDPL/DPLTYPES.H` + `DPL.H` — the dVS/DPL node structs and API.
|
||
2. `sda4/DPL3/VRENDER/` — Division's own i860 renderer source (rasterizer
|
||
semantics; the ramp/texture/light questions below should be settled here).
|
||
3. `ALPHA_1/REL410/BT/BTDPL.INI` — per-arena ground truth the game actually
|
||
loads: `fog=start end r g b`, `light=r g b pitch yaw roll`, `ambient=`,
|
||
`clip=`, `objectpath=` (arena content dirs), keyed by egg map/time/weather.
|
||
4. `restoration/` — WebGL re-render of the on-disk scene formats
|
||
(`divformats.py` parsers, `gallery_template.html` shading model). Proven
|
||
against 26 recovered scenes; its mesh shader is the reference shading
|
||
model (see below).
|
||
5. Live wire captures: `VPXLOG=<file>` logs every port I/O; scratch tooling
|
||
(`dump_nodes.py`-style parsers) has decoded node flushes from fifodumps.
|
||
|
||
## VPX board interface — verified on live captures
|
||
|
||
Register offsets (host side of the i860 board FIFO link):
|
||
`0` inputData (board→host), `1` outputData (host→board), `2` inputStatus
|
||
(bit0 = byte available), `3` outputStatus (always ready), `4/5` FIFO data
|
||
port link B low/high bytes, `16` resetRoot, `17` analyseRoot.
|
||
|
||
### Action codes (FIFO message stream)
|
||
|
||
| action | meaning |
|
||
|---|---|
|
||
| 1 | create `[type][name]` |
|
||
| 3 | flush `[name][type][struct]` — struct offsets below are into the payload `d`, where `d+8` = struct offset 0 |
|
||
| 7 | dcs_link `[parent][child]` — articulation tree |
|
||
| 9 | draw_scene — frame commit |
|
||
| 11 | list_add `[parent][child]` |
|
||
| 23 | set_geom_verts header `[name][0][n_verts][stride_floats]` + vertex bursts; stride 3..16 floats/vert: xyz, +uv at 3 (stride 5) or 6 (stride ≥8), +normal at 3 when stride ≥8 |
|
||
| 25 | set_geom_conns header `[name][n_polys][loop_len][0]` + index words (closing duplicate dropped) |
|
||
| 26 | texel upload header `[node][nbytes][w][h]` + texel bursts (4/8/16 bpp, bpp inferred from byte/texel ratio) |
|
||
| 31 | per-frame camera `[?][view][3x3 rows][eye]` |
|
||
|
||
### Node types (create/flush `type` field)
|
||
|
||
| type | node | verified layout (flush payload `d`) |
|
||
|---|---|---|
|
||
| 2 | zone | raw-logged only |
|
||
| 3 | view | win l/b/r/t/dist `d+24..40`, vw/vh `d+44/48`, near/far `d+52/56`, back_color `d+60..68`, **fog enable/mode `d+72`, near `d+76`, far `d+80`, rgb `d+84..92`** (mode 5 = linear; game ANIMATES fog and re-flushes ~27×; boot flushes carry heap garbage — sanity-check floats) |
|
||
| 4 | instance | object ref scanned at `d+8..` (resolves to a type-7 node) |
|
||
| 5 | dcs | 4×4 row-major at floats 4..19 (`d+16..`); row 3 = translation; world = local × parent (row-vector convention) |
|
||
| 6 | lmodel | raw-logged only |
|
||
| 7 | **object** (instance target in the scene-graph walker) — **CONFLICT:** the flush handler currently raw-dumps `t==7` as a *light* candidate ("t7 light=" lines in the tex log). Both can't be right; the real dpl_LIGHT wire type is still unidentified. Reconcile against DPLTYPES.H + a live dump. |
|
||
| 9 | geogroup | material ref at `d+64` |
|
||
| 11 | material | texture ref `d+8`, ramp ref `d+20`, emissive `d+24..32`, ambient `d+36..44`, diffuse `d+48..56`, opacity `d+60`, specular `d+72..80`, shininess `d+84` (84-byte struct; refs resolve lazily — a ref may precede its create) |
|
||
| 12 | dpl_TEXTURE params | texmap ref `d+8`, then minify/magnify/alpha/wrap_u/wrap_v/detail (u32s) + u0/v0/du/dv/anim_time (floats) + anim_bhv — logged (`t12` lines), **semantics not yet applied** |
|
||
| 13 | texmap header | texels ptr, u_size, v_size, bits_per_texel (+ hwareSize/hwareOffs/bilinear) |
|
||
| 14 | ramp | lo RGB `d+8..16`, hi RGB `d+20..28`; texels are intensities colorized through the material's ramp |
|
||
|
||
### Scene assembly & conventions
|
||
|
||
- Draw hierarchy: DCS (type 5) children (via list_add) → instance (4) →
|
||
object (7) → **first LOD child** (host keeps active LOD at list head; the
|
||
wire carries no switch distances) → geogroups (9) → geometry.
|
||
- World is **y-down** (DCS matrices carry reflections); Division screen x is
|
||
mirrored vs GL eye x (flip x after projection).
|
||
- Vertices reach the renderer already in world space; the modelview holds
|
||
only the camera (action 31: row-major world→eye rotation + eye pos,
|
||
`eye_p = R·(p − e)`).
|
||
- Frustum from view win[]: `glFrustum(win[l/r/b/t] · near / win[4], …)`.
|
||
- **The camera only goes live after the first analog (throttle/stick)
|
||
input** — until then the game submits no worklist and the screen is the
|
||
back_color clear. Hands-off runs look like "solid sky"; check the radar
|
||
head (win0 BMP dump) to confirm the mission actually loaded.
|
||
|
||
## In-process renderer — status (2026-07-04)
|
||
|
||
The device now runs the restoration gallery's shading model verbatim as a
|
||
GLSL 1.10 program (fixed-function fallback; stderr prints
|
||
`VPX render: gallery shading model active (GLSL)`):
|
||
|
||
```
|
||
acc = Σ_i lightCol_i · max(dot(N, −lightDir_i), 0) (world space, 2 dir lights)
|
||
lit = mix(ramp.lo, ramp.hi, clamp(acc, 0, 1))
|
||
c = (matAmb·sceneAmb + matDiff·lit) · tex + matEmis
|
||
out = mix(fogColor, c, max(clamp((fogEnd−d)/(fogEnd−fogStart), 0, 1), immune))
|
||
```
|
||
|
||
Interim until the light node is decoded: hardcoded sun + sceneAmb 0.45,
|
||
overridable via `VPX_AMBIENT="r,g,b"` / `VPX_SUN="r,g,b,pitch,yaw"` (deg,
|
||
y-down). Fog comes from the type-3 view flush. Opacity/specular parsed but
|
||
not applied. Polys without wire normals get Newell flat face normals.
|
||
|
||
**Ramp interpretation (open):** wire textures are intensity texels baked
|
||
through the material ramp at upload; the gallery applies the ramp to *light*
|
||
(disk textures are RGB). The device currently avoids double-application by
|
||
using an identity light-ramp on textured polys. VRENDER source should settle
|
||
whether Division ramps texels, light, or both.
|
||
|
||
## Data taps available to an external renderer (today)
|
||
|
||
- `VPXLOG=<path>` (host env): full port-I/O log incl. every FIFO byte —
|
||
replayable; parse with the action/type tables above.
|
||
- `VPX_DUMPDIR=<dir>` + touch `<dir>/DUMP`: per-head BMP dumps (`win<g>.bmp`,
|
||
g=0 radar, 3/4 MFD heads, 5-9 exploded singles).
|
||
- Device tex log: `t7`/`t12`/`t13` raw flush lines for undecoded nodes.
|
||
- `RIO_TAP=<path>`: serial wire tap (not renderer-relevant, but proves rig
|
||
liveness).
|
||
- **Offer:** if a structured per-frame scene export (JSON/binary worklist
|
||
after graph assembly, i.e. post-DCS world-space polys + materials) would
|
||
feed your renderer better than raw FIFO, ask here — straightforward to add
|
||
behind an env var.
|
||
|
||
## Open questions (either side, answer in place with date)
|
||
|
||
1. Real dpl_LIGHT wire type + layout (type 7 conflict above). BTDPL.INI
|
||
`light=` lines give expected values to match against.
|
||
2. Type-12 texture param semantics (wrap/clamp/uv-window/animate) — cause of
|
||
smeared wall textures?
|
||
3. Ramp on texel vs ramp on light (VRENDER).
|
||
4. Terrain + sky in arena missions: what content/nodes carry them? (Live
|
||
arena1 renders structures but almost no ground; sky is back_color only.)
|
||
5. Opacity blending order/mode (glass cockpits, effects).
|
||
6. Scene ambient: which node carries it at runtime (lmodel type 6?).
|
||
|
||
## External renderer — approach & status
|
||
|
||
*(Yours — please fill in: approach, inputs consumed, what you need from the
|
||
device side, decisions taken. Append, date-stamp.)*
|
||
|
||
## Log
|
||
|
||
- **2026-07-04 (Claude, in-process side):** doc created. Gallery shading
|
||
model ported into the device and verified live in arena1 (lit faces,
|
||
plausible wire fog). Full material parse landed. Next in-process steps:
|
||
light node decode, type-12 params, restoration-toolkit reference render of
|
||
arena1 content.
|
||
|
||
- **2026-07-06 (Claude): actions 0x29/0x2b DECODED — the 2D HUD layer
|
||
(dpl2d display lists).** The game draws its out-the-window HUD (targeting
|
||
reticle, twist/pitch tick ladders, carets, weapon lamps, torso-twist dial)
|
||
through DPL's 2D overlay API (`dpl2d_*`, see CODE/RP/MUNGA_L4/libDPL/dpl/
|
||
DPL_2D.H + vpx/DPL2DTAG.H; game-side builder = L4VIDRND.CPP
|
||
ReticleRenderable etc.), NOT as scene-graph instances.
|
||
- `0x29` = dpl2d_NewDisplayList: `[remote handle][0]`, fire-and-forget.
|
||
- `0x2b` = dpl2d_FlushDisplayList: the host streams the `dpl2d_DISPLAY`
|
||
chunk structs verbatim (`remote, next, tail, size, open, data[30]`,
|
||
WORDS_PER_DISPLAY_CHUNK=30 → the fixed 140B payload). First chunk:
|
||
`remote`=handle, `tail`=total chunk count; continuations have remote=0.
|
||
Data words CONCATENATE across chunks (tag args split freely at chunk
|
||
boundaries); a flush REPLACES the list.
|
||
- Data = tag stream per DPL2DTAG.H (0=open_polyline … 22=set_alpha).
|
||
Arg words are float32 except call_displaylist (handle) and the
|
||
circle/clip mode/filled flags. `open_lines` points come in PAIRS
|
||
(independent segments); polygon = filled convex; matrix = 6-float 2D
|
||
affine, state push/pop covers matrix+color+alpha+linewidth.
|
||
- View binding (dpl2d_SetViewDisplayList) = one word appended after the
|
||
96B view struct in full-length view flushes (wire-payload word 25).
|
||
Root list in BT = a wrapper that calls the real HUD list; two sub-lists
|
||
re-flush ~10 Hz (the live carets).
|
||
- Coordinates are view-rect space (x0..x1/y0..y1 from the view struct,
|
||
y up); colors 0..1, composited over the frame (board video-out mix).
|
||
- Implemented in vrboard.py (`dl2d` store) + vrview.py/vrview_gl.py
|
||
(overlay pass, both backends) — validated on a live BT mission capture:
|
||
reticle centered, heading tape + yellow caret, pitch ladder, weapon
|
||
lamps, twist dial all render. Open question 0x29/0x2b in the protocol
|
||
spec §13 “fire-and-forget unknowns” can be closed.
|
||
- Related negative result for subsystem debugging: in a 25-min live
|
||
weapons-test capture the game emitted ZERO `0x23` fire/pick actions
|
||
while all four trigger buttons were pressed repeatedly (RIO tap
|
||
confirms delivery) — the fire path never reached the renderer; weapon
|
||
triggers are dying game-side (control-mode/mapping or weapon power
|
||
gating), not render-side.
|
||
|
||
- **2026-07-10 (Claude, in-process side): RED PLANET now runs the full pod +
|
||
ONE OPEN RENDERER ITEM FOR DAVID.** RP 4.10 (RPL4OPT) now drives the whole
|
||
cockpit through the same stack (shared MUNGA engine): networking, RIO, dual-
|
||
AWE sound, plasma, and — as of today — the VDB cockpit gauges (radar + all
|
||
4-quadrant MFDs, operator-verified). So the external renderer's live inputs
|
||
now include RP content (Mars race tracks, `speck`/racer vehicles), not just
|
||
BT. The RP gauge fix was NOT renderer-related (it was an SVGA bank-
|
||
granularity mismatch in RP's L4GAUGE.INI, 4KB vs BT's 64KB, mishandled by
|
||
DOSBox-X's CL-GD5434 bank emu — fixed by matching to 64KB; see VDB-NOTES.md).
|
||
- **OPEN, renderer-side (Dave's domain):** RP's out-the-window view FLICKERS
|
||
= Z-FIGHTING on the co-planar TRACK DECALS (checkered start/finish + road
|
||
markings laid exactly on the track surface). BT mech-arenas don't have
|
||
stacked co-planar surfaces so it never showed. Confirmed NOT frame-drops
|
||
(bridge skipped=0) and NOT texture aliasing (mipmaps didn't fix) and NOT
|
||
scene-graph churn (stable). Fix is a polygon DEPTH-OFFSET / small depth
|
||
bias on the decal surfaces in the GL mesh draw. If your latest drop already
|
||
added polyoffset or a decal path, great — flag it and we'll re-verify.
|
||
- **Heads-up: local edits to vrview_gl.py you may collide with on a new
|
||
drop** (all in this repo's dpl3-revive/patha copy): (a) `_tex()` gained
|
||
`VRVIEW_MIPMAP=1` (build_mipmaps + trilinear/aniso, default OFF, keeps
|
||
point-sample look otherwise) for RP's checkerboard-floor shimmer; (b) HUD
|
||
shader got a `u_scale` uniform (`VRVIEW_HUDSCALE`) + the dpl2d Y-down flip;
|
||
(c) fog moved per-FRAGMENT (v_fog varying + u_fog_on/u_fogrange in the FS)
|
||
to stop terrain/cloud escaping fog. Merge points if your drop rewrites
|
||
`_tex()` / HUD / fog shaders.
|
||
- **Offer:** when your update lands, I'll diff it against our dpl3-revive/,
|
||
reconcile the above, and re-verify live through the bridge on BOTH games
|
||
(BT arena + RP race) so nothing regressed and we see if it fixed the RP
|
||
z-fighting. Ping via a `collab:` commit or just drop the files.
|