User-reported: a hard black strip at the horizon, between the arena walls and
the sky geometry. Those pixels aren't covered by any polygon -- the game relies
on the board's own background clear there (the sky dome doesn't extend down to
the wall tops). Our renderer cleared the framebuffer to black, so the gap read
as a black band.
Fill the background with the FOG far-color instead: anything no polygon covers
is at effectively infinite distance = fully fogged, so the horizon gap now
blends seamlessly into the same haze the distant geometry fogs toward (filled
far pixels -> FOG_RGB via the fog blend; unfilled background = FOG_RGB directly
-> continuous). Offline-verified: the black strip becomes a smooth blue-purple
haze band. (The game's actual view-flush back_color may differ slightly from
the fog far-color; the fog color is the physically-consistent proxy until
back_color is parsed off the wire -- noted in code.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
BTDPL.INI's fog values for this mission (testarn.egg = arena1/day/clear,
[ardayclear] leaf: fog=200.0 1250.0 0.32 0.3 0.5, clip=0.25 1300.0) are exact,
verified ground truth. The depth term took two tries: `tz` (texz) looked like
the right candidate (BT411 says fog is linear on "W") but measured live it
DECREASES with distance -- the wrong direction/shape. Re-derived from the
actual firmware source instead of guessing again: AS860/XFPROJ.SS's own
comment on the projection code ("pz=wz*invz, where pz=1 at near clip, 0 at
infinity") establishes the z-buffer value as a normalized hither/distance
quantity, giving distance = hither * 2^20 / raw_zbuf_value.
gpu_raster.py: added a 10th shader output channel carrying the raw z-buffer
float (previously only a filled/empty flag existed), and the fog blend using
this formula + BTDPL.INI's exact color. Verified offline: distance climbs
smoothly (394->1051) across a receding ceiling, landing inside the authored
200-1250 range with no further fudging; visually the horizon shows the
authored blue-purple haze, correctly absent near the camera.
IG-SHADING-MODEL.md: documented the false start (tz) and the source-grounded
fix, with the honest caveats that remain (2^20 exact scale, hither as a fixed
per-mission constant vs. per-view from the wire).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Found my own prior documentation (IG-SHADING-MODEL.md sec 2, written earlier
this session before compaction) that already establishes: the wire's 0x1a
texture uploads arrive ALREADY colourised (the material ramp is pre-baked on
this path, per the shipped libDPL header model cross-referenced against
BT411's context/rendering.md). Tinting them again with the lit-color plane
double-colours the result -- exactly the "more colors but neither correct"
outcome reported live (sky/ground now differ, but neither right).
Keep the lit-color plane for FLAT (untextured) polys only, where it replaces
a hardcoded placeholder with real per-polygon data; textured polys sample
unmodified, as documented. The underlying r24/g24/b24 mechanism itself is
still real and worth understanding precisely (see open-questions tracking),
but applying it as a texture tint was the wrong model.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three real issues found while watching the live-render session against the
actual running game and fixed:
1. STALE-FRAME SKIP (live_render.py): the renderer treated every single
draw_scene as a frame to fully render+present. The game ticks at a
measured ~28 draw_scene/s (matches the documented FAST 28Hz SOS clock),
faster than our render, so a backlog built continuously -- we were
perpetually rendering OLD content, which read as "frozen or 30s+ behind."
Fix: SockSource.has_pending() detects an already-buffered next record;
when backlogged, skip render+present (still consume/step normally) so we
always show the freshest available state instead of crawling through
history. pump() now drains aggressively so the backlog check is honest.
2. MID-MISSION CATCH-UP (live_render.py --catchup): reconnecting to an
already-running pod produced a black screen. Root cause: draw_scene only
means "render what's already loaded" -- it carries no geometry itself.
The socket tee forwards only NEW traffic to a fresh client, so a from-
scratch firmware boot has an empty scene graph. Fix: replay vpxlog's
archival VPX_FIFODUMP (a second, independent sink of the same wire,
written since pod boot) as the initial queue before going live -- exactly
the same catch-up vpxlog's OWN native bridge already relies on for this
same reason.
3. LIT-COLOR PLANE (gpu_raster.py): user-reported "textures are mostly
grayscale" and "no fog effect." Found the actual mechanism in the real
firmware source (sda4 DIVPXMAP.H + EOF.C): TREEclmpintoMEM (op 0x5a)
writes dvpx_r24/g24/b24 -- a per-polygon lit color -- which EOF.C later
copies into dvpx_eofr/g/b (the final screen color) via a straight 24-bit
CPY in the simple (unfogged) case. Our renderer decoded this instruction
but never used it, instead hardcoding flat polys to a placeholder
(60,60,70) and sampling textures with no lighting modulation at all.
Verified live on the wire: the value is a DIRECT 0-255-ish brightness
(not a [0,1] float to rescale -- observed range ~49..395 with real
per-polygon variation, only the brightest few clipping), consistent with
a straight clamp-to-8-bit write. Now decoded (addr 118/126/134, empirically
confirmed against DIVPXMAP.H's r24/g24/b24 spacing) and applied: flat polys
get the real lit color instead of the placeholder; textured polys are
tinted by it (texture * lit/255). Offline-tested (no crash, visibly
plausible result: a lavender-tinted floor with real near/far gradient
replacing the flat gray-blue placeholder) -- NOT yet cross-checked
against a ground-truth screenshot, so treat the exact hue as provisional
even though the underlying mechanism is verified real.
Honest scope note: this changes gpu_raster's rendered colors on purpose, so
it no longer bit-matches the OLDER frame_*.png CPU reference from before this
fix (that reference predates lit-color decode). The geometry/perspective/
raster correctness those bit-identity tests proved is unaffected -- only the
color post-process changed, layered on top.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
User-visible bug: connected the reconstructed renderer to the REAL running pod
for the first time (M4d demo) and the render fell increasingly behind the live
game, plus looked near-grayscale. Root-caused with direct profiling instead of
guessing:
1. VPX_RENDER=1 was set in launch_live.ps1 (copied from launch_pod.ps1's
defaults without thinking it through) -- it starts vpxlog.cpp's OWN native
GL render thread (rt_main), which contends with our renderer for the same
CPU/GL resources. Live-caught: with it set, wire delivery stalled at wire
command 8 for 90+s; removing it, the same mission reached command 21,488
in the same window. This was the dominant cause of the visible lag.
2. gpu_raster.build_prims() scanned the whole per-draw program window
(0x08158000-0x08170000, ~24k words) via a Python-level r32() call per word
every frame. Added emu860c.dump_range(lo,hi)->bytes (one bulk C read) and
rewrote build_prims to index a numpy array instead -- verified byte-identical
dump_range output first, then reconfirmed the whole render is STILL
bit-identical to the CPU reference (frames 0/5/11, differ>24 = 0.000%).
3. Profiling after (1) showed the per-texture compositing loop was the real
remaining cost: 62ms/frame doing full-832x512-frame fancy-indexed texture
sampling for EACH of 13 textures, regardless of how few pixels use each one.
Added dpl_sampler.composite_masked (sample+composite only the pixels a
texture actually covers; verified equivalent to composite() on a partial
mask in dpl_sampler's own conformance test) and hoisted the invariant
out of the loop.
Combined: render time 95.8ms -> 54.9ms/frame (~1.75x from this change alone,
~2.1x vs the original ~117ms/frame baseline), still bit-identical to the CPU
reference throughout.
launch_live.ps1: removed VPX_RENDER, renamed the automatic-variable shadow
-> (flagged by PSScriptAnalyzer), added -Pin <fifodump> to
pre-load the full texture set (bit-identical to offline) instead of the
incremental default, for a demo that wants correct color immediately rather
than waiting on the live mission's own upload order.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
live_render.py connects to vpxlog.cpp's existing VPX_FIFOSOCK tee (the DOSBox-X
C012 device), consumes the game's wire AS IT ARRIVES, and runs the firmware and
the socket CONCURRENTLY -- pumping the socket whenever the firmware's receive
point drains -- rendering each draw with the shared verified GPU raster.
gpu_raster.py factors the M4c-raster renderer out of m4b_gpu so offline and live
share ONE render path (the 6-edge fp64 raster + verified texel decode).
feed_sock.py mimics vpxlog's FIFOSOCK server (streams a real capture in delayed
chunks) so the live path is validated end-to-end WITHOUT the flaky live pod.
Result: 12 frames streamed live at 2.3 fps; with --pin (equalized texture
availability) the live frames are BIT-IDENTICAL to the offline reference
(12/12 differ>24 = 0.000%) -- the seam is faithful. The default incremental
texture model is more authentic (the board only has texels it has received);
its divergence from offline is purely the M5-B tid%ntex placeholder, orthogonal.
Live topology now closed end-to-end offline:
game/capture -> C012 VPX_FIFOSOCK -> live_render -> firmware -> GPU -> frames.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CydandClaude Opus 4.8