vwemirror/TeslaRel410
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Live-view fixes: stale-frame skip, mid-mission catch-up, lit-color plane

Browse Source 
Three real issues found while watching the live-render session against the
actual running game and fixed:

1. STALE-FRAME SKIP (live_render.py): the renderer treated every single
   draw_scene as a frame to fully render+present. The game ticks at a
   measured ~28 draw_scene/s (matches the documented FAST 28Hz SOS clock),
   faster than our render, so a backlog built continuously -- we were
   perpetually rendering OLD content, which read as "frozen or 30s+ behind."
   Fix: SockSource.has_pending() detects an already-buffered next record;
   when backlogged, skip render+present (still consume/step normally) so we
   always show the freshest available state instead of crawling through
   history. pump() now drains aggressively so the backlog check is honest.

2. MID-MISSION CATCH-UP (live_render.py --catchup): reconnecting to an
   already-running pod produced a black screen. Root cause: draw_scene only
   means "render what's already loaded" -- it carries no geometry itself.
   The socket tee forwards only NEW traffic to a fresh client, so a from-
   scratch firmware boot has an empty scene graph. Fix: replay vpxlog's
   archival VPX_FIFODUMP (a second, independent sink of the same wire,
   written since pod boot) as the initial queue before going live -- exactly
   the same catch-up vpxlog's OWN native bridge already relies on for this
   same reason.

3. LIT-COLOR PLANE (gpu_raster.py): user-reported "textures are mostly
   grayscale" and "no fog effect." Found the actual mechanism in the real
   firmware source (sda4 DIVPXMAP.H + EOF.C): TREEclmpintoMEM (op 0x5a)
   writes dvpx_r24/g24/b24 -- a per-polygon lit color -- which EOF.C later
   copies into dvpx_eofr/g/b (the final screen color) via a straight 24-bit
   CPY in the simple (unfogged) case. Our renderer decoded this instruction
   but never used it, instead hardcoding flat polys to a placeholder
   (60,60,70) and sampling textures with no lighting modulation at all.
   Verified live on the wire: the value is a DIRECT 0-255-ish brightness
   (not a [0,1] float to rescale -- observed range ~49..395 with real
   per-polygon variation, only the brightest few clipping), consistent with
   a straight clamp-to-8-bit write. Now decoded (addr 118/126/134, empirically
   confirmed against DIVPXMAP.H's r24/g24/b24 spacing) and applied: flat polys
   get the real lit color instead of the placeholder; textured polys are
   tinted by it (texture * lit/255). Offline-tested (no crash, visibly
   plausible result: a lavender-tinted floor with real near/far gradient
   replacing the flat gray-blue placeholder) -- NOT yet cross-checked
   against a ground-truth screenshot, so treat the exact hue as provisional
   even though the underlying mechanism is verified real.

Honest scope note: this changes gpu_raster's rendered colors on purpose, so
it no longer bit-matches the OLDER frame_*.png CPU reference from before this
fix (that reference predates lit-color decode). The geometry/perspective/
raster correctness those bit-identity tests proved is unaffected -- only the
color post-process changed, layered on top.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Cyd
2026-07-20 10:42:51 -05:00
co-authored by Claude Opus 4.8
parent ef7db9f2e4
commit 854ec0b216
2 changed files with 147 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files
emulator/firmware-decomp/emu860c/gpu_raster.py
+52 -16
View File
@@ -24,9 +24,11 @@ FX_SHADER = """
#extension GL_ARB_gpu_shader_fp64 : enable
layout(local_size_x = 64, local_size_y = 1) in;
layout(std430, binding = 0) readonly buffer Prims { double prims[]; };
// 10 dvec4 / prim (40 doubles): [0..5] edges (up to 6, A B C used),
// [6] zdepth+nedges, [7] u+hasuv, [8] v+tid, [9] tz+0
layout(std430, binding = 1) buffer Outs { uint outbuf[]; }; // 6 words/pixel
// 11 dvec4 / prim (44 doubles): [0..5] edges (up to 6, A B C used),
// [6] zdepth+nedges, [7] u+hasuv, [8] v+tid, [9] tz+0,
// [10] litR,litG,litB,0 -- the polygon's TREEclmpintoMEM lit color
// (dvpx_r24/g24/b24; DIVPXMAP.H), CONSTANT per polygon (not a plane).
layout(std430, binding = 1) buffer Outs { uint outbuf[]; }; // 9 words/pixel
uniform int n_prims;
uniform int width;
void main() {
@@ -37,8 +39,9 @@ void main() {
double zbest = -1e30lf;
double uu = 0.0lf, vv = 0.0lf, tz = 1.0lf;
float tid = 0.0, hasuv = 0.0;
float litR = 255.0, litG = 255.0, litB = 255.0;
for (int p = 0; p < n_prims; p++) {
int b = p*40;
int b = p*44;
bool allpos = true, allneg = true;
for (int e = 0; e < 6; e++) {
int eb = b + e*4;
@@ -55,14 +58,18 @@ void main() {
vv = prims[b+32]*fx + prims[b+33]*fy + prims[b+34];
tz = prims[b+36]*fx + prims[b+37]*fy + prims[b+38];
tid = float(prims[b+35]);
litR = float(prims[b+40]); litG = float(prims[b+41]); litB = float(prims[b+42]);
}
int pix = y*width + x;
outbuf[pix*6+0] = (zbest > -1e29lf) ? 0x3f800000u : 0xff800000u;
outbuf[pix*6+1] = floatBitsToUint(float(uu));
outbuf[pix*6+2] = floatBitsToUint(float(vv));
outbuf[pix*6+3] = floatBitsToUint(float(tz));
outbuf[pix*6+4] = floatBitsToUint(tid);
outbuf[pix*6+5] = floatBitsToUint(hasuv);
outbuf[pix*9+0] = (zbest > -1e29lf) ? 0x3f800000u : 0xff800000u;
outbuf[pix*9+1] = floatBitsToUint(float(uu));
outbuf[pix*9+2] = floatBitsToUint(float(vv));
outbuf[pix*9+3] = floatBitsToUint(float(tz));
outbuf[pix*9+4] = floatBitsToUint(tid);
outbuf[pix*9+5] = floatBitsToUint(hasuv);
outbuf[pix*9+6] = floatBitsToUint(litR);
outbuf[pix*9+7] = floatBitsToUint(litG);
outbuf[pix*9+8] = floatBitsToUint(litB);
}
"""
@@ -95,6 +102,12 @@ def build_prims(dump_range):
if len(edges) < 3:
continue
zp = tzp = up = vp = None; tid = None; q = p
# r24/g24/b24 (DIVPXMAP.H): the polygon's lit color, written via
# TREEclmpintoMEM (op 0x5a) as a CONSTANT (3rd word of the 3-word
# instruction), not a linear plane -- confirmed live: addr 118/126/134
# (DIVPXMAP.H names them 117/125/133; the +1 is this bytecode's own
# addressing, taken empirically from the wire, not the header comment).
litr = litg = litb = None
for _ in range(70):
w = rd(q); op = (w >> 8) & 0xff; ad = w & 0xff
if op == 0x21 and zp is None:
@@ -107,16 +120,30 @@ def build_prims(dump_range):
vp = (f32(rd(q + 4)), f32(rd(q + 8)), f32(rd(q + 12)))
if op == 0xf7 and ad in (141, 142) and tid is None:
tid = (rd(q + 4) >> 2) & 0x3f
if op == 0x5a and ad == 118 and litr is None:
litr = f32(rd(q + 8))
if op == 0x5a and ad == 126 and litg is None:
litg = f32(rd(q + 8))
if op == 0x5a and ad == 134 and litb is None:
litb = f32(rd(q + 8))
q += 4
if zp is None:
continue
hasuv = 1.0 if (up and vp and tzp) else 0.0
up = up or (0, 0, 0); vp = vp or (0, 0, 0); tzp = tzp or (0, 0, 1)
# clamp direct to [0,255] -- the observed range (~49..395) is already
# in 8-bit brightness units, not a [0,1] float to rescale (see
# M4B-RESULTS.md "lit color plane" note); default 255 = no-op tint
# for polys where the color plane wasn't found (unchanged behavior).
litr = 255.0 if litr is None else max(0.0, min(255.0, litr))
litg = 255.0 if litg is None else max(0.0, min(255.0, litg))
litb = 255.0 if litb is None else max(0.0, min(255.0, litb))
eslots = [(e[0], e[1], e[2], 1.0) for e in edges[:6]]
while len(eslots) < 6:
eslots.append((0.0, 0.0, 0.0, 0.0))
recs4 += eslots + [(*zp, float(len(edges))), (*up, hasuv),
(*vp, float(tid or 0)), (*tzp, 0.0)]
(*vp, float(tid or 0)), (*tzp, 0.0),
(litr, litg, litb, 0.0)]
if not recs4:
return None
return np.array(recs4, dtype=np.float64).reshape(-1, 4)
@@ -132,15 +159,15 @@ class Renderer:
prims = build_prims(dump_range)
if prims is None:
return None
n_prims = len(prims) // 10
n_prims = len(prims) // 11
b0 = self.ctx.buffer(prims.tobytes())
out = self.ctx.buffer(reserve=W * H * 6 * 4)
out = self.ctx.buffer(reserve=W * H * 9 * 4)
b0.bind_to_storage_buffer(0)
out.bind_to_storage_buffer(1)
self.prog_gl['n_prims'] = n_prims
self.prog_gl['width'] = W
self.prog_gl.run(group_x=(W + 63) // 64, group_y=H)
raw = np.frombuffer(out.read(), dtype=np.uint32).reshape(H, W, 6)
raw = np.frombuffer(out.read(), dtype=np.uint32).reshape(H, W, 9)
b0.release(); out.release()
zb = raw[..., 0].view(np.float32)
uu = raw[..., 1].view(np.float32).astype(np.float64)
@@ -148,9 +175,17 @@ class Renderer:
tz = raw[..., 3].view(np.float32).astype(np.float64)
tid = raw[..., 4].view(np.float32).astype(np.int64)
hasuv = raw[..., 5].view(np.float32)
# lit color (dvpx_r24/g24/b24 -> TREEclmpintoMEM): the polygon's
# computed lighting, clamped to [0,255] -- see build_prims. Used as
# the direct color for flat polys, and as a multiplicative tint on
# sampled texture color for textured ones.
lit = np.stack([raw[..., 6].view(np.float32),
raw[..., 7].view(np.float32),
raw[..., 8].view(np.float32)], axis=-1)
filled = zb > -1e29
img = np.zeros((H, W, 3), np.uint8)
img[filled & (hasuv <= 0.5)] = (60, 60, 70)
flat = filled & (hasuv <= 0.5)
img[flat] = np.clip(lit[flat], 0, 255).astype(np.uint8)
tzc = np.where(np.abs(tz) < 1.0, np.sign(tz) + (tz == 0), tz)
ucoord = (uu / tzc * 2048).astype(np.int64)
vcoord = (vv / tzc * 2048).astype(np.int64)
@@ -169,5 +204,6 @@ class Renderer:
# texture, so full-frame sampling here was O(ntex x W x H))
ui = wrap_index((ucoord[sel] * tu) >> 8, tu, wrap_u)
vi = wrap_index((vcoord[sel] * tv) >> 8, tv, wrap_v)
composite_masked(img, sel, arr[vi, ui], cut_mode)
samp = arr[vi, ui].astype(np.float64) * (lit[sel] / 255.0)
composite_masked(img, sel, np.clip(samp, 0, 255).astype(np.uint8), cut_mode)
return img
emulator/firmware-decomp/emu860c/live_render.py
+95 -39
View File
@@ -41,6 +41,31 @@ PRESENT = '--present' in sys.argv
PIN = None
if '--pin' in sys.argv:
PIN = sys.argv[sys.argv.index('--pin') + 1]
# --catchup <fifodump>: replay the ARCHIVAL dump (vpxlog's VPX_FIFODUMP -- a
# second, independent sink of the same wire, written since pod boot) before
# going live. Required when connecting mid-mission: the socket tee only
# forwards NEW traffic to a fresh client, so without this our firmware boots
# with an EMPTY scene graph -- draw_scene only says "render what's already
# loaded," it carries no geometry itself, so a from-scratch board shows black.
CATCHUP = None
if '--catchup' in sys.argv:
CATCHUP = sys.argv[sys.argv.index('--catchup') + 1]
def parse_fifodump(path):
data = open(path, 'rb').read()
out = []; off = 0
while True:
i = data.find(b'VPXM', off)
if i < 0:
break
ln = struct.unpack_from('<I', data, i + 4)[0]
body = data[i + 8:i + 8 + ln]; off = i + 8 + ln
if len(body) >= 4:
a = struct.unpack_from('<I', body, 0)[0]
if a < 0x100:
out.append((a, body[4:]))
return out
class SockSource:
@@ -59,17 +84,23 @@ class SockSource:
print("connected to VPX_FIFOSOCK :%d" % port, flush=True)
def pump(self):
"""Read whatever is available; set eof on clean close. Non-fatal timeout."""
try:
data = self.sock.recv(1 << 16)
if data:
self.buf += data
else:
"""Drain everything CURRENTLY available (loop while a full-size read
keeps coming back), not just one small recv() -- so has_pending()
reflects the true backlog instead of an arbitrary slice of it."""
while True:
try:
data = self.sock.recv(1 << 16)
except socket.timeout:
return
except OSError:
self.eof = True
except socket.timeout:
pass
except OSError:
self.eof = True
return
if not data:
self.eof = True
return
self.buf += data
if len(data) < (1 << 16):
return # short read: likely caught up for now
def next_record(self):
i = self.buf.find(b'VPXM', self.off)
@@ -86,29 +117,40 @@ class SockSource:
return (action, body[4:])
return self.next_record() # skip non-message burst
def has_pending(self):
"""True if another complete record is ALREADY buffered locally (no
socket read needed) -- i.e. we're backlogged relative to what has
already arrived. Used to skip rendering stale intermediate frames."""
i = self.buf.find(b'VPXM', self.off)
if i < 0 or i + 8 > len(self.buf):
return False
ln = struct.unpack_from('<I', self.buf, i + 4)[0]
return i + 8 + ln <= len(self.buf)
src = SockSource(int(SPEC.split(':')[1]))
r = boot(fw='vrend410', queue=[]) # queue grows as wire arrives
# Read the catch-up file AFTER connecting the socket (not before): the socket
# then buffers everything from connect-time forward while we parse, so any
# small race window biases toward a harmless duplicate rather than a missed
# command (a stray duplicate create/list_add is far less damaging than a
# missing one -- the latter is exactly what produced the black screen).
catchup_recs = parse_fifodump(CATCHUP) if CATCHUP else []
if CATCHUP:
print("catch-up: replaying %d records from %s" %
(len(catchup_recs), os.path.basename(CATCHUP)), flush=True)
N_CATCHUP = len(catchup_recs)
r = boot(fw='vrend410', queue=list(catchup_recs))
shim = CpuShim()
RECV = emu_main.MAPS['vrend410']['receive']
g = igc_gpu.GpuTile()
print("GPU:", g.ctx.info['GL_RENDERER'], flush=True)
recs = [] # running wire, for the texture store
tex_dirty = False
recs = list(catchup_recs) # running wire, for the texture store
tex_dirty = bool(catchup_recs)
renderer = Renderer(g.ctx, []) # texlist filled once textures arrive
pinned = False
if PIN:
pdata = open(PIN, 'rb').read(); prec = []; poff = 0
while True:
pi = pdata.find(b'VPXM', poff)
if pi < 0:
break
pln = struct.unpack_from('<I', pdata, pi + 4)[0]
pbody = pdata[pi + 8:pi + 8 + pln]; poff = pi + 8 + pln
if len(pbody) >= 4 and struct.unpack_from('<I', pbody, 0)[0] < 0x100:
prec.append((struct.unpack_from('<I', pbody, 0)[0], pbody[4:]))
renderer.texlist = list(build_texstore(prec).items())
renderer.texlist = list(build_texstore(parse_fifodump(PIN)).items())
pinned = True
print("PINNED %d textures from %s" % (len(renderer.texlist), os.path.basename(PIN)), flush=True)
@@ -154,6 +196,7 @@ def present(img, nth):
frames = 0
skipped = 0
prev_draw = False
t0 = time.time()
fps_t = t0
@@ -170,20 +213,33 @@ while not stopped:
print("sentinel", flush=True); break
if pc == RECV:
if prev_draw:
if tex_dirty and not pinned:
renderer.texlist = list(build_texstore(recs).items())
tex_dirty = False
img = renderer.frame(emu860c.dump_range)
if img is not None:
if not present(img, frames):
break
frames += 1
if time.time() - fps_t >= 2.0:
print("live: %d frames, %.1f fps (cmd %d)"
% (frames, frames / (time.time() - t0), r.qi), flush=True)
fps_t = time.time()
if MAXF and frames >= MAXF:
break
# Skip stale intermediate draws when a backlog exists: the game
# runs at a fixed real tick rate (measured ~28 draw_scene/s on
# this rig's FAST clock) that our render can fall behind, so
# rendering EVERY draw_scene means perpetually catching up on
# old content. Only render+present when we're at the live edge --
# this bounds perceived lag to render-time, not backlog depth.
# Also skip unconditionally while still draining the catch-up
# history (r.qi <= N_CATCHUP) -- no point rendering thousands of
# historical frames just to reach the live edge.
if r.qi <= N_CATCHUP or src.has_pending():
skipped += 1
else:
if tex_dirty and not pinned:
renderer.texlist = list(build_texstore(recs).items())
tex_dirty = False
img = renderer.frame(emu860c.dump_range)
if img is not None:
if not present(img, frames):
break
frames += 1
if time.time() - fps_t >= 2.0:
print("live: %d frames (%d skipped), %.1f fps (cmd %d)"
% (frames, skipped, frames / (time.time() - t0), r.qi),
flush=True)
fps_t = time.time()
if MAXF and frames >= MAXF:
break
prev_draw = False
# make sure the next record is present before the firmware consumes it
while r.qi >= len(r.queue):
@@ -197,5 +253,5 @@ while not stopped:
if h(shim) == 'done':
break
dt = time.time() - t0
print("live done: %d frames in %.1fs = %.1f fps (cmd %d)"
% (frames, dt, frames / max(dt, 1e-9), r.qi), flush=True)
print("live done: %d frames (%d skipped) in %.1fs = %.1f fps (cmd %d)"
% (frames, skipped, dt, frames / max(dt, 1e-9), r.qi), flush=True)