Commit the texel-format fix + shared dpl_sampler module (was uncommitted)

These changes were made and verified earlier this session (the all-cyan bug
fix is already recorded as SOLVED in project memory) but never got committed --
render_final.py has referenced dpl_sampler as a module in its docstring since
ac4142e7, yet dpl_sampler.py itself was never added, leaving the repo unable
to run the M4b/M4c/M4d renderer scripts that all import it.

texstore.py: fix the texel word format. SVT texels are [pad,B,G,R] (ground
truth: dpl3-revive patha/vrboard.py do_texels + stage_assets.py); reading RGB
as bytes (0,1,2)=(pad,B,G) zeroed red, producing the all-cyan render. Correct
read is bytes (3,2,1).

dpl_sampler.py: the authentic IG-board texture-value model (wrap repeat/clamp,
near-black CUT keying) factored into its own tested module, distilled from the
shipped libDPL headers -- see IG-SHADING-MODEL.md (also added, referenced by
render_final.py's docstring but likewise never committed).

igc_exec.py: OP48/OP49 candidate opcodes (pixel global-X/Y seed into mem) added
to the CPU golden-model executor alongside the existing SCMEMA seed handling.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Cyd
2026-07-20 10:07:16 -05:00
co-authored by Claude Opus 4.8
parent a689bce660
commit bf234842cb
6 changed files with 315 additions and 16 deletions
@@ -0,0 +1,139 @@
# The IG-board shading model (ground truth from libDPL)
Authoritative reference for the software rasterizer, distilled from the **shipped
Division library headers** in the reconstructed game source:
- `C:\VWE\BT411\engine\MUNGA_L4\libDPL\dpl\dpltypes.h` — the public `dpl_*` enums
- `C:\VWE\BT411\engine\MUNGA_L4\libDPL\dpl\dpl.h` — the `dpl_Set*` API
- `C:\VWE\RP411\DivLoader\{VGCDivLoader.h,tagIdents.h}` — the `.biz`/BGF asset
format the same model is authored in (DivVertex, BIZ_IDENTITIES)
- `C:\VWE\BT411\context\rendering.md` / `docs\BGF_FORMAT.md` — how the modern
D3D9 port reproduces this same model (the ramp/BSL/effect-card fidelity work)
These are the host-side commands the game issued to the VelociRender board; our
firmware-decomp emulates the board that consumes them. Where our renderer differs
from this model, this file is the reference.
---
## 1. Vertex type mask — the per-geometry shading selector
`dpl_VERTEX_TYPE` (dpltypes.h:189) is a bit mask; it is exactly the `DivVertex.mask`
field in the asset format (VGCDivLoader.h:65):
| bit | token | meaning |
|------|-------------------------|--------------------------------------------|
| 0x01 | `dpl_vertex_coord` | has x,y,z (always) |
| 0x02 | `dpl_vertex_normal` | has Nx,Ny,Nz → **LIT** by the map light |
| 0x04 | `dpl_vertex_rgba` | per-vertex RGBA (effect cards) |
| 0x08 | `dpl_vertex_luminance` | per-vertex luminance `l` |
| 0x10 | `dpl_vertex_texture2D` | has u,v |
| 0x20 | `dpl_vertex_texture3D` | has u,v,w (perspective texture) |
| 0x40 | `dpl_vertex_radius` | sphere/point radius |
**The key rule (rendering.md §"IG-board shading model"):** shading is chosen PER
GEOMETRY by the vertex type:
- **No-normal geometry** (terrain / mesas / sky / buildings / mech — the WHITE
baked verts) is **UNLIT**, and colorized by the material's 2-endpoint RAMP.
- **Normal-bearing geometry** (~150 vehicle/missile files) is **LIT** by the map
light; the ramp is NOT applied (applying it = the "dusty white blobs" bug —
gate ramp on `!hasNormals`).
- Exception: cockpit-frame (`*_cop`) batches are pure-emissive constant colour.
---
## 2. Material ramp — how the board colours grayscale texels
`dpl_SetMaterialRamp` + `dpl_SetRampComponents(r0,g0,b0, r1,g1,b1)` (dpl.h:326,578):
a material carries a **2-endpoint colour ramp**. The (grayscale / bit-sliced)
texture's **luminance L∈[0,1] indexes the low→high gradient**:
```
out_rgb = rampLo + L * (rampHi - rampLo) # per channel
```
This is how the IG board coloured terrain from gray bit-slice art (rendering.md /
bgfload.cpp:341): rock `0.25,0.21,0.16 → 0.8,0.5,0.4` (warm tan); grass
`0.13,0.13,0.07 → 0.38,0.33,0.23`. Materials with no diffuse relied ENTIRELY on
the ramp.
> **Applicability to the wire-capture renderer:** the `0x1a` texmaps in the
> `netdeath` capture arrive as *already-colourised RGB* (HUD labels, emblems,
> panel/terrain art — 012% grayscale), i.e. the ramp is pre-baked on this path.
> So `render_final` must **not** re-apply a ramp (it would double-colour). The
> ramp model matters for a raw grayscale/BSL texmap path, not for these uploads.
---
## 3. Texture value model — wrap and alpha (implemented: `dpl_sampler.py`)
`dpl_SetTextureProperty(tex, prop, value)` (dpl.h:344) sets `dpl_TEX_PROP` fields
to `dpl_TEX_VALUE` tokens (dpltypes.h:103,121). The two that change rasterizer
*output*:
**WRAP** (`dpl_tex_prop_wrap`/`wrapu`/`wrapv`):
- `dpl_tex_value_repeat` — texel index wraps mod size (tiling terrain) — the board default
- `dpl_tex_value_clamp` — texel index clamps to the edge (a one-shot decal / HUD
label / emblem — tiling a "PLAYER 1" label is the wrong-wrap tell)
**ALPHA / cutout** (`dpl_tex_prop_alpha`):
There is **no stored alpha channel** — the texel word is `[pad, B, G, R]` (see §3a).
The board realises `dpl_tex_value_cut` (**PUNCH**: the cockpit/effect-card cutout
that retires the "opaque rectangle" reading — the wreck flame that read as a
"twisted drill bit", rendering.md §effect-cards) by **near-black keying** at draw
time: a texel whose `R+G+B <= ~24` is a HOLE (poly not drawn there, no z claim).
Ground truth: `dpl3-revive/patha/vrview_gl.py` ("alpha cutout (texel sum <= 24/255)").
Filtering (`point`/`bilinear`/`trilinear`/`mipmap_*`) is a quality axis; pvision
runs the board in texmode 0 = **point (nearest)**, which is what we model.
`dpl_sampler.py` implements wrap + near-black cutout (numpy-vectorized, with a
synthetic conformance self-test). Cutout is a per-MATERIAL property the wire
capture does not yet expose per drawn surface, so `render_final` defaults it OFF
(opaque) and leaves the hook for when a `SetTextureProperty` stream is decoded.
## 3a. Texel format — SVT `[pad, B, G, R]` (the ex-CYAN bug)
The `0x1a` upload's texel word is **`[pad, B, G, R]`** ("SVT xbgr") — ground truth
from the real renderer's own decode (`dpl3-revive/patha/vrboard.py:448` `do_texels`,
`stage_assets.py:43`). **Byte 0 is a PAD (0 in every texel across the capture);
the colour is bytes 1,2,3 = B,G,R, so RGB = bytes (3,2,1).** `mode` 1 is the
board's internal 8-bit storage path — the WIRE upload is always 4 bytes/texel.
`texstore.py` had been reading RGB as bytes (0,1,2) = `(pad, B, G)` — zero red —
which rendered the whole arena **CYAN**. Fixed to (3,2,1): the arena reads as its
true grayscale metal panels + coloured decals. This, not the texid binding (§5),
was the cyan.
---
## 4. Fog — the arcade model (not yet in the software renderer)
`dpl_FOG_TYPE` (dpltypes.h:238): the arcade uses **`dpl_fog_type_pixel_lin`** —
per-PIXEL linear fog on perspective W (rendering.md §fog: table fog on the
z-buffer collapses because the z-buffer is perspective-nonlinear; the board fogs
on W). Authored per map/time/weather as `fog = <near> <far> <r> <g> <b>` in the
DPL env INI (e.g. cavern/night/clear: near 90, far 1100, dark blue 0.1,0.1,0.12).
Night fog is dark BY DESIGN. Our renderer has the per-pixel Z/W planes to do this
faithfully; the near/far/colour are map-specific (not applied blind).
---
## 5. The remaining blocker — texid → texmap-handle binding
`render_final` still binds each drawn quad's 6-bit texid to an uploaded handle by
`texlist[tid % len]` — a placeholder. Investigated 2026-07-19/20: the draw's
texture-select word (op `0xf7`, ad 141/142) is **not** a handle — it is the
board's *resolved texture-descriptor reference* (per-texmap: high byte differs,
`0xf26a`/`0xf081`/`0xf1ea`/`0xcafa`…), computed at geometry-CREATE time from the
texmap (traced: the select-word value is written to scratch ~`0xbfc98` during the
`create` cmds, not the opaque texel store), and `battle_prog.pkl` is the
firmware's already-compiled program, so the game's material/texture *names* are
gone by then. Cracking it needs the deferred **board texture-RAM model**: RE the
firmware texmap→descriptor allocation and map the select word to an upload handle.
**Note (2026-07-20):** the monochromatic frame was NOT this binding — it was the
texel-format misread (§3a). With SVT decode fixed, most arena surfaces are
grayscale metal, so the frame reads correctly *despite* the placeholder binding;
the binding now only mis-tints the handful of coloured decals. Much lower priority.
@@ -0,0 +1,130 @@
"""dpl_sampler.py -- the authentic IG-board texture-value model, distilled from
the shipped Division library headers (BT411 engine/MUNGA_L4/libDPL/dpl/dpltypes.h
`dpl_TEX_VALUE` + dpl.h `dpl_SetTextureProperty`) cross-checked against the real
renderer (dpl3-revive patha/vrview_gl.py, stage_assets.py, vrboard.py).
The board applies a texmap over a polygon under per-texture PROPERTY values
(`dpl_TEX_PROP` set to a `dpl_TEX_VALUE`). Two affect the rasterizer's *output*:
* WRAP (dpl_tex_prop_wrap / wrapu / wrapv):
dpl_tex_value_repeat -> texel index wraps modulo the map size (default)
dpl_tex_value_clamp -> texel index clamps to the edge
A tiling terrain uses REPEAT; a one-shot decal / HUD label / emblem CLAMPs
(tiling a "PLAYER 1" label is the tell-tale wrong-wrap artefact).
* ALPHA / cutout (dpl_tex_prop_alpha, dpl_tex_value_cut):
the texel word is [pad, B, G, R] (SVT xbgr) -- there is NO stored alpha
channel. The board realises `dpl_tex_value_cut` (PUNCH: the cockpit /
effect-card cutout that retires the "opaque rectangle" bug) by NEAR-BLACK
KEYING at draw time: a texel whose R+G+B is <= a small threshold is a
HOLE (the poly is not drawn there, and does not claim z). Ground truth:
patha/vrview_gl.py ("alpha cutout (texel sum <= 24/255)").
Filtering (point/bilinear/...) is a quality axis; pvision runs texmode 0 = point
(nearest), which is what the rasterizer models.
Pure + numpy-vectorized so the frame renderer applies it over a whole tile at
once; independent of the wire capture so it is conformance-tested on synthetic
input (see __main__).
"""
import numpy as np
# dpl_TEX_VALUE tokens we act on (names mirror dpltypes.h)
WRAP_REPEAT = 'repeat'
WRAP_CLAMP = 'clamp'
CUT_OFF = 'off' # dpl_tex_value off -> opaque
CUT_KEY = 'cut' # dpl_tex_value_cut -> near-black keying (PUNCH)
# Near-black cutout threshold (R+G+B), from patha/vrview_gl.py (sum <= 24/255).
CUT_THRESHOLD = 24
# The wire's per-texmap `mode` word (0x1a upload header word[4]) selects the
# board's internal storage path (mode 1 = 8-bit). It is NOT reliably a cutout
# flag -- cutout is a per-MATERIAL property (dpl_SetTextureProperty), which the
# wire capture does not yet expose per drawn surface. So wrap defaults to REPEAT
# (the board default) and cutout defaults to OFF until a SetTextureProperty
# stream is decoded; callers can override per texmap.
def mode_to_texflags(mode):
"""Decode a wire texmap `mode` word into (wrap_u, wrap_v, cut_mode)."""
return WRAP_REPEAT, WRAP_REPEAT, CUT_OFF
def wrap_index(coord, size, wrap):
"""Vectorized texel index for integer texel coords under a wrap mode."""
c = np.asarray(coord, dtype=np.int64)
if wrap == WRAP_CLAMP:
return np.clip(c, 0, size - 1)
return np.mod(c, size) # REPEAT (default)
def composite(dst_rgb, inside, src_rgb, cut_mode, cut_thresh=CUT_THRESHOLD):
"""Apply a textured polygon's texels to the framebuffer under the board
texture-value model. Returns the boolean DRAW mask (pixels the poly actually
owns -- the caller gates the z-buffer write with it so CUT holes don't claim
z).
dst_rgb : HxWx3 uint8 framebuffer (modified in place on the draw mask)
inside : HxW bool -- candidate pixels (edge AND z test already done)
src_rgb : HxWx3 -- sampled texel colour (RGB)
cut_mode : CUT_OFF (opaque) or CUT_KEY (near-black keying)
"""
if cut_mode == CUT_KEY:
lum = src_rgb.astype(np.int32).sum(axis=2)
draw = inside & (lum > cut_thresh) # near-black texels are holes
else:
draw = inside
dst_rgb[draw] = src_rgb[draw]
return draw
def composite_masked(dst_rgb, sel, src_rgb_compact, cut_mode, cut_thresh=CUT_THRESHOLD):
"""Like composite(), but src_rgb_compact holds only the SEL-true samples
(shape (K, C), K = sel.sum()), in the order sel's True entries appear under
numpy boolean indexing. Lets a caller sample/composite only the pixels a
texture actually covers instead of the whole frame -- the same board
texture-value model, just without doing (frame_size / coverage) redundant
work when many textures each cover a small fraction of the frame (see
gpu_raster.Renderer.frame, where this was the dominant per-frame cost).
"""
if cut_mode == CUT_KEY:
lum = src_rgb_compact.astype(np.int32).sum(axis=1)
keep = lum > cut_thresh
idx = np.flatnonzero(sel)[keep]
dst_rgb.reshape(-1, dst_rgb.shape[-1])[idx] = src_rgb_compact[keep]
else:
dst_rgb[sel] = src_rgb_compact
if __name__ == '__main__':
# -- wrap conformance --
assert list(wrap_index([-1, 0, 3, 4], 4, WRAP_REPEAT)) == [3, 0, 3, 0]
assert list(wrap_index([-1, 0, 3, 4], 4, WRAP_CLAMP)) == [0, 0, 3, 3]
# -- cutout conformance on a 2x2 scene --
dst = np.zeros((2, 2, 3), np.uint8)
inside = np.ones((2, 2), bool)
# (0,0) near-black (hole under CUT), others bright
src = np.array([[[2, 3, 4], [200, 10, 10]],
[[10, 200, 10], [10, 10, 200]]], np.uint8)
# OFF: everything drawn, keying ignored
d = dst.copy(); m = composite(d, inside, src, CUT_OFF)
assert m.all() and (d[0, 0] == [2, 3, 4]).all()
# CUT: the near-black texel (sum 9 <= 24) is a hole (not drawn, no z claim)
d = dst.copy(); m = composite(d, inside, src, CUT_KEY)
assert not m[0, 0] and m[0, 1] and m[1, 0] and m[1, 1]
assert (d[0, 0] == 0).all() and (d[0, 1] == [200, 10, 10]).all()
# -- composite_masked must match composite() exactly, on a partial `sel`
# (a subset of `inside`, as gpu_raster.Renderer.frame uses it per texture) --
sel = np.array([[True, False], [True, True]]) # skip (0,1) this time
for cm in (CUT_OFF, CUT_KEY):
d_full = dst.copy(); composite(d_full, sel, src, cm)
d_masked = dst.copy(); composite_masked(d_masked, sel, src[sel], cm)
assert (d_full == d_masked).all(), (cm, d_full, d_masked)
print("composite_masked matches composite() on a partial mask, both cut modes: PASS")
print("dpl_sampler conformance PASS "
"(repeat/clamp wrap + off/cut near-black keying)")
@@ -3,10 +3,9 @@ texel scale/tiling) + per-quad texid -> texture (best-effort binding, since the
exact texid->handle needs the board texture-RAM model). Real colours from real
textures, perspective-correct.
Now applies the authentic IG-board texture-value model (dpl_sampler, distilled
from libDPL dpl_TEX_VALUE): the texmap `mode` bit selects alpha handling, and
alpha textures render with the board's CUT (punch) model -- fully-transparent
texels are holes, not opaque rectangles. See IG-SHADING-MODEL.md."""
Textures decode via the SVT [pad,B,G,R] format (texstore -> RGB); the board
texture-value model (wrap + near-black cutout) is applied through dpl_sampler,
distilled from libDPL dpl_TEX_VALUE + the real renderer. See IG-SHADING-MODEL.md."""
import pickle, struct, os, sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, r'C:\VWE\TeslaRel410\dpl3-revive\patha')
@@ -62,12 +61,11 @@ for a in setups:
z=zp[0]*xx+zp[1]*yy+zp[2]
inside&=z>zbuf
if not inside.any(): continue
# z is claimed only on pixels the poly actually OWNS -- deferred past the
# alpha-cut below so PUNCH holes don't win the depth test.
# z is claimed only on pixels the poly actually OWNS (deferred past any cutout)
if up and vp and tzp:
# best-effort texid -> texture (exact binding = board tex-RAM model, pending)
h,(tu,tv,mode,arr)=texlist[(tid or 0)%len(texlist)]
wrap_u,wrap_v,alpha_mode=mode_to_texflags(mode)
wrap_u,wrap_v,cut_mode=mode_to_texflags(mode)
tzv=tzp[0]*xx+tzp[1]*yy+tzp[2]
tzv=np.where(np.abs(tzv)<1.0, np.sign(tzv)+ (tzv==0), tzv)
# VERIFIED divide: texel8 = (texu/texz)*2048 (3 int + 8 texel bits), tiled to tex size
@@ -75,9 +73,9 @@ for a in setups:
vcoord=((vp[0]*xx+vp[1]*yy+vp[2])/tzv*2048).astype(np.int64)
ui=wrap_index((ucoord*tu)>>8, tu, wrap_u) # 8-bit texel -> tex-size index
vi=wrap_index((vcoord*tv)>>8, tv, wrap_v)
samp=arr[vi,ui] # HxWx4 RGBA
# apply the board alpha model (opaque / cut / blend); returns the drawn mask
drawmask=composite(img, inside, samp[...,:3], samp[...,3], alpha_mode)
samp=arr[vi,ui] # HxWx3 RGB (SVT xbgr -> RGB in texstore)
# board texture-value model: opaque, or CUT (near-black keying), see dpl_sampler
drawmask=composite(img, inside, samp, cut_mode)
else:
img[inside]=(60,60,70)
drawmask=inside
@@ -31,7 +31,7 @@ print("texstore: %d textures %s"%(len(store),[hex(h) for h in list(store)[:8]]))
by_size=sorted(store.items(), key=lambda kv:-kv[1][0]*kv[1][1])
panel=None
for h,(u,v,m,arr) in store.items():
if u==128 and v==128: panel=(u,v,arr); break
if u==128 and v==128: panel=(u,v,arr); break # store returns RGB (SVT-decoded)
if panel is None:
h,(u,v,m,arr)=by_size[0]; panel=(u,v,arr)
pu,pv,parr=panel
+15 -5
View File
@@ -1,8 +1,16 @@
"""M3 texel store: parse the game wire's action-0x1a texture uploads into a
handle -> (u_size, v_size, mode, RGB array) dict. Format: header[32] =
{handle, byte_size, u, v, mode, ...}, then v rows of 256 bytes, texels in
0x00BBGGRR little-endian words (bytes R,G,B,0). Verified: decodes to the real
game textures (HUD 'PLAYER n' labels, squadron emblem, panel/terrain art)."""
{handle, byte_size, u, v, mode, ...}, then v rows of 256 bytes, then n=u*v texels.
TEXEL WORD = [pad, B, G, R] bytes -- "SVT xbgr" (ground truth: dpl3-revive
patha/vrboard.py:448 `do_texels` + stage_assets.py:43, the actual renderer's own
decode). Byte 0 is a PAD (verified: 0 in every texel across the capture); the
colour is bytes 1,2,3 = B,G,R, so RGB = bytes (3,2,1). There is NO alpha byte --
cutout ("dpl_tex_value_cut") is done by NEAR-BLACK keying at draw time, not a
stored channel (patha/vrview_gl.py). `mode` 1 is the board's internal 8-bit
storage path; the WIRE upload is always 4 bytes/texel regardless of mode.
Reading RGB as bytes (0,1,2) = (pad,B,G) -- zero red -- was the all-CYAN bug."""
import struct
import numpy as np
@@ -22,8 +30,10 @@ def build_texstore(recs):
n = u * v
if n and len(rb) >= n * 4:
arr = np.frombuffer(rb[:n * 4], dtype=np.uint8).reshape(v, u, 4)
# v-flip (rows arrive bottom-up: 'PLAYER' reads mirrored otherwise)
store[handle] = (u, v, mode, arr[::-1, :, :3].copy())
# texel = [pad,B,G,R] -> RGB = bytes (3,2,1); v-flip (rows arrive
# bottom-up: 'PLAYER' reads mirrored otherwise)
rgb = arr[::-1, :, 3:0:-1].copy() # (R,G,B) from (3,2,1)
store[handle] = (u, v, mode, rgb)
i = j
else:
i += 1
+22
View File
@@ -256,6 +256,28 @@ class Tile:
for i in range(NPIX):
if not self.enab[i]:
self.mem[i][a >> 3] &= ~(1 << (a & 7))
elif m == 'OP48':
# FCMEMA candidate: seed pixel GLOBAL X into mem[addr:len]
_, addr, ln, operand = ins
ln = max(1, min(20, ln if ln > 0 else 8))
mask = (1 << ln) - 1
for y in range(TILE_H):
ybase = y * TILE_W
for x in range(TILE_W):
i = x + ybase
if self.enab[i]:
self._wr(self.mem[i], addr, ln, (x + self.ox) & mask)
elif m == 'OP49':
# y-variant candidate
_, addr, ln, operand = ins
ln = max(1, min(20, ln if ln > 0 else 8))
mask = (1 << ln) - 1
for y in range(TILE_H):
ybase = y * TILE_W
for x in range(TILE_W):
i = x + ybase
if self.enab[i]:
self._wr(self.mem[i], addr, ln, (y + self.oy) & mask)
elif m == 'SCMEMA':
# seed: write the pixel's GLOBAL X screen address into mem[addr:len]
_, addr, ln, operand = ins