9c959e68919b49f75f736be1d48ec9ab770d94b9
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  CydandClaude Fable 5
|
9c959e6891 |
BT410 5.3.60: first rendered frame from a running reconstructed mission
emulator/render-bridge/first_mission_frame.png -- arena city, textured buildings, horizon, 28fps, captured from the GL bridge while our build ran a live mission with the mech walking. The skeleton went in with it: [skl] video\mad.skl -> 26 nodes, 19 objects. Corrects yesterday's RIO framing. I called serial1 an unplugged cable. Wrong twice: VRio.App is running on this rig (the dev-rig default since 7/17), and the emulator log recorded real byte counts on that port -- RX overruns of 471, 503, 528 -- which an unconnected pipe cannot produce. The port was LIVE and streaming during every crashing run. So the defect is not that we mishandle a disconnected port, it is that we mishandle a live RIO stream, which is worse: every production pod has a real RIO. The shipped binary on the same rig logs 'lost RIO analog request', keeps sending characters, and runs the mission; ours logs 'RIO never came back from test mode!' and dies partway through the load. The overrun counts say the guest is not draining the port fast enough. L4RIO.CPP's handshake is authentic archive code, but MechRIOMapper (BT/MECHMPPR.CPP) is ours -- that is the thread. pod_render_norio.conf stays the way to get a running mission today, but it is a workaround: it disables the cockpit's primary input device. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
475ae3106e |
BT410 5.3.59: A MISSION RUNS -- the RIO serial port was blocking every load
pod_render_norio.conf is pod_render_rec.conf with one line changed, serial1=disabled instead of the vrio named pipe. Same binary, same egg. With it, our build launches and the mech walks: [launch] state=2 minPriorityEmpty=1 ticks=85316 [queues] p0=- p1=- p2=BUSY p3=- p4=- BTL4Application::RunMissionMessageHandler Turning Plasma Score Display On [sim] pos=(180.466,10,-358.703) yaw=-0.275605 spd=14.4 [sim] pos=(251.959,10,-250.498) yaw=-0.89103 spd=14.4 No fault, 290 log lines and counting. This is the first time the reconstruction has reached a running mission on the pod. The hang was never starvation, a deadlock, or a refill loop -- it was VOLUME, the first candidate I listed and then talked myself out of. 530 renderer events queue at priority 0 during load; BackgroundTasks::Execute runs exactly one task per call round-robin over seven tasks, and the 1ms frame budget is always blown here so no extra background passes happen. That is ~9 events per real second, so a load legitimately takes ~60s. The fault arrived at ~40s, before the backlog could clear. Every 'the queue never drains' reading was really 'the process dies before it can'. The disproof was already in hand: the post tally climbed to 530 and went FLAT, which means nothing was refilling. p0=BUSY with nextReady=1 is equally consistent with a large finite backlog still draining, and I read it as refill. Why the RIO port: the pipe has no server attached, which should behave as an unplugged cable. The emulator log shows steady serial1 RX overruns, our build prints 'RIO never came back from test mode!', and the SHIPPED binary prints 'lost RIO analog request' and launches anyway on this identical rig. So an unplugged RIO is survivable and our handling of it is not -- a real defect, not a rig artifact, since a pod with an unplugged RIO cable should still boot. The wild reference at EulerAngles::operator=(const LinearMatrix&)+0x19 is still unexplained, but it is now reproducible on demand by enabling serial1 rather than being a coin flip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
d3b332a62a |
BT410 5.3.58: the load gate is blocked by a REFILL loop on priority 0, not starvation
Per-priority occupancy, measured every second until the fault: [queues] p0=BUSY p1=- p2=BUSY p3=- p4=- nextReady=1 That single line kills the two leading theories. p3/p4 empty means nothing above priority 0 is competing, so the pump is free to serve it -- no starvation. nextReady=1 means PeekAtNextEvent always has a READY event, so priority 0 is not holding a timed event whose alarm never comes due -- no deadlock. What remains is refill: priority 0 is replenished as fast as the pump drains it, ~143 events per simulated second. It is also fully deterministic. Two runs of the same binary reported pump=352/1499 at frame 1001 and 495/2643 at frame 2002, identical to the byte. So 'crashes about half the time' was never a race -- it was comparing runs that differed in binary or conf. Only InterestManager::PostRendererEvent posts at priority 0 (it maps every renderer event there while the app is not RunningMission), so naming the message names the flooder. Application::Post now tallies priority-0 posts by message ID and reports the busiest three. Notably, all three producers of NotifyOfNewInterestingEntity are entity-CREATION paths, so if that ID dominates then something is creating entities in a loop -- which would explain the unbounded growth behind the fault as well as the hang. pod_render_noskl.conf now carries the same probes, so one instrumented binary can be run with the skeleton walk on and off. Walk-off runs are the only configuration of ours known to reach 'Turning Plasma Score Display On'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
4cf09917ce |
BT410 5.3.57: the crash is deterministic, and the real blocker is the load gate
The previous commit's attribution was wrong and is corrected in the roadmap. WRONG: 'the crash is the skeleton walk's' rested on 0 crashes in 2 runs with the walk disabled -- a 25% coin flip presented as evidence. The oldest preserved dump has no [skl] line and ends on the old 'couldn't figure out how to MakeEntityRenderables' fallback, so it predates the walk and faults identically. WRONG: 'it lands at different points each time'. Every dump carries the same numbers to the byte (cr2 7000FA64, EIP 66D9). What varies is how far the log gets, not where the fault is. Resolving the address through btl4opt.map names it exactly: EulerAngles::operator=(const LinearMatrix&)+0x19, a read through the matrix reference. A binary scan finds all three call sites of that operator pass a stack local, so the 1.79GB pointer still has no static explanation -- that is now its own open item rather than a guess. Two theories killed cleanly by the new BT_STACK_LOG probe and a binary scan: ESP drift (drift=0 over 2002 frames; exactly one callee-cleans function exists in the whole binary) and an undersized stack (our PE and the shipped one have identical 1MB/8K geometry). What the probe found matters more: the application is parked in state=2, which is LoadingMission, not WaitingForLaunch. Priority 0 is where the interest manager queues renderer events during load, the gate needs that priority empty, and it never empties -- so the mission never launches and the renderer holds a blank screen by design. The fault arrives ~30s into that wait. Runs that DO launch never print a single [launch] line. So 'crashes half the time' and 'hangs during load' are one event seen twice. A 15x disagreement between two clocks looked like a reconstruction slip -- BTL4.CPP passes GetTicksPerSecond() where ApplicationManager wants a frame rate. Checked against the shipped binary before touching it: same instruction sequence, same kind of static float pushed. Authentic. Documented so nobody 'fixes' it. Also swept every subsystem DefaultData against its real C++ base. Fifteen chain past their immediate parent, but fourteen skip only classes that add no handlers and no attributes, and no class's attribute-ID base disagrees with its index chain -- so there are no gap slots there. The one real defect: Generator is a HeatSink but chained to Subsystem::MessageHandlers, so it ignored every ToggleCooling message. Fixed (compiles next build). Tooling: podrun.sh stages the build over BTL4REC.EXE, exports the host-side VPX board env -- without which the run dies at the iserver handshake rather than merely rendering nothing -- and archives every run's log, marking it -CRASH when it faulted. Before this the only preserved dump was an accident, on a rig where each run costs four minutes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
f54e1a3c19 |
BT410 5.3.56: the intermittent pod crash is the skeleton walk's -- attributed by same-binary A/B
Added BT_NO_SKL, which skips the skeleton build so ONE executable can be run with and without it. That is the only clean way to test an intermittent fault, and it settles attribution: walk enabled ~50% of runs die (page fault at 66D9) at DIFFERENT points walk disabled 0 crashes in 2 runs shipped exe 0 crashes in 2 runs, same rig and conf So it is our new code, not the rig, and the varying fault point points at memory corruption surfacing later rather than a bad instruction in the walk. Hypotheses killed by reading the private library headers: the matrix array is the right size (dpl_MATRIX is float32[4][4]); and neither s_dplobject nor the common dpl_node header retains an object name, weakening the dangling-string theory (a private name cache inside the library is still possible and is the best surviving suspect). Also checked something that would have been much worse than a crash: ~NotationFile REWRITES its file when dirty, and these are the game's shipped .SKL data files. MAD.SKL is untouched, and ReadSKLFile now carries the engine's own Verify(!IsDirty()) guard. Next tests are listed cheapest-first in the roadmap: keep the NotationFile alive, copy the Object= names, count board allocations, and re-check whether MakeEntityRenderables runs twice per mech. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
15d509c8a8 |
BT410 5.3.52: frame rate measured properly -- the missing mech skeleton is the real gap
The pod updates every few seconds. Measured by sampling a head window every
2s (emulator/render-bridge/headrate.py): OURS changed in 5 of 12 samples,
SHIPPED in 1 of 12. So the reconstruction is not slower than the shipped
binary -- the emulated board is just expensive. Caveat recorded with it:
ours was being driven by the throttle hooks while the shipped exe ignores
them and sat parked, so treat those as same-order, not a win.
What did cost us 3x was mine: BT_MECH_LOG/BT_LAUNCH_LOG write per-frame lines
and DEBUG_STREAM=cout is redirected to COM3, so every one goes through an
emulated serial port. Turning them off took the wire from 480 to ~1480
bytes/sec. pod_render_quiet.conf is the conf to use for timing work.
And a correction to my own earlier framing: wire bytes/sec is NOT a frame
rate. Shipped pushes ~8900 B/s against our ~1480 and the difference is
CONTENT, not speed -- shipped submits the mech and we do not:
OURS: L4VIDEO.cpp wrong video resource type for object mad.skl
SHIPPED: (no such line)
The mech's model resource is a SKELETON. The engine's default
MakeEntityRenderables accepts only Object/Rubble and rejects anything else
with exactly that message, because skeletons are the GAME renderer's job. So
our arena renders but the MECH IS ABSENT, and with it the cockpit interior --
one unimplemented path explaining both the missing model and the lower wire
volume.
Next brick is now precisely scoped: answer MechClassID by reading the .SKL
notation pages into a dpl_DCS hierarchy and hanging the per-node .BGF
geometry off it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
CydandClaude Fable 5
|
09192367c0 |
BT410 5.3.51: THE 3-D WORLD RENDERS -- reconstruction to emulated board to a picture
emulator/render-bridge/first-3d-frame.png: the arena from the pod -- sky, horizon, ground, the arena structures along the skyline -- at ~31fps on the VelociRender bridge. The whole chain now works in our build: MakeVideoRenderer -> BTL4VideoRenderer over DPLRenderer -> board boot -> Renderer::LoadMission -> DPLReadEnvironment for the art paths -> 40/40 arena objects loaded -> mission launch -> per-frame submission over the wire. The mistake worth remembering: the run was never stalled after InitializePlayerLink. I called that from a 150-second sample and it was just too short. CheckLoadMessageHandler reposts every second and will not advance until the min-priority event queue drains, and with the video renderer in the mission that takes minutes. A BT_LAUNCH_LOG trace showed the queue draining and then both RunMissionMessageHandler calls, the plasma display, and the first sensor tick -- matching the shipped binary line for line. The renderer deliberately holds a blank screen until RunningMission (L4VIDEO.CPP:5100), so 'black' was the app still loading, not a rendering bug. Zero unbuildable entities, zero geometry failures. The frame is static only because the mech is parked with no input, and the camera in the bridge title is the bridge's own viewer, not the game's eyepoint. Also banked the launch-gate trace (BT_LAUNCH_LOG) behind an env var. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |