b366da6f1ec43bf7d8d5d4b04f45e85e2c9b8563
19
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  CydandClaude Fable 5
|
07eb74d1f7 |
BT410 5.3.80: joint articulation is live -- 22 nodes, and the twist reaches the board
RecurseSKLFile now builds a joint renderable for any node whose page name resolves to a live skeleton Joint: HingeX/Y/Z -> HingeRenderable (watching Joint::GetHinge), Ball -> BallJointRenderable (watching GetEulerAngles), otherwise the static path. Each holds the rest offset in one DCS and the live rotation in a child DCS, and its Execute diffs the watched value and calls DPL_FLUSH_DCS -- the engine's own mechanism (L4VIDRND.CPP:1026+). The value comes from the mech's JointSubsystem via ResolveJoint, so sim and renderer read one source. Gated on BT_JOINTS while it proves out. [skl] video\max.skl -> 26 nodes, 1 objects, 1 eye, 22 articulated The bridge reported anim_abs=1 joints=0 twist=+0.00 before; it now reports joints=1 twist=-0.86, matching the game's [torso] twist=-0.856. With the mech stationary, frames that differed by 0.0% now differ by 62-80%. A crash it exposed: Mech::ResolveJoint passed segment->GetJointIndex() straight to GetJoint unchecked, and a segment with no joint reports -1 -- GetNthImplementation then indexes [base + -1*4] and dies (guest 00426A1D). Torso never hit it because it only asks for its own authored joint name; the walk asks for every page. Now bounds-checked against GetJointCount. Open: the canopy does not stay rigid in the view, though it and the eye hang off the same articulated node. Cancelling the bridge's cage compensation (CAGE_TWIST_SIGN=0) did not close it. Leading hypothesis: SetupCull builds worldToEyeMatrix from GetSegmentToWorld(siteeyepoint) -- the SIMULATION's segment transform -- independent of the render tree, so the canopy follows our render chain and the eye follows the sim's, and they diverge whenever one carries the twist and the other does not. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
3c289ff10a |
BT410 5.3.78: the torso twists from the button path; two buffer overruns fixed
BT_FORCE_TORSO sweeps the twist BUTTONS -- the members the streamed direct mappings write -- so the chain runs without a hand on the RIO. Live on the pod: twist sweeps -0.015 to -0.891 rad and back as cmdL/cmdR alternate, at the resource rate 0.873 rad/s, inside the authored +/-2.443 limits, with jointtorso resolved. End to end: attribute id -> direct-mapping destination -> TorsoSimulation integrator -> skeleton joint -> the canopy eye hanging off that chain. 5.3.68's table and 5.3.71's integrator both confirmed against real button semantics. Two buffer overruns, one of them mine. Reading TORSO.CPP back caught the ctor still zeroing dynamicsState[22] after I shrank the array to [16] by carving the command members out of its front: 24 bytes past the end of every Torso, on the heap, every mission, since 5.3.68. A sweep for the same shape across BT/BT_L4/MUNGA found a pre-existing one -- MECHMPPR.CPP zeroing reserved[22] against reserved[21]. Both now bound by ELEMENTS(). The rule for this tree: any array carved out of a reserve block must have its initialiser bound by ELEMENTS, because the carve is exactly the edit that silently invalidates a literal. Neither explains the residual host fault (it predates 5.3.68 at 3/3), but both were real corruption running in every mission -- and the Torso one was introduced by the very fix that cut the fault rate, which is worth remembering whenever a rate MOVES instead of going to zero. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
e69d761fb8 |
BT410 5.3.77: the overrun drain loop is not the trigger either
serialnamedpipe's P_RX_BLOCKED path drains its whole backlog when the guest
hasn't read in time -- directserial's `while (doReceive());`. That looked
like the trigger, since a named pipe's backlog is unbounded where a real
port's is capped by the line rate. Bounded it to one byte (real UART overrun
semantics) and re-ran the conf that had faulted 3/3: FAULT 283s, FAULT 226s,
FAULT 204s. Killed.
The measurement that explains why it was never plausible: overruns during a
run are 1-3 per report period, because vRIO sends about a byte every 1-3ms --
the backlog is shallow and the loop had nothing to teleport. The 494/499
counts all land after the game exits. I read the code and inferred a burst
without measuring the queue depth it operates on.
Default restored to drain-all: it is the validated directserial behaviour the
RIO's rxpollus/rxburst tuning was calibrated against, and changing it on a
dead hypothesis would risk real-cockpit timing for nothing.
VPX_RX_OVERRUN_ONE=1 opts into the bounded form.
The roadmap now carries a fault ledger of every dead hypothesis so none get
re-run. What survives: a deterministic DPMI-host path walking a
{next,handler} chain into a node whose pointer is an unhooked IVT value,
entered under RIO interrupt load. Naming the owning routine needs a trace of
entries to host 0x66CF, clean vs faulting -- a DPMI32VM reversing
sub-project, now decoupled from the reconstruction (vRIO down = 100%
reliable, logged conf = ~70%).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
CydandClaude Fable 5
|
edb1670900 |
BT410 5.3.75: the quiet conf faults 3/3 -- logging SUPPRESSES the fault
Hypothesis: our BT_* logging inflates the load window, the window is the exposure, so a quiet run should load faster and fault less. Both halves wrong. pod_render_quiet -- which differs from pod_render_rec by exactly the six BT_* logging vars -- faulted 3/3, same signature, against ~25-40% for the logged conf. That is the strongest evidence yet that the residual is a timing-sensitive HOST bug rather than anything about our data: writing to COM3 changes when interrupts land relative to the DPMI host's handler-chain walk, and nothing about our attribute tables or renderables could plausibly be modulated by whether we print to a serial port. Interrupt phase can. Two immediate consequences: pod_render_rec is the LESS fault-prone rig, so use it; and the 'shipped survives' baseline weakens further, since shipped also writes plenty to COM3 and so sits on the suppressed side of the same effect on top of its shorter window. The wall-clock numbers also correct an earlier assumption: 150-275s boot+load, not ~60s. The 60s figure was the load DRAIN in game ticks; DOS boot, two diagnose passes and card init dominate the rest. Next is clearly emulator-side: the host keeps no EBP frames (the frame walk returned one bogus entry), so naming the routine that owns the chain needs a breakpoint-style trace of entries to 0x66CF with the head node, diffing a clean run against a faulting one. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
e97862507c |
BT410 5.3.74: the faulting instruction decoded -- a handler-chain walk with a bad node
The SegPhys-corrected dump read real code at last (the earlier zeros were the
probe reading EIP as a bare linear address; CS=00FF has a base):
66D4 push 0 / push edi / push ebx / push esi
66D9 call dword near [ebx+4] <- faults
66DC cmp eax,0 / jz done
66E5 mov ebx,[ebx] / jmp loop
A linked-list walk with a per-node callback -- node = {next@+0, handler@+4},
four args, "handled" on nonzero. A DPMI exception/interrupt handler chain.
The arithmetic closes exactly: the read is DS_base + ebx + 4, and
7000FA64 - F000CA64 = 80003000, so DS base is 0x80003000 and the famous cr2
is that wrap. My earlier '[EBX+0x3004]' reading was fabricated from the cr2
alone -- there is no 0x3004 displacement, which is why the constant appeared
in no binary.
So: a chain node's NEXT pointer holds F000CA60, the value parked in unhooked
IVT slots -- the walk expected a terminator and got an interrupt vector, then
read its +4 as a function pointer. Identical registers on every catch, so
one deterministic path, entered often enough that our 60s load window catches
it 25-40% of the time while shipped's 15s window mostly does not.
Open: which routine owns the chain, and where the bogus next came from. The
probe now walks the EBP frame chain to name the caller.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
CydandClaude Fable 5
|
2ce3528f1c |
BT410 5.3.73: the bad pointer is the emulator's default-handler stub
The IVT scan on the second probe catch settles EBX: F000:CA60 is the emulator's default unhandled-interrupt callback -- the value filling every vector nobody hooked (it matched ~56 of them), while the hardware-IRQ block IVT[08-0F] holds the DPMI host's own 0D1F reflector stubs. The serial IRQs are properly hooked; the host read one of the UNHOOKED vectors' default value and probed [value + 0x3004] as a flat pointer, wrapping to the famous cr2. Also corrected: the first dump read ESP/EIP as bare linear addresses -- the 'stack' bytes were DOS conventional memory (they decode as 16-bit real-mode code), since SS=0107 has a nonzero base. SegPhys-corrected dumps are built and armed for the next catch. The zeros at the faulting EIP are consistent with DPMI32VM being a virtual-memory host that pages its own regions -- 'Reference to a page you don't own' is its pager's abort message for an address outside every region it owns. Nailed: deterministic host-side path; the serial stream correlates as exposure, not as trigger bytes; the bad read is [unhooked-vector-default + 0x3004]. Open: which vector, which host routine -- the corrected dumps should name the return chain on the next catch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
f2687a3146 |
BT410 5.3.72: the fault caught in the act; the inside view gets its cockpit
VPX_PF_WATCH (fork, cpu/paging.cpp): on a guest page fault at the watched linear address, dump guest registers, the last 64 serial RX deliveries with guest cs:eip at each, the code bytes at the faulting EIP and the stack top. serialnamedpipe's doReceive feeds the ring. Armed in podrun.sh and launch_pod.ps1. The first catch decoded the residual fault completely: CS:EIP 00FF:000066D4 in the DPMI host, EBX = F000CA60 -- an IVT entry read as a dword, segment F000 offset CA60, a BIOS default interrupt handler -- and the faulting access is [EBX+0x3004], whose 0x80000000 segment-base wrap gives exactly cr2 7000FA64. The host probes a word 0x3004 bytes past a real-mode vector value treated as a flat pointer: harmless for its own low-memory handlers, a fault for BIOS F000:xxxx defaults. The serial ring shows a steady 1-byte/1-3ms vRIO stream with nothing special at the fault -- the stream determines which vectors get walked, not the crash itself. The 0x3004 appears nowhere in DPMI32VM.OVL or 32RTM.EXE as an immediate, so the probe now also dumps code bytes at EIP; faulthunt.sh loops runs until the next catch. Shipped baseline streak: 4/4 clean -- consistent with exposure, not yet discriminating. The inside view now loads the COCKPIT skeleton: the fleet-wide X-variant naming convention (MAD->MAX etc., all 64 skeletons present) selects the same 25-joint chain with a single object -- max_cop.bgf, the MAX_COP canopy shell with the PUNCH-texel windows from the capture forensics. The donor names the same mechanism from the decomp side (inside = SkeletonType_A with '_cop' selection). Fallback to the body skeleton when no X file exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
bd1070110b |
BT410 5.3.71: torso button commands wired into the sim; residual fault rate measured
TorsoSimulation now carries the donor's digital command handling (binary @004b5cf0): elevate up/down, twist left/right with limit clamps, the centre-button recenter latch slewing home across frames, and the ramp machinery kept verbatim including the binary's punchline -- the shipped build unconditionally overwrites the ramp with 1.0f, authored dead weight preserved as the binary's shape. recenterActive carved from the dynamicsState reserve. With 5.3.68's table fix the RIO/TM twist buttons land in these members and move the torso, and the canopy eye rides the twist chain. Also corrected in the roadmap: the 5.3.70 TriggerState 'wart' was not a wart -- CheckFireEdge already compares fireImpulse's bit pattern as a signed int, sign-correct for button ints and floats alike, and the donor binds and types the member identically. Fault rate, measured: riostreak.sh ran four consecutive vRIO-live runs -- CLEAN/CLEAN/FAULT/CLEAN, faults only ever in the load window (ticks 42-53k), clean runs always past launch. Post-fix tally 8 runs / 2 faults (~25%; was 3/3 before the Torso fix). Our load window is ~60s where shipped's is ~15s: if the residual is a constant-rate hazard during load, shipped's expected per-run rate is only ~7% -- 'shipped survives' may be exposure luck, and the residual is most likely an emulator-fork serial/DPMI interaction rather than a game defect. Next probe: hook the fork's page-fault path on the fixed cr2 and dump recent serial-IRQ deliveries + guest cs:ip history, plus a shipped-exe streak for the baseline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
8b9c2e6b11 |
BT410 5.3.70: the full direct-mapping contract audited; the fault is not fully dead
The canopy-eye confirming run (5th post-Torso-fix run with vRIO live) faulted with the old signature during the load drain -- four clean runs then one fault. The two-runs rule keeps earning its keep. The BT_MAP_LOG audit now prints the whole resolved contract, not just NULLs. The L4 list carries 13 direct mappings: mapper stick/throttle/reverse/looks (ids 3/4/6/10-12), Torso torsoCenter (id 14, matching the donor), and TriggerState (id 19 = the known 0x13 binding) on six weapons. Every destination resolves and every size is right -- the 8-byte joystick write lands in an 8-byte ControlsJoystick, buttons are ints into ints. One type wart noted, not fixed: MechWeapon binds TriggerState onto fireImpulse, a Scalar -- a ButtonGroup direct writes an int bit-pattern, so a real trigger press stores 1.4e-45f. Size-safe, but the fire FSM will never read a real button press as a pull; fire worked in tests only via the BT_FORCE_FIRE hook, which writes a proper float. The RIO trigger needs an int home for TriggerState. Surviving fault theories, in current order: an emulator/DPMI-host serial-IRQ interaction whose probability guest timing merely modulates (the fault's constants -- fixed host EIP, fixed cr2, always during the load drain -- fit this better than the corruption story ever did, since the smash point never moved across builds where our BSS moved); a second corruption source outside the direct mappings; sampling noise. riostreak.sh is measuring the post-fix rate over four consecutive runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
d4f105f42d |
BT410 5.3.69: the cockpit eye sits in the canopy -- site handling in the walk
RecurseSKLFile now walks "site=" children. Sites get no draw component -- which is why the walk's 26 nodes matched the real capture all along: sites were never DCS nodes -- and siteeyepoint spawns the cockpit camera with the donor's exact construction (bt411 btl4vid.cpp:462, decomp FUN_004579a8): offset = the site's own local rest transform, parent = the site's PARENT joint's DCS, not the hull root. World orientation, torso twist and gait all arrive through chain composition. The root-DCS eye remains only as the zero-construction fallback for skeletons with no siteeyepoint. Measured on the live pod with the RIO streaming: [eye] cockpit eye on 'jointeye', 26 nodes / 19 objects / 1 eye, and the bridge camera sits at Y = hull + 7.7 -- the exact chain sum jointlocal 5.29 + hip 0.37 + torso 0.31 + jointeye 1.69. canopy_eye1.png is the pilot's view, the mech's own nose wedge visible below the sightline. Own-mech cage handling is the next layer. Also: eye_count threading through ReadSKLFile/RecurseSKLFile, and the [skl] summary now reports it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
8231bee58c |
BT410 5.3.68: THE LIVE-RIO FAULT IS FIXED -- it and the TM crash were one defect
The [map] audit named it in one run: subsystem 17 = Torso, attribute ids 12/13 unresolved. The donor's decompiled torso.hpp carries the authentic enum with binary offsets -- ids 3..15, including StickPosition(9), TorsoUp/Down/ Left/Right(10-13), TorsoCenter(14), MotionState(15). Our table stopped at seven entries with MotionState at id 9, on the strength of a comment claiming the rest were messages. The streamed control mappings bind BY ID as direct WRITE destinations, so the truncation produced two different crashes from one cause: in TM mode the button ids 12/13 resolved NULL (the boot write-fault the moment the joystick polled), and in RIO mode the analog id 9 resolved to our motionState StateIndicator -- every analog packet from a live vRIO wrote raw floats over a watcher-socketed object, and the corrupted chains walked into unmapped memory ~30-40s later. That was the 'intermittent' pod fault at the fixed DPMI-host address. vRIO down = no analog = no corruption, which is why the norio conf launched: every observation from the whole hunt drops out of this mechanism. Fix: the full 13-entry donor table; five int command members carved from the dynamicsState reserve (class size unchanged); ctor zeros them. Verified, two agreeing runs each way: TM mode launches with a clean audit and the Thrustmaster joystick driving the torso (stickY=0.907 -> torsoElev 0.349); RIO mode with vRIO STREAMING launches, zero faults, weapons cycling. pod_render_rec is no longer poisoned and the norio workaround is obsolete. The audit-before-the-archive-call pattern (BT_MAP_LOG walks the same streamed table CreateStreamedMappings consumes, naming what will not resolve) turned a two-day intermittent-corruption hunt into a one-run lookup. The streamed RES tables are binding contracts on our attribute enums. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
1233a1d5f6 |
BT410 5.3.66: RIO-fault eliminations, and TM mode's own NULL-deref surfaced
Three clean eliminations on the live-RIO fault, all on the ack-fixed emulator: not the ack heuristic (reproduces after the fix), not the debug logging (pod_render_quiet faults identically with no BT_* env), and not the failed test-mode handshake (the quiet run's init PASSED and it faulted anyway). The Thrustmaster isolator -- L4CONTROLS=THRUSTMASTER so the game never opens the RIO while vRIO keeps streaming into serial1 -- was blocked by its own discovery: TM mode dies at boot in ControlsInstanceDirectOf<int>::Update+0x6, mov eax,[eax+0x1c] with EAX=0, at guest 00485E1A. That address resolves cleanly in our map and the loader names BTL4REC.EXE CODE 0x75E1A directly: a real defect in our controls wiring for the non-RIO path, and its own reconstruction brick. Standing facts for the hunt: our exe + live vRIO = fault in ~30-40s; shipped exe + the same stream on the same emulator = fine (two baselines). The ISR and LBE4ControlsManager are archive code; ours in the per-event path are the mapper (MECHMPPR) and the event/receiver plumbing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
d143f833ab |
BT410 5.3.64: THE COCKPIT EYE IS LIVE -- root+eye composition, and the emulator
deadlock it exposed The camera gap root-caused and fixed through three layers: LAYER 1 (ours): "chain the engine after the skeleton" was a non-fix. The engine's MakeEntityRenderables only accepts Object/Rubble resources, so chaining it with a Skeleton printed "wrong video resource type" and built NOTHING -- the fifodump proved it: zero vr_flush_dcs_artic records. The real fix mirrors the engine's Mover composition (L4VIDEO.CPP:4795-4860) inside our mech case: a Dynamic RootRenderable (whose ctor seeds its DCS from localToWorld and whose Execute re-flushes on change -- the ONLY source of per-frame wire articulation), the skeleton hung UNDER its DCS via ReadSKLFile's new parent_dcs parameter, and for the inside view a DPLEyeRenderable on that root with the published EyepointRotation. Also explains the mystery monolith in the first frame: the skeleton was parked at the world origin with the camera inside its shins. LAYER 2 (1995 library, read from our own linked symbols): dpl_DrawSceneComplete = velocirender_frameack(0), and frameack's unsolicited-reply path prints "dpl error - unsolicited input during frame ack", adds the 1995 authors' own puzzled "flush artic??" when the stray action is 0x1f, and calls exit(9). Documented before it ever fires. LAYER 3 (emulator, the actual deadlock): with per-frame articulation flowing for the first time, the game stopped drawing at frame 232 -- one 0x1f burst per frame forever, no receives, no error. The VPX device feeds the frame ack only after 6 CONSECUTIVE empty polls and reset that counter on EVERY outputData write; per-frame articulation writes made the count unreachable, so dpl_DrawSceneComplete never went true. Fix: the reset is gated on !frame_outstanding -- once a draw is outstanding the only receive the game will do next is the frame ack. The iserver-drain concern the reset guarded is boot-time only, when no frame is outstanding. Verified: two agreeing runs of ours on the fixed emulator (611 draws / 222 artic batches in run 2 -- the 232 wall is gone), camera travelling with the mech (cam -329.7,0,60.5 -> -362.3,0,54.5 across 8s), anim_abs 0 -> 1 on the bridge, view now INSIDE the cockpit cage. Shipped binary re-run on the fixed emulator: launches and runs, no regression. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
af010c18c2 |
BT410 5.3.63: the mech gets both skeleton and engine renderables
MakeEntityRenderables no longer stops after the skeleton. It builds ours AND
chains DPLRenderer, which is what creates the RootRenderable and the
DPLEyeRenderable the camera follows. Verified over three runs:
[mer] 208 class=3001 view=1 res=1
[vid] type=1 file='mad.skl'
[skl] video\mad.skl -> 26 nodes, 19 objects
[chain] into DPLRenderer, skeleton=yes, stack at 0x12dde7
No fault, mission launches, sim runs. Worth noting separately: the engine's
eyepoint construction -- prime suspect for the page fault through most of a
session -- runs perfectly here, with the stack at 0x12dde7, right beside the
main loop's frame. Both are consistent with the corrected finding that the
fault is a live-RIO artifact at a fixed DPMI host address.
The camera still does not move: two frames six seconds apart differ by zero
pixels while [sim] shows the mech travelling from (487,20,288) to (493,20,301),
and the bridge reads cam (0.0, 10.0, 0.0) -- world origin plus eye height. So
the eye renderable was necessary but not sufficient.
Eliminated by reading our own code rather than guessing: the viewpoint entity
IS the mech (BTL4APP.CPP:354), the video renderer IS linked to it
(APP.CPP:1337), and SetupCull would Fail() loudly if the mech lacked
"siteeyepoint" or EyepointRotation -- neither Fail appears. That leaves the
path from SetupCull's worldToEyeMatrix to what the bridge reads off the wire,
which is bridge territory rather than reconstruction territory.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
CydandClaude Fable 5
|
3c42766f48 |
BT410 5.3.62: correction -- the fault address was never in our code
Every conclusion that named EulerAngles::operator=(const LinearMatrix&) as the
fault site was wrong, including the disassembly built on top of it. Three
independent proofs:
1. ROTATION.CPP is ours, so the function was instrumented directly -- print
this, &matrix and a local's address on the first twelve calls plus any wild
pointer. The run faulted with ZERO [euler] lines: never called.
2. After the probe was added, 0x66D9 disassembles to a two-byte conditional
jump (jnl) that touches no memory. The dump reports ErrCode 0004, a data
READ.
3. EIP 0x66D9 and cr2 0x7000FA64 are identical to the byte across every build
this session, including builds where the code at that offset changed
completely. A fault in our CODE segment moves when the code moves. This
one does not -- it lives in the DPMI host, which loads low and never gets
relinked.
The map lookup was a coincidence: our _TEXT is section-relative from 0, so its
offsets overlap the host's low addresses, and the map will name a function for
any small number. Before resolving a fault address against the map, confirm it
belongs to our segment -- cheapest check is whether it moves on relink.
What is actually true, by same-binary A/B:
vRIO DOWN, serial1 still a namedpipe -> mission LAUNCHES
vRIO UP, same conf, same binary -> FAULT
The trigger is a LIVE RIO stream, not an unplugged cable. The emulator reports
continuous serial1 RX overruns, the fault lands wherever the app happened to be
(terrain one run, the mech's inside view another), and the shipped binary
survives the identical conditions. A fault at a fixed host address, at
arbitrary points in the app, requiring an interrupt-driven stream, is an
interrupt re-entrancy signature rather than a wild pointer.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
CydandClaude Fable 5
|
9c959e6891 |
BT410 5.3.60: first rendered frame from a running reconstructed mission
emulator/render-bridge/first_mission_frame.png -- arena city, textured buildings, horizon, 28fps, captured from the GL bridge while our build ran a live mission with the mech walking. The skeleton went in with it: [skl] video\mad.skl -> 26 nodes, 19 objects. Corrects yesterday's RIO framing. I called serial1 an unplugged cable. Wrong twice: VRio.App is running on this rig (the dev-rig default since 7/17), and the emulator log recorded real byte counts on that port -- RX overruns of 471, 503, 528 -- which an unconnected pipe cannot produce. The port was LIVE and streaming during every crashing run. So the defect is not that we mishandle a disconnected port, it is that we mishandle a live RIO stream, which is worse: every production pod has a real RIO. The shipped binary on the same rig logs 'lost RIO analog request', keeps sending characters, and runs the mission; ours logs 'RIO never came back from test mode!' and dies partway through the load. The overrun counts say the guest is not draining the port fast enough. L4RIO.CPP's handshake is authentic archive code, but MechRIOMapper (BT/MECHMPPR.CPP) is ours -- that is the thread. pod_render_norio.conf stays the way to get a running mission today, but it is a workaround: it disables the cockpit's primary input device. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
475ae3106e |
BT410 5.3.59: A MISSION RUNS -- the RIO serial port was blocking every load
pod_render_norio.conf is pod_render_rec.conf with one line changed, serial1=disabled instead of the vrio named pipe. Same binary, same egg. With it, our build launches and the mech walks: [launch] state=2 minPriorityEmpty=1 ticks=85316 [queues] p0=- p1=- p2=BUSY p3=- p4=- BTL4Application::RunMissionMessageHandler Turning Plasma Score Display On [sim] pos=(180.466,10,-358.703) yaw=-0.275605 spd=14.4 [sim] pos=(251.959,10,-250.498) yaw=-0.89103 spd=14.4 No fault, 290 log lines and counting. This is the first time the reconstruction has reached a running mission on the pod. The hang was never starvation, a deadlock, or a refill loop -- it was VOLUME, the first candidate I listed and then talked myself out of. 530 renderer events queue at priority 0 during load; BackgroundTasks::Execute runs exactly one task per call round-robin over seven tasks, and the 1ms frame budget is always blown here so no extra background passes happen. That is ~9 events per real second, so a load legitimately takes ~60s. The fault arrived at ~40s, before the backlog could clear. Every 'the queue never drains' reading was really 'the process dies before it can'. The disproof was already in hand: the post tally climbed to 530 and went FLAT, which means nothing was refilling. p0=BUSY with nextReady=1 is equally consistent with a large finite backlog still draining, and I read it as refill. Why the RIO port: the pipe has no server attached, which should behave as an unplugged cable. The emulator log shows steady serial1 RX overruns, our build prints 'RIO never came back from test mode!', and the SHIPPED binary prints 'lost RIO analog request' and launches anyway on this identical rig. So an unplugged RIO is survivable and our handling of it is not -- a real defect, not a rig artifact, since a pod with an unplugged RIO cable should still boot. The wild reference at EulerAngles::operator=(const LinearMatrix&)+0x19 is still unexplained, but it is now reproducible on demand by enabling serial1 rather than being a coin flip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
d3b332a62a |
BT410 5.3.58: the load gate is blocked by a REFILL loop on priority 0, not starvation
Per-priority occupancy, measured every second until the fault: [queues] p0=BUSY p1=- p2=BUSY p3=- p4=- nextReady=1 That single line kills the two leading theories. p3/p4 empty means nothing above priority 0 is competing, so the pump is free to serve it -- no starvation. nextReady=1 means PeekAtNextEvent always has a READY event, so priority 0 is not holding a timed event whose alarm never comes due -- no deadlock. What remains is refill: priority 0 is replenished as fast as the pump drains it, ~143 events per simulated second. It is also fully deterministic. Two runs of the same binary reported pump=352/1499 at frame 1001 and 495/2643 at frame 2002, identical to the byte. So 'crashes about half the time' was never a race -- it was comparing runs that differed in binary or conf. Only InterestManager::PostRendererEvent posts at priority 0 (it maps every renderer event there while the app is not RunningMission), so naming the message names the flooder. Application::Post now tallies priority-0 posts by message ID and reports the busiest three. Notably, all three producers of NotifyOfNewInterestingEntity are entity-CREATION paths, so if that ID dominates then something is creating entities in a loop -- which would explain the unbounded growth behind the fault as well as the hang. pod_render_noskl.conf now carries the same probes, so one instrumented binary can be run with the skeleton walk on and off. Walk-off runs are the only configuration of ours known to reach 'Turning Plasma Score Display On'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
CydandClaude Fable 5
|
4cf09917ce |
BT410 5.3.57: the crash is deterministic, and the real blocker is the load gate
The previous commit's attribution was wrong and is corrected in the roadmap. WRONG: 'the crash is the skeleton walk's' rested on 0 crashes in 2 runs with the walk disabled -- a 25% coin flip presented as evidence. The oldest preserved dump has no [skl] line and ends on the old 'couldn't figure out how to MakeEntityRenderables' fallback, so it predates the walk and faults identically. WRONG: 'it lands at different points each time'. Every dump carries the same numbers to the byte (cr2 7000FA64, EIP 66D9). What varies is how far the log gets, not where the fault is. Resolving the address through btl4opt.map names it exactly: EulerAngles::operator=(const LinearMatrix&)+0x19, a read through the matrix reference. A binary scan finds all three call sites of that operator pass a stack local, so the 1.79GB pointer still has no static explanation -- that is now its own open item rather than a guess. Two theories killed cleanly by the new BT_STACK_LOG probe and a binary scan: ESP drift (drift=0 over 2002 frames; exactly one callee-cleans function exists in the whole binary) and an undersized stack (our PE and the shipped one have identical 1MB/8K geometry). What the probe found matters more: the application is parked in state=2, which is LoadingMission, not WaitingForLaunch. Priority 0 is where the interest manager queues renderer events during load, the gate needs that priority empty, and it never empties -- so the mission never launches and the renderer holds a blank screen by design. The fault arrives ~30s into that wait. Runs that DO launch never print a single [launch] line. So 'crashes half the time' and 'hangs during load' are one event seen twice. A 15x disagreement between two clocks looked like a reconstruction slip -- BTL4.CPP passes GetTicksPerSecond() where ApplicationManager wants a frame rate. Checked against the shipped binary before touching it: same instruction sequence, same kind of static float pushed. Authentic. Documented so nobody 'fixes' it. Also swept every subsystem DefaultData against its real C++ base. Fifteen chain past their immediate parent, but fourteen skip only classes that add no handlers and no attributes, and no class's attribute-ID base disagrees with its index chain -- so there are no gap slots there. The one real defect: Generator is a HeatSink but chained to Subsystem::MessageHandlers, so it ignored every ToggleCooling message. Fixed (compiles next build). Tooling: podrun.sh stages the build over BTL4REC.EXE, exports the host-side VPX board env -- without which the run dies at the iserver handshake rather than merely rendering nothing -- and archives every run's log, marking it -CRASH when it faulted. Before this the only preserved dump was an accident, on a rig where each run costs four minutes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |