vwemirror/BT411
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

pod: fix the podkit INFINITE LOOP, version-control podkit.ps1, and write the remote runbook

Browse Source 
Deploying 4.11.854 to the cart hit a wall that was entirely self-inflicted and
entirely undocumented.  Both are fixed here.

THE BUG.  podkit.ps1 (which pushes the frozen rig config into a fresh install)
had, at line 30:

    while ($keep.Count -gt 0 -and $keep[-1].Trim() -eq '') { $keep = $keep[0..($keep.Count-2)] }

Once $keep trims to a SINGLE blank line, $keep.Count-2 is -1, and PowerShell's
$keep[0..-1] returns TWO elements (index 0 and index -1) instead of shrinking --
so the array GROWS and the loop never ends, RSS climbing past 60 MB.

It fires whenever everything outside environ.ini's marker block is blank, i.e.
any environ.ini that was ALREADY kitted -- which is exactly what you get when you
carry config forward from the previous install, the normal upgrade path.  So this
would have bitten every future deploy, not just this one.

Signature: a BLANK cmd console on the cab, BT411Run stuck "Running", no btl4.exe,
no podrun.log.  And it CASCADES: each hung run holds environ.ini so every later
attempt blocks behind it -- while over SSH your client times out and the REMOTE
powershell keeps running, so "it returned instantly and did nothing" actually
means "it is still hung".  Six stuck processes had piled up before I spotted it.

Fixed to `-gt 1`, patched on the pod (podkit.ps1.bak is the original), and the
script is now IN THE REPO (tools/podkit.ps1) with the trap explained inline --
previously it existed only on the cab, so a restore would silently bring the bug
back.  Pod and repo copies are byte-identical.

THE RUNBOOK (context/pod-hardware.md).  Written because I improvised instead of
reading the one paragraph that already existed, on a live stream.  Now covers:
  * ssh bt411-pod -- and WHY it looked like auth was never set up: the key has
    existed since 2026-08-06, but ssh will not OFFER it without a ~/.ssh/config
    entry, so you get "Permission denied (publickey,password,...)".  Also that
    the Tailscale NODE KeyExpiry is not an SSH credential and Tailscale SSH is
    not enabled on the pod.
  * bare taskkill/setx return "The system cannot find the path specified" over
    this SSH+cmd session -- call System32 tools by ABSOLUTE path.
  * the deploy sequence, identical to a tester's: mkdist -> scp -> Expand-Archive
    -> podkit.  Local config is never clobbered because mkdist packs git-TRACKED
    content only and bindings/environ/glass_layout are gitignored.
  * PODTEST.EGG is not in the repo -- only podkit carries it; without it the
    launcher runs and nothing appears, with no error.
  * schtasks /run /tn BT411Run for GUI work (session 1); /end first, because a
    task already Running refuses /run with 2147946720.
  * the measured panel identities (below).

PANEL IDENTITY, measured over SSH (WMI is session-independent, so no GUI needed):
the cab's two RAR0005 panels SHARE one EDID code and have blank serials --
exactly the collision flagged as unproven in ec080cd -- so monitor:id:RAR0005 is
ambiguous and they must use the per-connector form (monitor:id:UID224795 /
monitor:id:UID200195).  The Dell 1908FP's code is unique.  The cab's shipped
glass_layout.cfg still uses the fragile monitor:DISPLAY4 device-name form.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018SgmXGNMXavXiafKXf9MDC
This commit is contained in:
Joe DiPrima
2026-08-08 17:31:40 -05:00
co-authored by Claude Opus 5
parent c1a87b602c
commit 06d3b93507
2 changed files with 117 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
context/pod-hardware.md
+59
View File
@@ -424,6 +424,65 @@ returns nothing rather than failing. Kill `btl4.exe` BEFORE scp'ing a new exe or
locked. The receipts in `podrun.log` are the remote eyes; a missing `DEBUG_STREAM` line in an
otherwise-logging run is real evidence that code path did not execute.
### The remote runbook — READ THIS BEFORE IMPROVISING (2026-08-08)
Every line below cost real time to rediscover on a live stream. Follow it in order.
**1. Connect.** `ssh bt411-pod` (alias in `~/.ssh/config`) → `bt411-pod.tail840fa4.ts.net`, user
`user`, key `~/.ssh/bt411_pod`.
⚠ **The key has existed since 2026-08-06 but ssh will NOT offer it without the config entry** — it
only tries default names (`id_rsa`/`id_ed25519`). Without it you get
`Permission denied (publickey,password,keyboard-interactive)`, which reads exactly like "auth was
never set up". It was. Also: the Tailscale **node** KeyExpiry (2027-02-02) is what keeps the cab on
the tailnet — it is **not** an SSH credential, and Tailscale SSH is **not** enabled on the pod (the
peer advertises no `sshHostKeys`), so port 22 is the pod's own Windows OpenSSH.
**2. Absolute paths for System32 tools.** Over this SSH+cmd session, bare `taskkill` / `setx`
return `The system cannot find the path specified.` and silently do nothing. Use
`C:\Windows\System32\taskkill.exe`. (`tasklist`, `schtasks`, `dir`, `copy` happen to resolve.)
**3. Deploy — the SAME procedure a tester uses.** No pod-special exe drops.
```
python tools/mkdist.py # -> dist/BT411_4.11.NNN.zip
scp dist/BT411_4.11.NNN.zip bt411-pod:C:/bt411/
ssh bt411-pod "powershell -NoProfile -Command \"Expand-Archive C:\bt411\BT411_4.11.NNN.zip -DestinationPath C:\bt411 -Force\""
ssh bt411-pod "powershell -NoProfile -ExecutionPolicy Bypass -File C:\bt411\podkit.ps1 -Content C:\bt411\BT411_4.11.NNN\content"
```
`runpod.bat` auto-resolves the **newest** `BT411_*` by date, so nothing else needs pointing.
Local config is **never clobbered**: `mkdist.py` packs git-TRACKED content only, and
`bindings.txt`/`environ.ini`/`glass_layout.cfg` are all gitignored. `podkit.ps1` pushes the frozen
rig masters (`podprofile.ini`, `glass_layout.cfg`, `PODTEST.EGG`) from `C:\bt411\` into the install.
⚠ **`PODTEST.EGG` is not in the repo** — only podkit carries it. If podkit fails, the launcher runs
and *nothing appears*, with no error.
**4. Launch.** `schtasks /run /tn BT411Run` → runs `C:\bt411\runpod.bat` as `user`,
LogonType=Interactive, so it lands in **session 1** and is visible on the panels. Log:
`<install>\content\podrun.log`.
**5. ⚠ THE PODKIT INFINITE LOOP — the trap that ate an evening.** `podkit.ps1` line 30 was
```powershell
while ($keep.Count -gt 0 -and $keep[-1].Trim() -eq '') { $keep = $keep[0..($keep.Count-2)] }
```
When `$keep` trims to ONE blank line, `$keep.Count-2` is `-1` and PowerShell's `$keep[0..-1]`
returns **two** elements instead of shrinking — infinite loop, RSS climbing past 60 MB. It fires
whenever everything outside `environ.ini`'s marker block is blank, i.e. **any `environ.ini` that was
already kitted** — exactly what you get carrying it forward from the previous install. Patched to
`-gt 1` on the pod 2026-08-08 (`podkit.ps1.bak` is the original); **this fix is NOT in any repo**,
so a restored/replaced podkit brings the bug back.
*Signature:* a **blank cmd console** on the pod, task stuck `Status: Running`, **no `btl4.exe`, no
`podrun.log`**. *The cascade:* each hung run holds `environ.ini`, so every later attempt blocks too —
and over SSH your client times out while the REMOTE powershell keeps running, so "it returned
instantly and did nothing" actually means "it is still hung". Recover with
`taskkill /F /PID <pid>` (absolute path) on the session-1 `cmd`+`powershell` pair, then
`schtasks /end /tn BT411Run` before re-running — a task already Running refuses `/run` with
`2147946720` (`0x800710E0`, "operator refused the request").
**6. Panel identity (measured 2026-08-08, over SSH — WMI is session-independent so this works
without a GUI):** `DISPLAY\RAR0005\…UID224795`, `DISPLAY\DEL4025\…UID249395` (DELL 1908FP),
`DISPLAY\RAR0005\…UID200195`. **The two RAR panels share one EDID code and have blank serials**, so
`monitor:id:RAR0005` is ambiguous — they must be bound by the per-connector form
(`monitor:id:UID224795` / `monitor:id:UID200195`). The Dell's code is unique. The cab's shipped
`glass_layout.cfg` still uses the fragile `monitor:DISPLAY4` device-name form.
## The 1995 player manual — alignment audit (2026-07-18) [T1, primary source]
`reference/manual/Tesla40_BT_manual.pdf` (34pp, from Nick). CONFIRMS the reconstruction on
every checked control behavior: