BT411/context/operator-console.md

---
id: operator-console
title: "The Operator Console + Relay (btoperator.py / btconsole.py)"
status: living
source_sections: "tools/btoperator.py; tools/btconsole.py (module docstring is authoritative on the wire); context/multiplayer.md §relay/§operator console; docs/OPERATOR_GUIDE.md"
related_topics: [multiplayer, steam-networking, content-archives, pod-hardware, build-and-run]
key_terms: [EGG, master, replicant, relay, seat, roster]
open_questions:
  - "MESH mode's operator buttons are inert: btconsole.main has no stdin reader, so LAUNCH/END/Re-arm are logged as sent and discarded (relay mode is unaffected). Left as-is deliberately -- mesh self-launches."
  - "_log_launch_readiness can print its 'the mission will STALL -- Stop, reduce the roster' warning on a perfectly healthy launch-with-whoever, because it counts by_host against the remapped roster"
  - "A straggler's late round-N FIN is indistinguishable on the wire from a pod dying mid-load, so it still triggers a ROUND ABORT (recoverable now via Re-arm, but the misclassification stands)"
---

# The Operator Console + Relay

The system-operator station: build a mission, run a session, watch pods arrive, launch rounds.
This is the piece the 1995 archive lost and we rebuilt.

> **Sysop-facing instructions live in `docs/OPERATOR_GUIDE.md`.** This file is the
> engineering description — how it is built and where it breaks.

---

## READ THIS FIRST — there are TWO programs and they are not interchangeable

The single most common mistake (made by me, twice, after a context compaction):

| File | What it is | How it runs |
|---|---|---|
| **`tools/btoperator.py`** | **The operator console.** A **PySide6 GUI** — the window the sysop actually uses to build missions, watch pods and press LAUNCH. | `python tools/btoperator.py` (from the repo root, no args) |
| **`tools/btconsole.py`** | **The headless console/relay.** All the wire logic: egg hand-out, seat assignment, the launch pair, the mission clock, the TCP+UDP frame router. **No UI.** | The GUI spawns it as a **QProcess**; also runnable standalone from the CLI |

**"Launch the console" means `tools/btoperator.py`.** Backgrounding `btconsole.py` is *not* it —
it has no window, and with no stdin the operator commands are dead. The division is deliberate:
all protocol logic stays headless so the CLI, the GUI and the test rigs share one tested
implementation; the GUI is only UI + process management.

The GUI drives the relay two ways: **stdin** (`launch`, `stop`, `rearm`) for a local child, and
the **control port** for a remote relay. Both paths reach the same handlers.

---

## Topology and ports

Pods **dial OUT** to the relay (this is what makes internet play work through NAT — no inbound
ports needed at the player's end). With console port `P` (default **1500**):

| Port | Protocol | Purpose |
|---|---|---|
| `P` /tcp | console protocol, relay-terminated | streams the **egg** to each pod, then the **RunMission** launch pair and **StopMission** |
| `P+1` /tcp | envelope-framed game relay | the frame router between pods (`{int32 route; uint32 length}` + payload) |
| `P+1` /udp | envelope-framed unreliable channel | the 60 Hz update-record fan-out (`{int32 route; int32 fromHost; uint32 seq}` + frame) |
| `P+7` /tcp | operator control channel | `AUTH <secret>` then `launch` / `stop` / `rearm` / `ping` / `get` / `set` (`CONTROL_PORT_OFFSET = 7`) |
| udp/15999 | LAN discovery | answers probes so pods can use `BT_RELAY=auto` |

**Pod-side env contract:** `BT_RELAY=<host>:<P>` (or `auto` for LAN discovery) and
`BT_SELF=<their [pilots] entry>` — or `BT_SELF=auto`, which makes the relay assign the next free
roster seat. `players/join.bat` sets these.

### Route table (`tools/btconsole.py:276-286`)

```
>= 2  unicast to that hostID (relay->client: the SENDER's hostID)
  -1  broadcast to every registered pod except the sender
  -2  HELLO           payload: int32 magic 'BTR1', int32 myHostID
  -3  PEER_UP         relay->client
  -4  PEER_DOWN       relay->client
  -5  UDP HELLO-ACK   (sets the client's udpUp)
  -6  SEAT_REQUEST    client->relay: assign me a free roster seat
  -7  SEAT_ASSIGN     relay->client: int32 hostID + NUL-terminated tag
  -8  SEAT_FULL       relay->client: no free seats
  -9  MATCHLOG        client->relay: upload (filename NUL + bytes); saved to
                      matchlogs/<peer>_<name>, cap 8 MB
 -10  READY           client->relay: mission load complete (the READY light)
 -11  REJOIN          relay->client: abandon this round and rejoin
```

---

## The roster, seats and identity

The **egg's `[pilots]` list is the roster** (`parse_egg_roster`, btconsole.py:372). Entry *i*
becomes **hostID `2 + i`** — `FIRST_GAME_HOST_ID = 2`, because `CONSOLE_HOST_ID = 1` is
`FirstLegalHostID` and belongs to the console itself. A pilot entry's text is its **tag**
(e.g. `10.99.0.1:1502`); the tag's **position in the received egg** is how a pod derives its own
host id, which is why a roster **trim remaps seat ids** and everything re-keys by position.

- **Seat requests** (route -6) let a walk-up pod claim a free seat and carry its callsign + mech.
- **Reservations** last `SEAT_RESERVE_SECONDS = 60`.
- **Reclaim by identity**: a returning player (same IP + callsign) gets their old seat back
  (`seat_identity`, btconsole.py:1258-1276). Watch for this in the log as
  `seat RECLAIMED by identity`.

**Launch with whoever connects:** the operator's LAUNCH can `_trim_roster` down to the seats that
actually have live beacons or claims, so a 8-seat egg runs a 3-player round.

### When a player leaves

Relay side: the beacon's FIN frees the seat (`seat_beacons` entry deleted, and `seat_reservations` /
`seat_prefs` dropped), but the TAG is **held for that player for 90 s** (`seat_reclaim`) so a crash
or a reconnect returns them to the same seat with the same mech. Seat assignment skips seats that
are claimed, reserved, operator-reserved, or reclaim-held — so a brand-new joiner in that 90 s window
gets the next free seat instead, or `ROSTER FULL` if that was the only one. The hold expires on its
own; nothing is held permanently.

**`PLAYER n LEFT` is NOT printed for every departure.** It is inside the beacon-close branch gated
on `if seat not in self.by_host:` — the relay's own comment reads *"never claimed: player left"*. A
player who was fully REGISTERED and then drops produces only `game[...] dropped` (+ PEER_DOWN). Any
UI that keys "seat is empty" off the LEFT line alone will therefore miss the common case — which is
exactly the bug below.

### The roster kept a departed player's name  [FIXED 2026-07-26]

Operator report: *"if a player disconnects, it leaves their name up in the roster ... it should turn
their seat back to empty."* The GUI only ever **wrote** walk-up names into the table: on a
disconnect `seat_info` was popped and the write-back loop hit `if not info: continue`, so the cell
kept the departed callsign for the rest of the session — while the pilot light beside it correctly
said `waiting`. The table and the light disagreed, and a free seat looked occupied.

Two changes, because the two clearing paths are not symmetric:

1. `SessionMonitor` now pops `seat_info` when a pilot goes idle from **either** signal — the
   `game[...] dropped` line as well as `PLAYER n LEFT` — so a registered player's departure clears
   the identity too.
2. `_refresh_pod_status` stashes the operator's configured callsign/mech **per occupancy** when a
   seat fills, and **restores it** when the seat empties. Per-occupancy (rather than once at egg
   load) means an edit the operator makes while a seat is empty is respected, not overwritten by a
   stale snapshot.

It restores the **configured pilot**, not a literal blank, on purpose: that cell is editable and is
what gets written to the egg on Save, so a placeholder like "-- empty --" would end up in the
mission file. The "unoccupied" signal is the grey `waiting` light next to it.
Regression guard: `scratchpad/test_operator_roster.py` (drives the real widget offscreen).

---

## The round lifecycle (the part that used to strand the operator)

```
 session start
   -> egg HELD          until len(console_conns) >= len(roster)      [btconsole.py:565-578]
   -> _release_eggs()   (or _release_eggs(present) -> _trim_roster)  [:627]
   -> pod ACK           per pod (clientID 0, msgID 4)                -> _pods_ready()
   -> READY (route -10) per pod: mission load complete
   -> operator LAUNCH   manual mode: waits for the operator            [_tick_launch]
   -> RunMission #1     then #2 after LAUNCH_STEP_SECONDS = 4s        launches_sent 1 -> 2
   -> mission clock     the egg's [mission] length, +STOP_GRACE = 2s  [_tick_stop]
   -> StopMission       clock expiry or operator End Mission
   -> round end         pods exit, join.bat relaunches them, they reconnect
   -> RE-ARM            so the next LAUNCH works                      <-- see below
```

Timings: `LAUNCH_SETTLE_SECONDS = 20` (egg → first launch; pods must reach WaitingForLaunch or the
handler `Fail()`s), `LAUNCH_STEP_SECONDS = 4`, `STOP_GRACE_SECONDS = 2`,
`ABORT_SETTLE_SECONDS = 8` (egg-release hold after a round abort).

### The all-ready gate

Firing a mission while a pod is still LOADING drowns that pod's load in the running mission's
update streams (same event queue) — the operator, who joins last, once loaded 5+ minutes. So the
RunMission pair is held until every registered pod has sent READY. Escape hatches: the holdouts
are announced every 10 s, and a **180 s cap force-fires** so one wedged pod cannot hold the night
hostage.

### Re-arm: why LAUNCH used to do nothing after a round  [FIXED 2026-07-26]

**The failure the operator reported:** "after a game ends I push LAUNCH and nothing happens until
I reset the session or reboot the console." Their own counter-observation is the tell: *with a
stable group it worked fine repeatedly.*

The manual LAUNCH branch is guarded on `launches_sent == 0`, but a finished mission leaves it at
**2**. The only automatic paths back to 0 were both **all-or-nothing**:

- `_check_launch_gate` — needs **every** seat of the last round to re-ACK
- `_maybe_reset_round` — needs **every** pod to be gone

A games night lives between those: most pods rejoin, one player closes their window. And the
"not all seats filled" diagnostic sat *inside* the `launches_sent == 0` guard, so in exactly that
state **nothing printed at all**. The GUI agreed: its LAUNCH button is gated on
`not monitor.launched`, and `launched` was cleared only by the two relay lines those same gates
emit — so the button was greyed out precisely when wedged. Only a session restart recovered,
because a fresh relay process means fresh state.

**Now:** an explicit operator LAUNCH is itself sufficient authority to start a new round.
`_rearm_for_new_round()` clears the round state and restores the template roster/egg while
**keeping** seats, beacons and connections. Plus a `rearm`/`newround` command and a **Re-arm**
button, so recovery never needs a restart.

**The trap to remember if you touch this:** `launch_requested` deliberately stays set across the
RunMission pair, so re-arming on `launches_sent >= 1` alone also matches the *normal* state
between #1 and #2 — that resets the pair mid-flight and re-releases eggs every tick, a **re-arm
storm** that kicks every pod into an identity-resync loop. The re-arm therefore also requires
`launch_at is None` (nothing in flight). Caught on the rig, 2026-07-26.

### The post-abort dead lobby  [FIXED 2026-07-26]

A distinct wedge with the same symptom, **live-proven** in `content/operator_relay.log` on
2026-07-25 (≈5 minutes of a night lost). `_abort_round` sets `eggs_released = False` **before** the
surviving pods' sockets close, so every later `_maybe_reset_round` hits its own
`if not self.eggs_released: return` and **the template restore never runs**: `roster` and
`expected_ids` stay trimmed to last round's attendance and `egg_path` stays the trimmed egg. All
three automatic release paths compare against `len(self.roster)`, so with N−1 pods back nothing
releases, walk-ups get `ROSTER FULL`, and the relay sits dead.

`_rearm_for_new_round` is **guarded on every channel** (ctl command, stdin, the GUI button -- the guard lived only on the LAUNCH path at first, so a ctl `rearm` mid-mission killed the mission clock and made the round unstoppable): it refuses while a mission is RUNNING (`launches_sent >= 2 and not stop_sent` -- NOT `launches_sent` alone, which stays 2 after a finished round and would resurrect the original wedge) and while a launch pair is in flight. It also re-keys `seat_beacons`/`seat_prefs` to the restored roster (`_rekey_seats_to_roster`, shared with the round reset) -- without that, round 2 trimmed the WRONG seats into the egg. Its template restore is **unconditional** — an earlier cut gated it
on `eggs_released`, which made Re-arm useless in the one state that most needs it. It also clears
`round_hold_until` so an abort's settle window is not inherited.

**Aborts happen for an ordinary reason:** nothing on the wire distinguishes "a straggler's round-N
FIN arriving late" from "a pod died mid-load", so a slow or remote pod closing ~18 s after
StopMission is read as the latter.

### The stop latch

`_tick_stop` returns early while `launches_sent < 2`, so a `stop_requested` set **between** rounds
used to sit latched and then StopMission the next mission in the very tick it launched — which from
the operator's seat is also "launch did nothing". It is now consumed and announced while idle.

---

## Liveness (the "reboot the console" half)  [FIXED 2026-07-26]

Nothing used to notice a pod that vanished **without** closing its TCP: a sleeping laptop, a
dropped Wi-Fi link, a killed process, a NAT idle-timeout. There was no `SO_KEEPALIVE` and no idle
deadline anywhere, so the dead connection sat in `console_conns` with `acked=True` **forever**,
which (a) permanently blocked `_maybe_reset_round` (it requires *no* conn acked) and (b) held a
phantom seat in the launch count. Only restarting the relay cleared it.

Now: keepalive on every accepted socket (`_enable_keepalive`), a `last_seen` stamp on every read
**and on every inbound UDP datagram**, and `_reap_dead_peers()` with `DEAD_PEER_SECONDS = 180`.

**The reaper's final doctrine (third revision — review 2026-07-26 — KEEP IT):** an app-level
deadline is valid only where silence is ABNORMAL, i.e. while a protocol response is OWED. The only
thing reaped is a console pad that was sent the egg and never ACKed within `DEAD_PEER_SECONDS`
(measured from EGG SEND, not from connect — a pad that sat through a long held-egg wait must get its
full window). Everything else is TCP keepalive's job (~130 s detection via `SIO_KEEPALIVE_VALS`).

Why each earlier revision was wrong, so nobody re-widens it:
- Rev 1 armed it whenever no mission ran → it would have reaped healthy pods during 12-minute
  BETWEEN-ROUNDS waits (a waiting pod is byte-silent by design).
- Rev 2 armed it for the whole STAGING window → it reaped every healthy **ACKed** pad 180 s into a
  manual-launch hold, because a pad sends exactly ONE message in its life (the ACK) and is then
  silent forever; and it reaped **registered game conns** that are quiet on TCP while loading
  (fatal with `BT_RELAY_TCP_ONLY=1` or blocked UDP: `_abort_round` bounced everyone, blaming a
  healthy player, every ~3 minutes). Game conns are now never reaped at all.

### Console-protocol reassembly  [FIXED 2026-07-26]

The pod's **egg ACK is the launch gate's only signal**, and it used to be detected by parsing a
single `recv()` at fixed offset 0 with a `len(data) >= 24` test. Two ways that lost an ACK on a real
network (loopback hid both): the 28-byte ACK arriving **split** (16 bytes then 12) was dropped by
the length test and never looked at again, and two **coalesced** messages meant only the first was
read. A missed ACK means that seat never counts, so the round can never be released — a silent,
unrecoverable wedge. `_console_read` now buffers per connection and walks every complete frame
(`16 + messageLength`), and an implausible length **drops the connection with a reason** rather than
mis-reading that pod for the rest of the night (a stream protocol cannot be resynced by guessing).
Bounds: `CONSOLE_MSG_MAX = 65536`, `CONSOLE_INBUF_MAX = 1 MiB`.

### UDP endpoint anti-hijack  [FIXED 2026-07-26]

`from_host` in a UDP envelope is the sender's **own claim**, and the relay used it directly to
refresh that host's downstream endpoint. So any datagram claiming host N silently **stole host N's
traffic** — a zombie pod from a previous round, a stale NAT mapping, or anyone who guessed a host
id — and the victim simply stopped receiving on a channel that still looked healthy. The claim is
now bound to the identity we actually authenticated: the **IP of that host's live TCP game
connection**. The **port is deliberately not checked** (it moves on a NAT rebind, which is the whole
reason the endpoint map refreshes per datagram), and a genuine IP change cannot happen without the
TCP connection breaking and re-registering, so this never rejects a legitimate pod. Rejections are counted and now SURFACED — `spoofed N` rides the `[relay-stats]` line whenever non-zero — and logged once per offending `(host, IP)` pair (the warn set is capped at 256 under a flood).

Limitation worth knowing: two pods on the **same machine** share an IP, so this cannot tell them
apart — same-machine trust is assumed.

Also: every pod-facing send used to flip the socket to blocking with **no timeout** on the relay's
**single-threaded** selector loop, so one wedged peer could freeze the whole relay (no launch tick,
no UDP fan-out, no control channel). All sends now go through `_send_all_guarded`
(`SEND_TIMEOUT_SECONDS = 10`, drops the peer on failure).

---

## Modes

- **Relay (internet)** — `btconsole.py --relay <port> <egg>`; pods dial out. The normal mode.
- **Mesh (LAN legacy)** — `btconsole.py <egg> <host:port> ...`; the console dials OUT to each pod's
  `-net` port. The authentic 1995 cabinet direction.
- **Remote relay** — the GUI drives someone else's relay over the control port instead of spawning
  a child. **Known broken:** the LAUNCH button can never enable in this mode.

**The console must STAY CONNECTED.** A disconnect trips the pod's console-loss path, which also
closes its game listener — an engine bug, not ours.

---

## Logs and evidence

| File | Notes |
|---|---|
| `content/operator_relay.log` | the relay's own stdout, teed by the GUI. **Truncated at each session start** — so a Start Session destroys the evidence of the wedge it is being used to clear. Copy it first when diagnosing. |
| `matchlogs/<peer>_<name>` | per-pod match reports, auto-uploaded on route -9 at each clean round end (relay modes only; a mid-round crash loses it) |
| `content/join.log` | the pod's own log; **never** uploaded, only on the player's machine |

Every relay log line is wall-clock stamped (`_StampedOut`) so post-mortems can measure delays.
The GUI's monitor regexes all use `.search()`, so the prefix is transparent to them.

---

## Mode-specific traps (verified 2026-07-26 by reading `btoperator.py`)

These are not theory — each was read out of the code, and each makes a control *look* live while
doing nothing:

- **MESH mode: the operator buttons are inert.** `_relay_send` reports ">> operator LAUNCH sent"
  whenever the child process is Running, but the **stdin command thread only exists in
  `relay_main`** (btconsole.py) — the mesh path (`btconsole.main`) has no stdin reader at all. So
  LAUNCH / END MISSION / Re-arm are silently discarded in mesh mode. `Apply mission settings` is
  likewise inert there: the mesh console reads the egg bytes once at startup. Mesh also never
  prints "WAITING FOR OPERATOR", so the LAUNCH button cannot even enable.
- **REMOTE-operator mode — FIXED 2026-07-26.** LAUNCH and END MISSION could never enable (both
  conditions required `console_proc is not None`, which the remote path never sets), the roster
  lights were permanently empty (`SessionMonitor(True, [])` → no tags → `seated` always 0), and
  `Launch local instances` was refused by the same guard even though the code below it already
  built `BT_RELAY` from the remote host. Now: all three enable conditions accept **either** channel
  (`console_proc` *or* `remote_link`), and the monitor **adopts the roster** from the relay's
  `roster: N pilot(s) -> hostIDs [...]: [...]` line, which the relay **re-issues from live state to
  every newly AUTHed operator ahead of the history replay** — relying on the startup line alone
  broke after ~33 minutes, when stats chatter aged it out of the 400-line history and a
  late-connecting remote operator got a blank roster and a dead LAUNCH button. So a remote operator
  gets real pilot lights and a real seat count at any time of night.
- **`Start session` silently overwrites the egg on disk** (including re-rasterized callsigns), with
  no prompt and no backup.
- **`Apply` writes only the six `[mission]` keys** (map/time/weather/scenario/temperature/length).
  Roster edits need `Save`; a **seat-count change is refused by the relay** with
  `egg file roster changed N->M seats -- IGNORED (restart the session to resize)`.
- **The Color dropdown offers every colour in the RES vehicletable**, but only
  `Black/Brown/Crimson/Green/Grey/Tan/White` actually paint — anything else silently renders GRAY
  (the relay warns). A "valid" mission can still produce a grey mech.
- **`operator_secret.txt` is opened by a bare relative name** by the relay, whose working directory
  is inherited from whatever directory `btoperator.py` was started in (the GUI never calls
  `setWorkingDirectory` on the relay child). Start from the repo root and it will *not* be the
  `content\operator_secret.txt` the tooltips point at.
- **After the relay dies on its own**, `_console_finished` leaves `console_proc` non-None, so
  `Launch local instances` stays enabled and will spawn clients against a dead relay.

## Gotchas / things that look like bugs

- **`egg HELD (n/N pods present)`** is normal staging, not an error — the egg waits so every pod's
  copy carries every walk-up seat pref.
- **`*** WARNING: ... EMPTY seat(s) ... the mission will STALL`** can fire *misleadingly* after a
  trim, because `_log_launch_readiness` counts `by_host` against the (remapped) roster. Do not
  act on it blindly.
- **A relay-side crash leaves the GUI looking alive** — `_console_finished` does not clear
  `console_proc`. Commands now report honestly instead of logging ">> sent" into the void.
- **Session restart is the big hammer**: it re-spawns the relay, so it clears *every* latch. That
  is why it always "fixed" things — and why it hid the real defects for so long.
- A **`--manual-launch`** relay waits for the operator; without it the launch pair fires
  automatically once all pods ACK.

## Key Relationships

- Uses: [[content-archives]] (the EGG is the mission + roster) · [[multiplayer]] (what the relayed
  frames mean: master/replicant, dead reckoning)
- Feeds: [[multiplayer]] · [[steam-networking]] (the alternate wire seam) · [[pod-hardware]]
- Sysop instructions: `docs/OPERATOR_GUIDE.md`