vwemirror/BT411
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

console review round 2: six must-fixes from the adversarial pass (one of its fixes corrected)

Browse Source 
The 4-dimension review of 4736cba..820caf8 returned 14 surviving findings.  All
six must-fixes plus the four deferables are in; one of the review's own
prescriptions was wrong and is fixed differently (below).

THE REAPER, FINAL DOCTRINE (blocker + major).  Rev 2 -- "reap anything silent in
the staging window" -- was still wrong twice over: a pad sends exactly ONE
message in its life (the egg ACK, L4NET.CPP:1259) and is then silent FOREVER, so
any manual-launch hold >180s reaped every healthy ACKed pad and force-relaunched
their clients; and a REGISTERED game conn is quiet on TCP while loading, so with
BT_RELAY_TCP_ONLY=1 (or blocked UDP) the reaper _abort_round()ed the whole night
every ~3 minutes blaming a healthy player.  The rule is now: an app-level
deadline is valid only while a RESPONSE IS OWED.  Only egg-sent-never-ACKed pads
are reaped, with the debt clock starting at EGG SEND (a pad that sat through a
long held-egg wait gets its full window -- an edge neither review round caught);
game conns are never reaped at all.  Half-open ghosts are TCP keepalive's job
(~130s, faster than the deadline anyway).

RE-ARM GUARDED ON EVERY CHANNEL (major).  The launch_at guard lived only on the
LAUNCH-press path; the ctl `rearm` command, stdin, and the always-lit GUI button
reached _rearm_for_new_round unconditionally -- one press mid-mission zeroed the
counters, permanently killing the mission clock AND making End Mission print "no
mission is running": an unstoppable round, the exact class this feature exists
to eliminate.  Now refused (loudly) while a mission is running or a pair is in
flight, and the button greys while launched.  THE REVIEW'S OWN FIX WAS WRONG
here: it prescribed refusing on `launches_sent >= 2`, but that stays 2 after a
FINISHED round -- applying it verbatim resurrected the original dead-LAUNCH
wedge, caught immediately by the regression suite.  "Running" is
`launches_sent >= 2 AND not stop_sent`.

RE-ARM RE-KEYS SEATS (major).  It restored the template roster but left
seat_beacons/seat_prefs keyed by the trimmed round's positional ids, so round
2's trim minted a departed player's tag into the egg and trimmed a present
player's out, shifting every callsign/mech a slot.  The re-key block is factored
out of _maybe_reset_round (_rekey_seats_to_roster) and shared.

RESTART SESSION NO LONGER BAKES A WALK-UP'S NAME INTO THE EGG (major).
_stop_session and _start_session now restore the stashed configured
callsign/mech into the cells BEFORE _collect_egg can snapshot them
(_restore_seat_defaults); previously the departed name went into the egg (name=
plus rasterized bitmaps) and was then re-captured as the seat's permanent
"configured default".

LATE REMOTE OPERATOR GETS A ROSTER (major).  The only line the GUI can adopt
tags from was printed once at relay startup and aged out of the 400-line control
history in ~33 minutes of stats chatter -- AUTH now re-issues the live roster
line ahead of the replay, so a remote operator connecting at any point gets
pilot lights and a working LAUNCH button.

DEFERABLES, all four: _drop_game blames the tag stashed AT REGISTRATION (the
live-roster resolve named the wrong tag for a trimmed-round conn dying after a
re-arm restore -- and the tag-trusting GUI would clear the wrong seat); an
operator-BLANKED callsign cell now restores (empty string is a real configured
value; the falsy skip left the departed name up and re-captured it); a returning
player displaces their own half-open registered ghost (same IP, no mission
running) instead of eating ROSTER FULL until keepalive fires; udp_spoofed now
rides the [relay-stats] line when non-zero and the warn set is capped at 256.

VERIFIED: rearm suite grown to 25 checks (ACKed pad never reaped however silent;
never-ACKed pad reaped; game conns never reaped; the egg-send debt clock; re-arm
refused mid-mission, allowed after round end) -- plus net 17/17, roster 22/22,
checkctx CLEAN.  Live rig: mid-mission `rearm` refused with the message and the
mission survived; End Mission -> re-arm -> full re-seat -> second mission
launched.  One bounded artifact observed and documented: each waiting pod
bounces once (identity resync) after an explicit re-arm.

Docs rewritten to the final doctrine (context/operator-console.md reaper +
re-arm + roster-replay + udp_spoofed sections; OPERATOR_GUIDE + tooltip now say
re-arm is between-rounds-only and warn about the one-bounce resync).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
arcattack
2026-07-26 10:11:07 -05:00
co-authored by Claude Opus 5
parent 820caf8765
commit c9e561d9ba
5 changed files with 260 additions and 95 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/btconsole.py
+138 -64
View File
@@ -657,6 +657,7 @@ class Relay:
_send_all_guarded(conn.sock, pkt, "egg chunk")
n += 1
conn.egg_sent = True
conn.egg_sent_at = time.time() # the ACK-debt clock starts NOW
print(f"[relay] console conn {addr[0]}:{addr[1]}: egg sent "
f"({n} chunks); awaiting pod ACK", flush=True)
except OSError as e:
@@ -803,6 +804,30 @@ class Relay:
print(f"[relay] seat-pref egg rewrite FAILED ({e!r}) -- "
f"serving the original egg", flush=True)
def _rekey_seats_to_roster(self):
"""Re-key live beacons/prefs by each TAG's position in the CURRENT
roster. Needed whenever the roster changes shape (round reset, re-arm):
seat ids are positional, so a beacon keyed by a TRIMMED round's host id
points at the wrong seat once the template roster is restored -- a
departed player's tag was being minted into the next round's egg while a
present player's was trimmed out (review 2026-07-26). Beacons whose tag
is no longer in the roster are dropped from the maps (their seats free
up); reservations are cleared wholesale, matching the round-reset
semantics."""
old_beacons = self.seat_beacons
old_prefs = self.seat_prefs
self.seat_beacons = {}
self.seat_prefs = {}
self.seat_reservations = {}
for old_id, conn in old_beacons.items():
tag = getattr(conn, "seat_tag", None)
if tag in self.roster:
new_id = FIRST_GAME_HOST_ID + self.roster.index(tag)
conn.seat_host = new_id
self.seat_beacons[new_id] = conn
if old_id in old_prefs:
self.seat_prefs[new_id] = old_prefs[old_id]
def _maybe_reset_round(self):
# All pods gone from a FINALIZED round -> restore the template so the
# next round's joins are honored (held egg, fresh prefs/trim). Live
@@ -827,78 +852,55 @@ class Relay:
self.stop_sent = False
self.stop_requested = False
self.mission_started_at = None
old_beacons = self.seat_beacons
old_prefs = self.seat_prefs
self.seat_beacons = {}
self.seat_prefs = {}
self.seat_reservations = {}
for old_id, conn in old_beacons.items():
tag = getattr(conn, "seat_tag", None)
if tag in self.roster:
new_id = FIRST_GAME_HOST_ID + self.roster.index(tag)
conn.seat_host = new_id
self.seat_beacons[new_id] = conn
if old_id in old_prefs:
self.seat_prefs[new_id] = old_prefs[old_id]
self._rekey_seats_to_roster()
print(f"[relay] round RESET -- roster/egg restored "
f"({len(self.seat_beacons)} player(s) already waiting)",
flush=True)
def _reap_dead_peers(self):
"""Age out peers that stopped speaking entirely.
"""Age out a peer that OWES a protocol response and has gone silent.
TCP keepalive (set on accept) handles most half-open sockets, but it is
best-effort and platform-dependent, and a peer can also be alive at the
TCP level while its game process is gone. A silent connection is what
used to pin the relay: an acked console conn blocks the round reset
forever, and a registered game conn holds its by_host seat, so BOTH
launch re-arm paths stay unreachable and the operator has to restart.
THE DOCTRINE (rewritten twice in review, 2026-07-26 -- keep it): an
app-level deadline is only valid where silence is ABNORMAL, i.e. while a
response is owed. Everything else is TCP keepalive's job (set on every
accept; ~130s detection via SIO_KEEPALIVE_VALS), which kills the socket
and lands in the normal recv/drop path.
Deliberately generous (DEAD_PEER_SECONDS): a pod legitimately sends
nothing while sitting at the front end between rounds, so this must only
catch the genuinely gone. Never applied while a mission is running --
an in-mission pod that goes quiet is the netcode's problem, not ours.
What a healthy peer's silence looks like, so nobody re-widens this:
* a console pad sends exactly ONE message in its life -- the egg ACK.
After ACKing it is silent FOREVER. The first cut reaped ACKed pads
180s into any staging hold (a manual-launch operator waiting on a
straggler), force-relaunching every healthy client. [blocker]
* a REGISTERED game conn is loading the mission: quiet on TCP until
READY, and its 15s UDP HELLOs never arrive if the pod runs
BT_RELAY_TCP_ONLY=1 or UDP is blocked (the documented TCP fallback).
Reaping it _abort_round()s the whole night, every ~3 minutes,
blaming a healthy player. [major]
* between rounds every pod is byte-silent by design (the seat beacon
is write-only: L4NET.CPP "the relay ignores its silence").
So the ONLY thing this reaps is a console pad that was sent the egg and
never ACKed within DEAD_PEER_SECONDS -- a wedged or dead client holding
the eggs_done gate open. Debt paid (acked) = exempt. No egg = exempt.
Game conns are never reaped here at all.
"""
if self.launches_sent >= 2:
return # mission running: leave peers alone
if not self.eggs_released:
#
# BETWEEN ROUNDS / BEFORE THE EGG GOES OUT, SILENCE IS CORRECT and
# can last a long time (a real night showed 12- and 5-minute gaps
# while the operator set up the next mission). A waiting pod has
# nothing to send: its seat beacon is deliberately write-only for the
# process lifetime -- L4NET.CPP: "the relay ignores its silence" --
# and its console pad has no egg yet, so it cannot ACK. Reaping here
# would drop HEALTHY players and force their client to relaunch,
# which is worse than the ghost this reaper exists to remove.
# (Caught in review 2026-07-26, before it ever ran on a real night.)
#
if self.launches_sent >= 2 or not self.eggs_released:
return
#
# So the reaper only runs in the ACTIVE STAGING WINDOW: the egg is out and
# we are waiting for ACK/READY, which is exactly when a pod that has gone
# quiet is a pod that is gone. A truly half-open socket is caught sooner
# and more cheaply by TCP keepalive (set on accept, and the pod sets it on
# its own beacon too) -- this deadline is only the backstop for platforms
# where the keepalive tunables do not take.
#
now = time.time()
for conn in list(self.console_conns):
if not conn.egg_sent:
continue # nothing was asked of it yet
if now - getattr(conn, "last_seen", now) > DEAD_PEER_SECONDS:
if not conn.egg_sent or conn.acked:
continue # nothing owed / debt already paid
# Measure the debt from EGG SEND, not from the last byte: a pad that
# connected early and sat through a long held-egg wait would
# otherwise be over the deadline the instant the egg went out.
owed_since = max(getattr(conn, "last_seen", now),
getattr(conn, "egg_sent_at", now))
if now - owed_since > DEAD_PEER_SECONDS:
print(f"[relay] console conn {conn.addr[0]}:{conn.addr[1]} "
f"REAPED -- silent for "
f"{now - conn.last_seen:.0f}s while staging "
f"(acked={conn.acked}); it can no longer hold the round "
f"open", flush=True)
f"REAPED -- egg sent {now - owed_since:.0f}s ago and "
f"never ACKed; it can no longer hold the round open",
flush=True)
self._drop_console(conn)
for conn in list(self.game_conns):
if conn.host_id is None:
continue # beacon / not yet HELLO'd: silent by design
if now - getattr(conn, "last_seen", now) > DEAD_PEER_SECONDS:
self._drop_game(conn, "silent for %.0fs while staging -- reaped"
% (now - conn.last_seen))
def _tag_of(self, host_id):
"""The LIVE roster tag for a host id ('?' if out of range)."""
@@ -1057,6 +1059,35 @@ class Relay:
start a new round. Reset only the ROUND state -- seats, beacons, prefs
and connections are left alone, so whoever is here stays here.
"""
#
# GUARDED ON EVERY CHANNEL (review 2026-07-26). The `launch_at is None`
# guard lived only on the LAUNCH-press path, but the ctl `rearm` command,
# the stdin reader and the always-lit GUI button all reached here
# unconditionally. Mid-mission (launches_sent==2) a re-arm zeroes the
# counters, which permanently kills the mission clock AND makes End
# Mission print "no mission is running" -- the round becomes
# unstoppable except by Stop Session, the exact outcome this feature
# exists to eliminate. Mid-pair (launch_at set) it destroys the
# RunMission pair and re-releases eggs to mid-launch pods -- the re-arm
# storm. Refuse both, loudly.
#
# "Running" is launches_sent >= 2 AND no StopMission issued yet. NOT
# launches_sent alone: after a FINISHED round (clock expiry or End
# Mission -> stop_sent True) launches_sent still reads 2 until the
# re-arm itself clears it -- refusing on that would resurrect the very
# wedge this feature exists to fix (caught by the regression suite when
# the review's own proposed guard was applied verbatim).
if self.launches_sent >= 2 and not self.stop_sent:
print("[relay] RE-ARM refused -- a mission is RUNNING; press "
"End Mission first", flush=True)
self.launch_requested = False # do not let the press linger
return
if self.launch_at is not None:
print(f"[relay] RE-ARM refused -- a launch sequence is in flight "
f"(fires in {max(0.0, self.launch_at - time.time()):.0f}s)",
flush=True)
self.launch_requested = False
return
print(f"[relay] RE-ARM for a new round ({why}); "
f"previous round state cleared", flush=True)
self.launches_sent = 0
@@ -1090,6 +1121,12 @@ class Relay:
FIRST_GAME_HOST_ID + len(self.roster)))
self.round_hold_until = 0 # do not inherit an abort's settle window
self._reload_egg_file()
# Re-key beacons/prefs to the RESTORED roster (review 2026-07-26): they
# were keyed by the previous round's TRIMMED host ids, and the launch
# fall-through would reinterpret those against the full roster -- a
# departed player's tag minted into round 2's egg, a present player's
# trimmed out, and every callsign/mech shifted a slot.
self._rekey_seats_to_roster()
for c in self.console_conns:
c.acked = False # last round's ACK must not count
c.egg_sent = False # every pod needs THIS round's egg
@@ -1346,7 +1383,19 @@ class Relay:
continue
host_id = FIRST_GAME_HOST_ID + i
if host_id in self.by_host:
break # actively PLAYING: not ours
# A half-open REGISTERED conn (machine slept, power
# cut) squats here until keepalive reaps it (~2 min),
# and its own player got ROSTER FULL meanwhile. No
# mission running + same source IP = this player's
# own ghost: displace it, exactly like a zombie
# beacon (review 2026-07-26).
ghost = self.by_host[host_id]
if (self.launches_sent < 2
and ghost.addr[0] == req_ip):
self._drop_game(
ghost, "displaced by its returning player")
else:
break # actively PLAYING: not ours
old = self.seat_beacons.get(host_id)
if old is not None: # zombie duplicate: displace it
print(f"[relay] displacing stale beacon on seat "
@@ -1459,9 +1508,14 @@ class Relay:
# trim remaps seat ids the GUI would light (and, since the roster
# display fix, clear) the WRONG row. The relay's roster here is the
# live -- possibly trimmed -- one, so this tag is the true identity.
# Stash the tag AT REGISTRATION: _drop_game resolves against the
# LIVE roster, and a trimmed-round conn dying after a re-arm restored
# the full roster would otherwise be blamed on the wrong tag -- and
# the tag-trusting GUI would clear the wrong seat (review 2026-07-26).
conn.reg_tag = self._tag_of(host_id)
print(f"[relay] {conn.name()} REGISTERED "
f"({len(self.by_host)}/{len(self.roster)}) "
f"tag='{self._tag_of(host_id)}'", flush=True)
f"tag='{conn.reg_tag}'", flush=True)
if len(self.by_host) >= len(self.roster):
self._release_eggs() # fallback; normally already fired
# PEER_UP exchange: newcomer learns everyone; everyone learns newcomer.
@@ -1550,8 +1604,10 @@ class Relay:
def _drop_game(self, conn, why):
# tag='...' for registered conns, same reason as the REGISTERED print:
# the GUI must not resolve a remapped post-trim hostID positionally.
drop_tag = (f" tag='{self._tag_of(conn.host_id)}'"
if conn.host_id is not None else "")
drop_tag = ""
if conn.host_id is not None:
_dt = getattr(conn, "reg_tag", None) or self._tag_of(conn.host_id)
drop_tag = " tag='%s'" % _dt
print(f"[relay] {conn.name()} dropped: {why}{drop_tag}", flush=True)
try:
self.sel.unregister(conn.sock)
@@ -1625,6 +1681,8 @@ class Relay:
if not hasattr(self, "_udp_spoof_warned"):
self._udp_spoof_warned = set()
key = (from_host, endpoint[0])
if len(self._udp_spoof_warned) > 256:
self._udp_spoof_warned.clear() # bounded under a flood
if key not in self._udp_spoof_warned:
self._udp_spoof_warned.add(key)
print(f"[relay] UDP from {endpoint[0]}:{endpoint[1]} CLAIMS "
@@ -1771,6 +1829,16 @@ class Relay:
# under the lock so no line is lost or duplicated between
# the replay and the live stream
with _CONTROL_LOCK:
# The GUI adopts its roster from the startup "roster:" line,
# which is printed ONCE and ages out of the 400-line history
# in ~33 minutes of stats chatter -- an operator AUTHing
# late into the night got a permanently blank roster and a
# dead LAUNCH button (review 2026-07-26). Re-issue the LIVE
# roster in the same adoptable format before the replay.
conn.queue_out(
(f"[relay] roster: {len(self.roster)} pilot(s) -> "
f"hostIDs {sorted(self.expected_ids)}: "
f"{self.roster}\n").encode("utf-8", "replace"))
conn.queue_out(b"[ctl] --- session history replay ---\n")
for past in _CONTROL_HISTORY:
conn.queue_out(past)
@@ -1879,9 +1947,15 @@ class Relay:
return
self.stats_at = time.time() + STATS_PERIOD
s = self.stats
# spoofed rides along only when non-zero: the anti-hijack counter was
# invisible (never printed anywhere), so an actual hijack attempt left
# no operator-visible trace beyond the one-shot warning line.
spoofed = s.get("udp_spoofed", 0)
spoof_part = f", spoofed {spoofed}" if spoofed else ""
print(f"[relay-stats] tcp rx/tx {s['tcp_rx']}/{s['tcp_tx']} | "
f"udp rx/tx {s['udp_rx']}/{s['udp_tx']} "
f"(dropped {s['udp_dropped']}, tcp-fallback {s['udp_tcp_fallback']}) | "
f"(dropped {s['udp_dropped']}, tcp-fallback {s['udp_tcp_fallback']}"
f"{spoof_part}) | "
f"registered {sorted(self.by_host)} udp-known {sorted(self.udp_endpoint)}",
flush=True)
tools/btoperator.py
+41 -5
View File
@@ -511,7 +511,8 @@ class Operator(QMainWindow):
"Clear the finished round's state and re-open the launcher, "
"keeping everyone who is already connected. "
"Use this if LAUNCH looks dead after a round instead of "
"restarting the session.")
"restarting the session. Between rounds only: refused while "
"a mission is running or a launch is firing.")
self.rearm_btn.clicked.connect(self._rearm_round)
self.rearm_btn.setEnabled(False)
self.launch_local_btn = QPushButton("Launch local instances")
@@ -817,12 +818,35 @@ class Operator(QMainWindow):
# ------------------------------------------------------------- session --
def _restore_seat_defaults(self):
"""Write every stashed configured (callsign, mech) back into its row and
clear the stash. MUST run before anything snapshots or saves the table:
Restart Session used to _collect_egg() with a departed walk-up's name
still in the cell, baking it into the next session's egg (name= AND the
rasterized bitmaps) and then re-capturing it as the seat's 'configured
default' -- the reported roster bug returning through the egg file
(review 2026-07-26)."""
for tag, (callsign, mech) in list(self._seat_defaults.items()):
for r in range(self.table.rowCount()):
item = self.table.item(r, 0)
if item is None or item.text().strip() != tag:
continue
cell = self.table.item(r, 1)
if callsign is not None and cell is not None:
cell.setText(callsign)
combo = self.table.cellWidget(r, 2)
if mech is not None and combo is not None:
combo.setCurrentText(mech)
break
self._seat_defaults.clear()
def _start_session(self):
self.end_sent = False
# A session stopped while a seat was OCCUPIED leaves its stash entry
# behind (the seat never emptied, so it was never popped); a new
# session must not restore last session's snapshot.
self._seat_defaults.clear()
# behind (the seat never emptied, so it was never popped). Write those
# configured values BACK into the cells before _collect_egg can bake a
# walk-up's name into the egg, then start clean.
self._restore_seat_defaults()
relay_host = self.f_relayhost.text().strip()
if (self.mode.currentIndex() == 0 and relay_host
and relay_host.lower() not in ("localhost", "127.0.0.1")):
@@ -916,6 +940,7 @@ class Operator(QMainWindow):
self.pod_status.setText("remote link closed")
def _stop_session(self):
self._restore_seat_defaults() # departed walk-up names must not outlive the session
if getattr(self, "remote_link", None):
link = self.remote_link
self.remote_link = None # closed-signal becomes a no-op
@@ -1130,8 +1155,16 @@ class Operator(QMainWindow):
callsign, mech = info
else:
# Empty: restore the configured pilot and forget the stash, so
# the next player to take this seat re-snapshots.
# the next player to take this seat re-snapshots. Restore on
# `is not None`: an operator-BLANKED cell is a real configured
# value, and skipping it as falsy left the departed name up and
# then re-captured it as the default (review 2026-07-26).
callsign, mech = self._seat_defaults.pop(tag, (None, None))
if callsign is not None and cell is not None and cell.text() != callsign:
cell.setText(callsign)
if mech is not None and combo is not None and combo.currentText() != mech:
combo.setCurrentText(mech)
continue
if callsign and cell is not None and cell.text() != callsign:
cell.setText(callsign)
if mech and combo is not None and combo.currentText() != mech:
@@ -1145,6 +1178,9 @@ class Operator(QMainWindow):
(self.monitor.ready or (self.monitor.relay_mode and seated > 0))
and not self.monitor.launched
and have_relay)
# Re-arm is refused by the relay mid-mission/mid-pair; grey it too so the
# button beside END MISSION does not invite the press (review 2026-07-26).
self.rearm_btn.setEnabled(have_relay and not self.monitor.launched)
if self.monitor.stats:
self.stats_label.setText(self.monitor.stats)