parked relay: a supervisor that keeps it alive + a REMOTE restart the operator can actually reach
Completes the travelling-operator deployment. The relay stays parked on the
home machine (so no player ever edits join.bat) and the console dials in from
anywhere -- but until now two things needed hands on that machine: bringing the
relay back when it died, and restarting it (the only way to clear a wedge or
pick up a roster resize). Both are covered now.
tools/btrelay_park.py -- the supervisor:
* runs the relay with cwd=content\ , which is what pins WHICH
operator_secret.txt is live (the trap documented last commit);
* relaunches on ANY exit, with 2/5/15/30/60s backoff when a relay dies inside
20s, so a permanent fault (port taken, missing egg) cannot become a spin;
* rotates content\parked_relay.log to .1 first, so the dead generation's
evidence survives the relaunch that replaces it;
* Ctrl-C stops it for good. NOT a Windows service on purpose: session 0
would hide the window an operator wants to tail.
tools/park_relay.cmd -- double-click to park; a shortcut to it in shell:startup
gives start-after-reboot with no admin rights.
`restart` on the control port (btconsole.py): sets restart_requested, the run
loop returns, relay_main exits RESTART_EXIT_CODE=42 and the supervisor relaunches
-- re-reading the egg. REFUSED while a mission is running (launches_sent >= 2
and not stop_sent): bouncing then would drop every pod out of a live round.
Local stdin gets the same command for parity.
btoperator.py: Restart Session in REMOTE mode now sends `restart` and reconnects
6s later instead of just tearing down our own link -- the old behaviour looked
like "Restart does nothing" against a parked relay. (QTimer had to be imported;
it was missing, which py_compile does not catch -- it would have been a runtime
NameError on the first click.)
VERIFIED by scratchpad/test_parked_supervisor.py, 12/12, and the full
test_remote_console_e2e.py re-run green afterwards (15/15):
* kill the relay -> a NEW pid is serving again, and the old log is kept as .1;
* remote `restart` -> ACKed, new pid, egg re-read, serving again;
* the supervisor distinguishes the two -- "operator requested a restart" vs
the crash/backoff path -- proven from its own log, not assumed;
* the mid-mission guard is present and wired to launches_sent/stop_sent.
Two harness defects fixed while proving it, both mine: a check written with
`or True` that could never fail (replaced with two real assertions against the
supervisor's log), and a recv that treated the relay's correct
close-after-restart as a ConnectionResetError failure.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
arcattack
co-authored by
Claude Opus 5
Claude Opus 5
parent
05f1ffb194
commit
05fdd319d6
@@ -316,6 +316,8 @@ CONSOLE_MSG_MAX = 65536
|
||||
CONSOLE_INBUF_MAX = 1 << 20
|
||||
|
||||
CONTROL_PORT_OFFSET = 7 # control port = console port + 7
|
||||
RESTART_EXIT_CODE = 42 # relay_main's exit code for `restart`
|
||||
# (btrelay_park.py relaunches on it)
|
||||
CONTROL_AUTH_TIMEOUT = 10.0 # unauthenticated sockets die after this
|
||||
CONTROL_OUTBUF_MAX = 262144 # slow-reader cutoff (bytes)
|
||||
CONTROL_SET_KEYS = {"map", "time", "weather", "scenario", "temperature",
|
||||
@@ -547,6 +549,7 @@ class Relay:
|
||||
"udp_dropped": 0, "udp_tcp_fallback": 0}
|
||||
self.stats_at = time.time() + STATS_PERIOD
|
||||
self.ctl_conns = [] # RelayControlConn list
|
||||
self.restart_requested = False # remote `restart` (parked relay)
|
||||
self.ctl_secret = control_secret()
|
||||
|
||||
# ---------------- lifecycle ----------------
|
||||
@@ -617,6 +620,15 @@ class Relay:
|
||||
self._tick_stop()
|
||||
self._tick_settle()
|
||||
self._tick_stats()
|
||||
if self.restart_requested:
|
||||
# Remote `restart` (parked-relay deployment): leave the loop so
|
||||
# relay_main can exit with RESTART_EXIT_CODE and the supervisor
|
||||
# relaunches us with a fresh egg read. This is the ONLY way a
|
||||
# travelling operator can resize the roster or recover a wedged
|
||||
# relay -- the control channel cannot do either in-process.
|
||||
print("[relay] RESTART requested by the operator -- closing "
|
||||
"for the supervisor to relaunch", flush=True)
|
||||
return
|
||||
|
||||
def _listener(self, port):
|
||||
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
@@ -1866,6 +1878,21 @@ class Relay:
|
||||
# out of a latched round (operator report 2026-07-25).
|
||||
print(f"[ctl] RE-ARM from {conn.name()}", flush=True)
|
||||
self._rearm_for_new_round("operator re-arm command")
|
||||
elif cmd == "restart":
|
||||
# Parked-relay deployment: the operator is not at the machine, so
|
||||
# this is their only route to a fresh relay (roster resize, or a
|
||||
# wedge nothing else clears). Refused mid-mission -- it would
|
||||
# drop every pod out of a running round.
|
||||
if self.launches_sent >= 2 and not self.stop_sent:
|
||||
conn.queue_out(b"[ctl] restart REFUSED -- a mission is "
|
||||
b"running; press End Mission first\n")
|
||||
print("[ctl] restart refused (mission running)", flush=True)
|
||||
else:
|
||||
conn.queue_out(b"[ctl] restarting -- reconnect in a few "
|
||||
b"seconds\n")
|
||||
conn.flush_out()
|
||||
print(f"[ctl] RESTART from {conn.name()}", flush=True)
|
||||
self.restart_requested = True
|
||||
elif cmd == "ping":
|
||||
conn.queue_out(b"[ctl] pong\n")
|
||||
elif cmd == "get":
|
||||
@@ -2016,6 +2043,9 @@ def relay_main(argv):
|
||||
elif command in ("rearm", "newround"):
|
||||
print("[relay] operator RE-ARM command received", flush=True)
|
||||
relay._rearm_for_new_round("operator re-arm command")
|
||||
elif command == "restart":
|
||||
print("[relay] operator RESTART command received", flush=True)
|
||||
relay.restart_requested = True
|
||||
elif command:
|
||||
print(f"[relay] unknown operator command: {command!r}",
|
||||
flush=True)
|
||||
@@ -2024,6 +2054,9 @@ def relay_main(argv):
|
||||
relay.run()
|
||||
except KeyboardInterrupt:
|
||||
print("[relay] shutting down")
|
||||
return 0
|
||||
if relay.restart_requested:
|
||||
return RESTART_EXIT_CODE # the supervisor's cue to relaunch
|
||||
return 0
|
||||
|
||||
|
||||
|
||||
+17
-1
@@ -24,7 +24,8 @@ import socket
|
||||
import sys
|
||||
import threading
|
||||
|
||||
from PySide6.QtCore import Qt, QProcess, QProcessEnvironment, QThread, Signal
|
||||
from PySide6.QtCore import (Qt, QProcess, QProcessEnvironment, QThread, QTimer,
|
||||
Signal)
|
||||
from PySide6.QtGui import QColor, QFont, QPalette, QAction
|
||||
from PySide6.QtWidgets import (
|
||||
QApplication, QCheckBox, QComboBox, QFileDialog, QFormLayout, QGroupBox,
|
||||
@@ -966,6 +967,21 @@ class Operator(QMainWindow):
|
||||
def _restart_session(self):
|
||||
"""Next round: same mission/roster, fresh session. Players just
|
||||
re-run their join script."""
|
||||
if getattr(self, "remote_link", None):
|
||||
# PARKED RELAY: we do not own that process, so tearing down our
|
||||
# own link would only disconnect us (the old behaviour, which
|
||||
# looked like "Restart did nothing"). Ask the relay itself to
|
||||
# restart -- its supervisor relaunches it, re-reading the egg, so
|
||||
# a roster resize or a wedge is recoverable from the road. Then
|
||||
# reconnect once it is back up.
|
||||
host = self.f_relayhost.text().strip()
|
||||
self.log.appendPlainText(
|
||||
"== asking the parked relay at %s to RESTART "
|
||||
"(reconnecting in a few seconds) ==" % host)
|
||||
self._relay_send("restart")
|
||||
self._stop_session()
|
||||
QTimer.singleShot(6000, self._start_session)
|
||||
return
|
||||
self.log.appendPlainText("== restarting session ==")
|
||||
self._stop_session()
|
||||
self._start_session()
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
#!/usr/bin/env python3
|
||||
"""btrelay_park.py -- keep the relay PARKED and alive on the host machine.
|
||||
|
||||
The travelling-operator deployment: the relay lives permanently on one machine
|
||||
so every player's join.bat address never changes, and the operator console
|
||||
dials into its control port from anywhere. For that to be true the relay has
|
||||
to outlive crashes, reboots and roster changes without anyone at the keyboard --
|
||||
which is what this supervisor is for.
|
||||
|
||||
python tools/btrelay_park.py [--egg OPERATOR.EGG] [--port 1500]
|
||||
[--manual-launch] [--once]
|
||||
|
||||
What it does:
|
||||
* starts the relay with cwd = content\\ (the working directory decides which
|
||||
operator_secret.txt is live -- see context/operator-console.md);
|
||||
* relaunches it when it exits, whether that was a crash or the operator's
|
||||
remote `restart` (RESTART_EXIT_CODE), with a short backoff so a relay that
|
||||
dies instantly cannot spin the CPU;
|
||||
* writes everything to content\\parked_relay.log, rotating the previous run
|
||||
to .1 so a crash's evidence survives the relaunch;
|
||||
* stops cleanly on Ctrl-C, taking the relay with it.
|
||||
|
||||
Deliberately NOT a Windows service: a service runs in session 0 and the relay
|
||||
is something the operator wants to see, tail and Ctrl-C. For start-at-boot,
|
||||
put a shortcut to tools\\park_relay.cmd in the Startup folder (shell:startup) --
|
||||
no elevation, trivially removable.
|
||||
"""
|
||||
import argparse
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
CONTENT = os.path.join(REPO, "content")
|
||||
RESTART_EXIT_CODE = 42 # must match tools/btconsole.py
|
||||
|
||||
# A relay that dies faster than this was not really running: back off so a
|
||||
# permanent fault (port in use, missing egg) cannot become a spin loop.
|
||||
HEALTHY_SECONDS = 20
|
||||
BACKOFF = [2, 5, 15, 30, 60]
|
||||
|
||||
|
||||
def rotate(path):
|
||||
try:
|
||||
if os.path.exists(path):
|
||||
prev = path + ".1"
|
||||
if os.path.exists(prev):
|
||||
os.remove(prev)
|
||||
os.replace(path, prev)
|
||||
except OSError:
|
||||
pass # a locked log must never stop the relay
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--egg", default="OPERATOR.EGG")
|
||||
ap.add_argument("--port", type=int, default=1500)
|
||||
ap.add_argument("--manual-launch", action="store_true", default=True,
|
||||
help="hold the launch for the operator (the default)")
|
||||
ap.add_argument("--auto-launch", dest="manual_launch",
|
||||
action="store_false",
|
||||
help="let the relay launch by itself once all pods stage")
|
||||
ap.add_argument("--once", action="store_true",
|
||||
help="do not relaunch -- run the relay a single time")
|
||||
args = ap.parse_args()
|
||||
|
||||
egg = args.egg if os.path.isabs(args.egg) \
|
||||
else os.path.join(CONTENT, args.egg)
|
||||
if not os.path.exists(egg):
|
||||
sys.exit("no such egg: %s" % egg)
|
||||
|
||||
secret = os.path.join(CONTENT, "operator_secret.txt")
|
||||
log_path = os.path.join(CONTENT, "parked_relay.log")
|
||||
|
||||
print("=" * 68)
|
||||
print(" BattleTech relay -- PARKED")
|
||||
print(" egg : %s" % egg)
|
||||
print(" ports : console %d | game %d | operator control %d"
|
||||
% (args.port, args.port + 1, args.port + 7))
|
||||
print(" secret : %s%s" % (secret,
|
||||
"" if os.path.exists(secret)
|
||||
else " (will be generated)"))
|
||||
print(" log : %s (previous run kept as .1)" % log_path)
|
||||
print(" launch : %s" % ("operator presses LAUNCH"
|
||||
if args.manual_launch else "automatic"))
|
||||
print(" Ctrl-C here stops the relay for good; everything else")
|
||||
print(" (crash, or the operator's remote `restart`) relaunches it.")
|
||||
print("=" * 68, flush=True)
|
||||
|
||||
cmd = [sys.executable, "-u", os.path.join(REPO, "tools", "btconsole.py"),
|
||||
"--relay", str(args.port), egg]
|
||||
if args.manual_launch:
|
||||
cmd.append("--manual-launch")
|
||||
|
||||
generation = 0
|
||||
strikes = 0
|
||||
while True:
|
||||
generation += 1
|
||||
rotate(log_path)
|
||||
started = time.time()
|
||||
print("[park] generation %d starting %s"
|
||||
% (generation, time.strftime("%H:%M:%S")), flush=True)
|
||||
try:
|
||||
with open(log_path, "w", encoding="utf-8") as log:
|
||||
proc = subprocess.Popen(cmd, cwd=CONTENT, stdout=log,
|
||||
stderr=subprocess.STDOUT)
|
||||
code = proc.wait()
|
||||
except KeyboardInterrupt:
|
||||
print("\n[park] Ctrl-C -- stopping the parked relay", flush=True)
|
||||
try:
|
||||
proc.terminate()
|
||||
except Exception:
|
||||
pass
|
||||
return 0
|
||||
except OSError as e:
|
||||
print("[park] could not start the relay: %r" % e, flush=True)
|
||||
return 1
|
||||
|
||||
lived = time.time() - started
|
||||
if args.once:
|
||||
print("[park] --once: relay exited %d after %.0fs" % (code, lived),
|
||||
flush=True)
|
||||
return code
|
||||
|
||||
if code == RESTART_EXIT_CODE:
|
||||
strikes = 0 # an ASKED-FOR restart is healthy
|
||||
print("[park] operator requested a restart -- relaunching "
|
||||
"(the egg is re-read, so roster changes take effect)",
|
||||
flush=True)
|
||||
time.sleep(1)
|
||||
continue
|
||||
|
||||
# Anything else is a death: crash, killed, or a startup failure.
|
||||
if lived >= HEALTHY_SECONDS:
|
||||
strikes = 0
|
||||
wait = BACKOFF[min(strikes, len(BACKOFF) - 1)]
|
||||
strikes += 1
|
||||
print("[park] relay exited %d after %.0fs -- relaunching in %ds "
|
||||
"(evidence in %s.1)" % (code, lived, wait, log_path), flush=True)
|
||||
try:
|
||||
time.sleep(wait)
|
||||
except KeyboardInterrupt:
|
||||
print("\n[park] Ctrl-C -- staying down", flush=True)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
sys.exit(main())
|
||||
except KeyboardInterrupt:
|
||||
sys.exit(0)
|
||||
@@ -0,0 +1,17 @@
|
||||
@echo off
|
||||
REM ===================================================================
|
||||
REM Park the BattleTech relay on this machine and keep it alive.
|
||||
REM
|
||||
REM Double-click to run it now, or put a SHORTCUT to this file in the
|
||||
REM Startup folder (Win+R -> shell:startup) so the relay comes back by
|
||||
REM itself after a reboot. No admin rights needed either way.
|
||||
REM
|
||||
REM The window stays open: it is the supervisor. Closing it (or Ctrl-C)
|
||||
REM stops the relay for good.
|
||||
REM ===================================================================
|
||||
title BattleTech relay (parked)
|
||||
cd /d "%~dp0.."
|
||||
python "tools\btrelay_park.py" %*
|
||||
echo.
|
||||
echo The parked relay has stopped.
|
||||
pause
|
||||
Reference in New Issue
Block a user