Issue #33/#34: relaunch-storm + duplicate-seat fixes (pod-grade session stability)

Night-2 capture: one flapping pod cascaded 3 ROUND ABORTs in 8s; every
client relaunch-churned (~1 gen/s) until the D3D device-creation 20s
deadline fired on every machine ('could not connect direct 3d'), and
claim-less restarts minted duplicate roster seats.

Console (relay):
- Round abort clears stale console-conn ACK flags and HOLDS the next
  egg release for an 8s settle window (_tick_settle re-releases when
  quiet) -- the abort->instant-re-release->drop->abort cycle is broken.
- HELLO from a stale identity gets a REJOIN reply (identity resync via
  the seat-request path, 5s/IP rate limit) instead of a bare drop.
- STATIC SEATS: seats remember (IP + callsign); a claim-less restart
  from the same machine+name RECLAIMS its own seat (displacing its
  zombie beacon) instead of minting a duplicate -- the pod contract:
  same box, same seat, always.
- Operator local instances log with BT_LOG_APPEND (the night-2 crash
  loop left a 2-line file; generations now leave a full trail).

Client:
- Relaunch storm damper: a generation that lived <15s sleeps 5s before
  respawning (storm decays to a gentle cadence; normal multi-minute
  round relaunches untouched; menu clicks explicitly never delayed).

Rig-verified: normal round + mid-MISSION drop = no abort (correct);
mid-LOAD kill = exactly ONE abort + 8s hold + seat reclaimed + no
cascade.  The night-2 'host 7' identity-derivation subtlety remains
under forensics (append logs + first-breath line will capture it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
arcattack
2026-07-23 23:35:25 -05:00
co-authored by Claude Opus 4.8
parent 26439c2e1a
commit d3ede2e635
4 changed files with 163 additions and 4 deletions
+13
View File
@@ -165,6 +165,13 @@ LRESULT CALLBACK WndProc(HWND hWnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nShowCmd) int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nShowCmd)
{ {
// Boot tick for the relaunch storm damper (btl4console.cpp, issue #33):
// generations that die young relaunch with a 5 s pause.
{
extern unsigned long gBTBootTick;
gBTBootTick = GetTickCount();
}
// Heap-corruption hunt (env BT_HEAPCHECK=1; default OFF -- ~100x slower): validate // Heap-corruption hunt (env BT_HEAPCHECK=1; default OFF -- ~100x slower): validate
// the whole debug heap on EVERY alloc/free, so an out-of-bounds write is caught at // the whole debug heap on EVERY alloc/free, so an out-of-bounds write is caught at
// the very next heap op (a stack near the culprit) instead of much later at some // the very next heap op (a stack near the culprit) instead of much later at some
@@ -526,6 +533,12 @@ int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine
sprintf(fe_arguments, "-net %d -platform glass", fe_spec.consolePort); sprintf(fe_arguments, "-net %d -platform glass", fe_spec.consolePort);
break; break;
} }
{
// The user just clicked JOIN/LAUNCH in the menu -- never
// storm-damp a human (btl4console.cpp, issue #33).
extern int gBTRelaunchUserInitiated;
gBTRelaunchUserInitiated = 1;
}
BTFE_RelaunchSelfAndExit(fe_arguments); // never returns BTFE_RelaunchSelfAndExit(fe_arguments); // never returns
} }
#ifdef BT_STEAM #ifdef BT_STEAM
+35
View File
@@ -120,9 +120,44 @@ static void
//########################################################################### //###########################################################################
//
// Storm-damper support: btl4main stamps the boot tick at WinMain entry;
// user-initiated relaunches (the menu's JOIN/LAUNCH click) set the
// user-initiated flag so a fast human is never made to wait.
//
unsigned long gBTBootTick = 0;
int gBTRelaunchUserInitiated = 0;
static unsigned long
BTBootTickOf()
{
return gBTBootTick != 0 ? gBTBootTick : GetTickCount();
}
void void
BTFE_RelaunchSelfAndExit(const char *arguments) BTFE_RelaunchSelfAndExit(const char *arguments)
{ {
//
// RELAUNCH-STORM DAMPER (issue #33, night-2 capture): a client whose
// generations die young (stale identity -> HELLO reject -> drop ->
// relaunch) used to cycle ~1/second -- the overlapping instances fought
// for the graphics adapter until every machine hit the D3D 20 s
// deadline, and each bounce re-triggered the relay's round abort. If
// THIS generation lived under 15 s, sleep 5 s before spawning the next
// one: a storm decays to a gentle 5 s cadence while normal multi-minute
// round relaunches are untouched.
//
{
unsigned long uptime_ms = GetTickCount() - BTBootTickOf();
if (uptime_ms < 15000UL && !gBTRelaunchUserInitiated)
{
MarshalLog("generation lived only %lu ms -- relaunch storm "
"damper: waiting 5 s", uptime_ms);
Sleep(5000);
}
gBTRelaunchUserInitiated = 0;
}
// //
// A MENU relaunch (empty arguments) must not leak the mission // A MENU relaunch (empty arguments) must not leak the mission
// process's marshal handoff into the child -- the child inherits our // process's marshal handoff into the child -- the child inherits our
+105 -3
View File
@@ -234,6 +234,7 @@ VEHICLE_TAGS = {"blkhawk", "loki", "bhk1", "madcat", "thor", "owens",
"sunder", "snd1", "vulture", "vul1", "lok2", "mad2"} "sunder", "snd1", "vulture", "vul1", "lok2", "mad2"}
MATCHLOG_CAP = 8 * 1024 * 1024 # matchlog upload frame cap (client caps at 8MB) MATCHLOG_CAP = 8 * 1024 * 1024 # matchlog upload frame cap (client caps at 8MB)
SEAT_RESERVE_SECONDS = 60.0 SEAT_RESERVE_SECONDS = 60.0
ABORT_SETTLE_SECONDS = 8.0 # issue #33: egg-release hold after a round abort
HELLO_MAGIC = 0x31525442 # 'BTR1' little-endian HELLO_MAGIC = 0x31525442 # 'BTR1' little-endian
FRAME_CAP = 1600 # NETWORKMANAGER_BUFFER_SIZE (NETWORK.h:89) FRAME_CAP = 1600 # NETWORKMANAGER_BUFFER_SIZE (NETWORK.h:89)
FIRST_GAME_HOST_ID = 2 # FirstLegalHostID(1) == the console; pods 2..N+1 FIRST_GAME_HOST_ID = 2 # FirstLegalHostID(1) == the console; pods 2..N+1
@@ -411,6 +412,7 @@ class Relay:
self._disc_read() self._disc_read()
self._tick_launch() self._tick_launch()
self._tick_stop() self._tick_stop()
self._tick_settle()
self._tick_stats() self._tick_stats()
def _listener(self, port): def _listener(self, port):
@@ -494,6 +496,17 @@ class Relay:
def _release_eggs(self, present=None): def _release_eggs(self, present=None):
if self.eggs_released: if self.eggs_released:
return return
# issue #33: after a ROUND ABORT, hold the next release until the
# settle window passes -- rejoiners re-ACK within a second and an
# instant re-release hands the flapping pod a fresh round to abort.
if time.time() < getattr(self, "round_hold_until", 0):
if not getattr(self, "_hold_noted", False):
self._hold_noted = True
print(f"[relay] egg release HELD "
f"({self.round_hold_until - time.time():.0f}s settle "
f"after the round abort)", flush=True)
return
self._hold_noted = False
self._reload_egg_file() # pick up between-rounds edits self._reload_egg_file() # pick up between-rounds edits
self.eggs_released = True self.eggs_released = True
if present is not None and len(present) < len(self.roster): if present is not None and len(present) < len(self.roster):
@@ -923,6 +936,40 @@ class Relay:
self.seat_reclaim.pop(claim_tag, None) self.seat_reclaim.pop(claim_tag, None)
print(f"[relay] seat RECLAIMED by returning player " print(f"[relay] seat RECLAIMED by returning player "
f"(tag '{claim_tag}')", flush=True) f"(tag '{claim_tag}')", flush=True)
# STATIC SEATS (issue #34, night-2 capture): a manually restarted
# client carries NO claim and used to mint a NEW seat while its
# old one sat reserved with the same callsign -- duplicate roster
# entries, and exactly what pods must never do. Fallback: match
# the requester's (IP + callsign) against remembered seat
# identities and RECLAIM that seat -- displacing a zombie beacon
# if one is still squatting on it. (Pods are fixed machines:
# same box + same name = same seat, always.)
if assigned is None and payload:
req_ip = conn.sock.getpeername()[0]
req_callsign = (payload.split(b"\0")[0]
.decode("ascii", "replace").strip()[:15])
if not hasattr(self, "seat_identity"):
self.seat_identity = {}
if req_callsign:
for i, tag in enumerate(self.roster):
if self.seat_identity.get(tag) != (req_ip, req_callsign):
continue
host_id = FIRST_GAME_HOST_ID + i
if host_id in self.by_host:
break # actively PLAYING: not ours
old = self.seat_beacons.get(host_id)
if old is not None: # zombie duplicate: displace it
print(f"[relay] displacing stale beacon on seat "
f"'{tag}' (same player rejoining)",
flush=True)
old.seat_host = None
self.seat_beacons.pop(host_id, None)
assigned = (host_id, tag)
self.seat_reclaim.pop(tag, None)
print(f"[relay] seat RECLAIMED by identity "
f"({req_callsign}@{req_ip} -> '{tag}')",
flush=True)
break
if assigned is None: if assigned is None:
for i, tag in enumerate(self.roster): for i, tag in enumerate(self.roster):
host_id = FIRST_GAME_HOST_ID + i host_id = FIRST_GAME_HOST_ID + i
@@ -962,6 +1009,15 @@ class Relay:
conn.seat_host = host_id # this conn IS the beacon now conn.seat_host = host_id # this conn IS the beacon now
conn.seat_tag = tag # tag survives a round reset conn.seat_tag = tag # tag survives a round reset
self.seat_beacons[host_id] = conn self.seat_beacons[host_id] = conn
# issue #34: remember who sat here (IP + callsign) so the same
# player's claim-less restart reclaims THIS seat, never a new one
if payload:
_cs = (payload.split(b"\0")[0]
.decode("ascii", "replace").strip()[:15])
if _cs:
if not hasattr(self, "seat_identity"):
self.seat_identity = {}
self.seat_identity[tag] = (conn.sock.getpeername()[0], _cs)
print(f"[relay] {conn.name()} seat request -> assigned host " print(f"[relay] {conn.name()} seat request -> assigned host "
f"{host_id} '{tag}' (reserved {SEAT_RESERVE_SECONDS:.0f}s)" f"{host_id} '{tag}' (reserved {SEAT_RESERVE_SECONDS:.0f}s)"
f"{pref_note}", flush=True) f"{pref_note}", flush=True)
@@ -982,8 +1038,25 @@ class Relay:
self._drop_game(conn, f"bad HELLO magic 0x{magic:x}") self._drop_game(conn, f"bad HELLO magic 0x{magic:x}")
return return
if host_id not in self.expected_ids: if host_id not in self.expected_ids:
# issue #33: a plain drop bounced the client straight into a
# relaunch loop (stale identity -> reject -> relaunch -> same
# stale identity, ~1/s). Tell it to REJOIN instead (the
# rejoin path re-requests a seat = identity resync), rate-
# limited per IP so a truly broken client can't spin fast.
ip = conn.sock.getpeername()[0] if conn.sock else "?"
if not hasattr(self, "hello_reject_at"):
self.hello_reject_at = {}
now = time.time()
if now - self.hello_reject_at.get(ip, 0) > 5.0:
self.hello_reject_at[ip] = now
try:
self._send_raw(conn, struct.pack(ENV_FMT_TCP,
ROUTE_REJOIN, 0))
except OSError:
pass
self._drop_game(conn, f"HELLO hostID {host_id} not in roster " self._drop_game(conn, f"HELLO hostID {host_id} not in roster "
f"{sorted(self.expected_ids)}") f"{sorted(self.expected_ids)} -- "
f"REJOIN sent (identity resync)")
return return
if host_id in self.by_host: if host_id in self.by_host:
self._drop_game(conn, f"HELLO hostID {host_id} already registered") self._drop_game(conn, f"HELLO hostID {host_id} already registered")
@@ -1046,17 +1119,32 @@ class Relay:
AFTER egg release but BEFORE launch -- the engine's all-connections AFTER egg release but BEFORE launch -- the engine's all-connections
gate would leave every survivor waiting on it forever. Bounce all gate would leave every survivor waiting on it forever. Bounce all
remaining pods back to their seats (REJOIN) and reset the round; the remaining pods back to their seats (REJOIN) and reset the round; the
reclaim window puts everyone back in the same seats in seconds.""" reclaim window puts everyone back in the same seats in seconds.
STORM HARDENING (issue #33, night-2 capture): the first abort's
rejoiners reconnected + re-ACKed within a second, RE-releasing eggs
while a flapping pod was still bouncing -- its next drop aborted the
NEW round, cascading (3 aborts in 8 s, every client relaunch-churning
into the 'Could not create a Direct3D device' 20 s deadline). Two
dampers: stale console-conn ACK flags are cleared (they counted
toward the next release), and egg release is HELD for a settle
window after any abort so the round re-forms only once the flapping
stops."""
if not self.eggs_released or self.launches_sent >= 2: if not self.eggs_released or self.launches_sent >= 2:
return return
print(f"[relay] ROUND ABORT ({why}) -- sending every pod back to " print(f"[relay] ROUND ABORT ({why}) -- sending every pod back to "
f"the join wait for a fresh round", flush=True) f"the join wait for a fresh round "
f"(egg release held {ABORT_SETTLE_SECONDS:.0f}s)", flush=True)
# reset FIRST so the cascade of resulting drops can't re-trigger # reset FIRST so the cascade of resulting drops can't re-trigger
self.eggs_released = False self.eggs_released = False
self.launch_at = None self.launch_at = None
self.launch_requested = False self.launch_requested = False
self.eggs_done_at = None self.eggs_done_at = None
self._launch_blocked_warned = False self._launch_blocked_warned = False
self.round_hold_until = time.time() + ABORT_SETTLE_SECONDS
for c in self.console_conns:
c.acked = False # an aborted round's ACK must not count
c.egg_sent = False # toward re-releasing the next one
survivors = list(self.by_host.values()) survivors = list(self.by_host.values())
for conn in survivors: for conn in survivors:
try: try:
@@ -1176,6 +1264,20 @@ class Relay:
# ---------------- stats ---------------- # ---------------- stats ----------------
def _tick_settle(self):
# issue #33: the post-abort settle hold blocks egg release while the
# rejoin flapping calms; once it expires, re-run the release check
# (the ACKs that arrived DURING the hold produced no later event to
# trigger it).
hold = getattr(self, "round_hold_until", 0)
if hold and time.time() >= hold and not self.eggs_released:
self.round_hold_until = 0
waiting = [c for c in self.console_conns if not c.egg_sent]
if waiting and len(self.console_conns) >= len(self.roster):
print("[relay] abort settle window over -- releasing eggs to "
f"{len(waiting)} waiting pod(s)", flush=True)
self._release_eggs()
def _tick_stats(self): def _tick_stats(self):
if time.time() < self.stats_at: if time.time() < self.stats_at:
return return
+10 -1
View File
@@ -894,7 +894,16 @@ class Operator(QMainWindow):
host, _, port = tag.rpartition(":") host, _, port = tag.rpartition(":")
net_port = int(port) - 1 net_port = int(port) - 1
args += ["-net", str(net_port)] args += ["-net", str(net_port)]
env.insert("BT_LOG", "operator_%d.log" % (r + 1)) # issue #33 forensics: append across the menu->game relaunch
# chain (each generation used to TRUNCATE the log -- the night-2
# crash loop left a 2-line file). Fresh file per launch press.
log_name = "operator_%d.log" % (r + 1)
try:
os.remove(os.path.join(CONTENT, log_name))
except OSError:
pass
env.insert("BT_LOG", log_name)
env.insert("BT_LOG_APPEND", "1")
proc = QProcess(self) proc = QProcess(self)
proc.setWorkingDirectory(CONTENT) proc.setWorkingDirectory(CONTENT)
proc.setProcessEnvironment(env) proc.setProcessEnvironment(env)