diff --git a/context/multiplayer.md b/context/multiplayer.md index b6bb068..c59a501 100644 --- a/context/multiplayer.md +++ b/context/multiplayer.md @@ -468,6 +468,38 @@ phase). Plan: `~/.claude/plans/partitioned-snuggling-piglet.md`. per-event cost) is an OPEN perf item -- deliberately not attempted pre-8-player-night. Diags: `[loadphase]` markers always on; the full queue dump gates on `BT_LOAD_DIAG=1`; `GeneralEventQueue::DumpBlockers` + the named fallback in `Event::DumpData`. +- **CONNECT/DISCONNECT AUDIT (2026-07-23) [T2, every cell verified]**: systematic sweep of + actor-death x lifecycle-phase, requested by the operator after the WaitingForEgg zombie. + VERIFIED-HANDLED (previously): relay-not-up patient retry; seated-quit seat-free+reclaim; + pre-launch-quit trim; mid-mission peer death (frozen mech, authentic; matchcheck flags); + normal end lobby loop; upload-fail file-keep; scanner/HELLO validation; mass-drop crash fix. + FIXED THIS AUDIT (all live-verified via scratchpad/audit_verify.sh + focused runs): + 1. **Pre-egg console-pad loss** (the REAL zombie phase): before the egg, the pod's ONLY link + is the console pad -- the game socket doesn't exist yet, so the first fix's RelayGameDown + hook could never fire (caught by the verification itself). Detection now lives in + `HostDisconnectedMessageHandler` case ConsoleHostType (relayMode && !scene-presented -> + `BTRelayRejoinNow`); mesh keeps 1995 console-less behavior. Verified 2x in one run. + 2. **Post-egg pre-mission relay loss**: RelayGameDown -> rejoin (the original fix). ✓ + 3. **Mid-mission relay loss = forever-mission**: the STOP is relay-sent (1995: console owns + the clock), so a pod losing its relay mid-match played alone forever. Now RelayGameDown + (scene presented) posts `StopMissionMessage` at +15s -> normal end path -> lobby relaunch + -> patient wait. Verified: scene LIVE -> relay killed -> 15s -> clean RunMissions exit. + 4. **Post-release pod death stalls the round**: survivors wait forever on the dead peer's + connection gate. Relay `_abort_round` (registered-conn drop while eggs_released && + launches<2): reset-first (re-trigger-proof), then route **-11 REJOIN** to survivors -> + `BTRelayRejoinNow` -> everyone re-seats in seconds. Verified: kill-during-load -> + ROUND ABORT -> survivor rejoined -> seat RECLAIMED immediately. + 5. **`BTRelayRejoinNow` carries `BT_SEAT_CLAIM`** (from the mirrored tag): without it the + rejoiner's OWN reclaim-hold blocked generic assignment -> ROSTER FULL retry-loop for the + whole 90s window (found by verification round 2). + 6. **Beacon NAT keepalive**: SIO_KEEPALIVE_VALS 60s/10s on the seat beacon -- long + between-rounds waits can't be silently unseated by NAT idle timeout. [T3 -- not + NAT-tested] + VERIFIED NON-ISSUES: relay-mode pods never bind the -net console listener (early dial-out + return @L4NET~900) -- no double-launch port collision on the internet path; mesh bind-fail + exits cleanly via PostQuitMessage. TOOLING: `BT_LOG_APPEND=1` (btl4main) preserves the log + across lobby-loop generations -- REQUIRED for multi-generation forensics (truncation erased + the evidence in verification round 1). - **LOBBY LOOP + SEAT RECLAIM (2026-07-23) [T2]**: the between-rounds arcade flow. Field problem (first playtest night): at mission end every pod EXITED -> seats dropped -> every round required everyone to re-run join.bat, and the operator could not adjust the mission. diff --git a/engine/MUNGA_L4/L4NET.CPP b/engine/MUNGA_L4/L4NET.CPP index 508abe4..421a5ea 100644 --- a/engine/MUNGA_L4/L4NET.CPP +++ b/engine/MUNGA_L4/L4NET.CPP @@ -17,6 +17,7 @@ #pragma hdrstop #include "l4app.h" +#include "..\munga\APPMSG.h" // StopMissionMessage (mid-mission relay-loss fallback) #include "l4host.h" #include "l4net.h" #include "..\munga\mission.h" @@ -161,8 +162,12 @@ enum RELAY_ROUTE_SEAT_FULL = -8, // relay->pod: roster full RELAY_ROUTE_MATCHLOG = -9, // pod->relay: matchlog upload // (payload: filename NUL + file bytes) - RELAY_ROUTE_READY = -10 // pod->relay: mission load complete + RELAY_ROUTE_READY = -10, // pod->relay: mission load complete // (operator's READY light; empty payload) + RELAY_ROUTE_REJOIN = -11 // relay->pod: abandon this round and + // relaunch into the join wait (a peer + // died after egg release; the stalled + // round cannot complete) }; #define RELAY_HELLO_MAGIC 0x31525442 // 'BTR1' little-endian // LAN auto-discovery (BT_RELAY=auto): probe broadcast on this UDP port; the @@ -627,6 +632,39 @@ const char *BTRelaySelfTag(void) return s_relaySelfTag; } +// +// BTRelayRejoinNow -- relaunch this pod into the patient join wait (the +// lobby-loop mechanism): used pre-mission when the relay dies OR when the +// relay orders a round abort (REJOIN). Strips the exe token off the full +// command line; never returns. +// +void BTRelayRejoinNow(void) +{ + // carry the seat identity so the relaunched generation reclaims its + // EXACT seat immediately (without it, its own reclaim-hold blocks + // generic assignment for the whole 90s window -- verified live) + if (s_relaySelfTag[0] != 0) + SetEnvironmentVariableA("BT_SEAT_CLAIM", s_relaySelfTag); + const char *full_command = GetCommandLineA(); + const char *arguments = full_command; + if (arguments != NULL) + { + if (*arguments == '"') + { + ++arguments; + while (*arguments != 0 && *arguments != '"') ++arguments; + if (*arguments == '"') ++arguments; + } + else + { + while (*arguments != 0 && *arguments != ' ') ++arguments; + } + while (*arguments == ' ') ++arguments; + } + extern void BTFE_RelaunchSelfAndExit(const char *); + BTFE_RelaunchSelfAndExit(arguments != NULL ? arguments : ""); +} + void BTRelayNotifyReady(void) { static int s_ready_sent = 0; @@ -1647,6 +1685,24 @@ void L4NetworkManager::HostDisconnectedMessageHandler(HostDisconnectedMessage* H break; case ConsoleHostType: { + // + // PRE-MISSION console loss in RELAY mode (connection audit + // 2026-07-23): before the egg arrives the pod's ONLY link is + // this console pad -- the game socket does not exist yet, so + // RelayGameDown can never catch an operator Stop Session here + // (the original WaitingForEgg zombie). Rejoin the patient + // walk-up; mesh mode keeps the 1995 console-less behavior. + // + { + extern Logical gBTSceneHasPresented; + if (relayMode && !gBTSceneHasPresented) + { + DEBUG_STREAM << "[relay] console pad lost pre-mission -- " + "relaunching into the join wait" << std::endl << std::flush; + extern void BTRelayRejoinNow(void); + BTRelayRejoinNow(); // never returns + } + } // // Knock down the number of consoles online // @@ -1881,6 +1937,18 @@ Logical L4NetworkManager::RelayRequestSeat() } if (got_seat) { + // NAT KEEPALIVE (connection audit 2026-07-23): the beacon can sit + // idle for many minutes between rounds; a NAT/firewall idling the + // TCP out would silently free the seat. 60s probes keep the + // mapping alive (struct defined inline -- mstcpip.h equivalent). + { + struct { u_long onoff, keepalivetime, keepaliveinterval; } + keepalive = { 1, 60000, 10000 }; + DWORD bytes_returned = 0; + WSAIoctl(seat_socket, 0x98000004 /*SIO_KEEPALIVE_VALS*/, + &keepalive, sizeof(keepalive), NULL, 0, + &bytes_returned, NULL, NULL); + } // PRESENCE BEACON (2026-07-22, "launch with whoever connects"): // KEEP the seat socket open for the process lifetime. The relay // reads a live beacon as "this player is seated": the operator @@ -2364,6 +2432,46 @@ void L4NetworkManager::RelayGameDown(const char *why) relayPadTail = 0; udpUp = False; + // + // PRE-MISSION relay death = the operator stopped/restarted the session + // while this pod was still waiting for its egg (field 2026-07-23: END + // MISSION -> lobby-loop relaunch -> STOP SESSION killed the relay -> + // the pod idled at WaitingForEgg on dead sockets FOREVER, and its stale + // -net listener then blocked any replacement -- the "no longer + // registering" zombie). The arcade-correct move: go back to the patient + // walk-up -- relaunch self into the join wait (the lobby-loop mechanism); + // the fresh process re-dials until the operator's next session appears. + // Mid-match (scene presented) keeps the existing peer-less behavior; the + // post-mission path has its own relaunch. + // + { + extern Logical gBTSceneHasPresented; + if (!gBTSceneHasPresented) + { + DEBUG_STREAM << "[relay] pre-mission relay loss -- relaunching " + "into the join wait" << std::endl << std::flush; + extern void BTRelayRejoinNow(void); + BTRelayRejoinNow(); // never returns + } + else + { + // MID-MISSION relay loss (connection audit 2026-07-23): the + // mission STOP is relay-sent (the 1995 console owned the clock), + // so a pod that loses its relay mid-match would otherwise play a + // peer-less FOREVER-mission. End it gracefully in 15s -- the + // normal stop path runs (fade, matchlog kept for manual send, + // lobby relaunch back to the patient join wait), so the player + // flows into the operator's next session automatically. + DEBUG_STREAM << "[relay] mid-mission relay loss -- ending the " + "mission in 15s (no console to end it)" << std::endl << std::flush; + Application::StopMissionMessage stop_message(0); + Time when = Now(); + when += 15.0f; + application->Post(HighEventPriority, application, + &stop_message, when); + } + } + HostManager::RemoteHostIterator remote_hosts(application->GetHostManager()); Host *host; while ((host = remote_hosts.ReadAndNext()) != NULL) @@ -2457,6 +2565,24 @@ Logical L4NetworkManager::CheckRelay(NetworkPacket *network_packet) client->ReceiveNetworkPacket(NULL, &peer_down); } } + else if (route == RELAY_ROUTE_REJOIN) + { + // ROUND ABORT (connection audit 2026-07-23): a peer died + // after egg release -- the engine's connection gate would + // wait on it forever, so the relay bounces everyone back to + // their seats for a fresh (re-trimmed) round. Mid-mission + // this cannot arrive (the relay only aborts pre-launch); + // guard anyway. + extern Logical gBTSceneHasPresented; + if (!gBTSceneHasPresented) + { + DEBUG_STREAM << "[relay] round ABORT from the operator " + "(a peer dropped mid-load) -- rejoining" + << std::endl << std::flush; + extern void BTRelayRejoinNow(void); + BTRelayRejoinNow(); // never returns + } + } else if (route >= FirstLegalHostID) { is_game_frame = True; diff --git a/game/btl4main.cpp b/game/btl4main.cpp index 3d25d1b..a5a6844 100644 --- a/game/btl4main.cpp +++ b/game/btl4main.cpp @@ -189,7 +189,12 @@ int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine // file (default btl4.log) -- REQUIRED for multi-instance runs from one cwd // (instance 2 would truncate instance 1's log). std::ofstream logfile; - logfile.open(getenv("BT_LOG") ? getenv("BT_LOG") : "btl4.log"); + // BT_LOG_APPEND=1: keep prior generations' lines (the lobby-loop + // relaunch truncates the shared BT_LOG otherwise -- multi-generation + // forensics need the whole trail). + logfile.open(getenv("BT_LOG") ? getenv("BT_LOG") : "btl4.log", + (getenv("BT_LOG_APPEND") != NULL && *getenv("BT_LOG_APPEND") == '1') + ? (std::ios::out | std::ios::app) : std::ios::out); std::cout.rdbuf(logfile.rdbuf()); // MP wire-format probe (env BT_NET_PROBE=1): print the exact packet constants diff --git a/tools/btconsole.py b/tools/btconsole.py index 766d6de..0fafca4 100644 --- a/tools/btconsole.py +++ b/tools/btconsole.py @@ -227,6 +227,7 @@ ROUTE_SEAT_ASSIGN = -7 # relay->client: int32 hostID + NUL-terminated tag ROUTE_SEAT_FULL = -8 # relay->client: no free seats ROUTE_MATCHLOG = -9 # client->relay: matchlog upload (filename NUL + bytes) ROUTE_READY = -10 # client->relay: mission load complete (READY light) +ROUTE_REJOIN = -11 # relay->client: abandon the round, rejoin (peer died mid-load) # the 18 certified mech tags (mirror of the FE kVehicles catalog) VEHICLE_TAGS = {"blkhawk", "loki", "bhk1", "madcat", "thor", "owens", "own1", "thr1", "lok1", "mad1", "avatar", "ava1", @@ -1040,6 +1041,30 @@ class Relay: except OSError as e: self._drop_game(conn, f"send failed {e!r}") + def _abort_round(self, why): + """ROUND ABORT (connection audit 2026-07-23): a registered pod died + AFTER egg release but BEFORE launch -- the engine's all-connections + gate would leave every survivor waiting on it forever. Bounce all + remaining pods back to their seats (REJOIN) and reset the round; the + reclaim window puts everyone back in the same seats in seconds.""" + if not self.eggs_released or self.launches_sent >= 2: + return + print(f"[relay] ROUND ABORT ({why}) -- sending every pod back to " + f"the join wait for a fresh round", flush=True) + # reset FIRST so the cascade of resulting drops can't re-trigger + self.eggs_released = False + self.launch_at = None + self.launch_requested = False + self.eggs_done_at = None + self._launch_blocked_warned = False + survivors = list(self.by_host.values()) + for conn in survivors: + try: + self._send_raw(conn, struct.pack(ENV_FMT_TCP, ROUTE_REJOIN, 0)) + except OSError: + pass + # the full state restore happens in _maybe_reset_round as they drop + def _drop_game(self, conn, why): print(f"[relay] {conn.name()} dropped: {why}", flush=True) try: @@ -1066,12 +1091,16 @@ class Relay: print(f"[relay] PLAYER {seat - FIRST_GAME_HOST_ID + 1} LEFT " f"(host {seat}) tag='{tag}' " f"({len(self.seat_beacons)} seated)", flush=True) + was_registered = (conn.host_id is not None + and self.by_host.get(conn.host_id) is conn) if conn.host_id is not None and self.by_host.get(conn.host_id) is conn: del self.by_host[conn.host_id] self.udp_endpoint.pop(conn.host_id, None) for other in list(self.by_host.values()): # snapshot: a failed # send drops that peer INSIDE the loop (mutates by_host) self._send_control(other, ROUTE_PEER_DOWN, conn.host_id) + if was_registered: + self._abort_round(f"host {conn.host_id} dropped mid-load") self._maybe_reset_round() # ---------------- game UDP side ----------------