MP: console-death freeze NOT REPRODUCIBLE + fix the latent game-listener-close bug (task #50)

Investigated the "console death froze peer replication" open question (2026-07-14)
with a direct 2-node repro (BT_REPL_LOG on a circling peer, affinity-pinned):

- Clean relay kill -> replicant kept circling smoothly through the kill, NO
  disconnect logged, node survived. Replication did NOT freeze.
- Deliberately STUCK console (scratchpad/btconsole_stuck.py: connect, start
  mission, then stop recv() while holding the socket OPEN = the exact
  "receive pad full -> close never seen" mode hypothesized) -> replicant kept
  moving the whole run. Replication did NOT freeze.

Conclusion: peer replication is INDEPENDENT of the console (pods replicate
peer-to-peer over the GAME socket; the console is a separate egg/mission/status
channel). The original freeze was a transient -- plausibly the same single-box
packet-jitter/CPU-contention artifact root-caused in 49d73dc -- or was fixed by
later MP work.

FIX (the one real bug found): L4NetworkManager::HostDisconnectedMessageHandler's
ConsoleHostType branch closed gameListenerSocket (the GAME listener) on a CONSOLE
disconnect -- a naming bug (the comment + commented-out OpenConnection intended a
CONSOLE re-listen, which CreateConsoleHost already does). Removed. Harmless to
established peer sockets (why a live match survives console loss) but it would
have blocked a NEW peer from joining after a console cycle. Verified: 2-node
still connects (All connections completed!) + the peer circles.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

scratchpad/btconsole_stuck.py

@@ -0,0 +1,95 @@
+#!/usr/bin/env python3
+"""btconsole_stuck.py -- a DELIBERATELY-STUCK console for reproducing the
+'console death froze peer replication' open question (task #50 / open-questions.md).
+
+It behaves like tools/btconsole.py (connect, stream egg, send RunMission x2, drain
+the pod->console stream) BUT after STUCK_AFTER seconds it STOPS draining while
+holding the socket OPEN -- i.e. it stops calling recv(). This is the authentic
+'receive pad full -> close never seen' failure mode (NOT a clean disconnect): the
+pod keeps queueing pod->console packets, the TCP window closes, and we watch
+whether the pod's CheckBuffers loop stalls the GAME peers behind the wedged
+console host.
+
+Usage:  python btconsole_stuck.py <EGG> host:port host:port  [STUCK_AFTER_secs]
+Env: none. STUCK_AFTER default 12s after the 2nd RunMission.
+"""
+import socket, struct, sys, time, threading, os
+
+CHUNK = 1000
+PKT_FMT_HDR = "<iiii"
+PKT_FMT_MSG = "<IiIiii"
+MESSAGE_LENGTH = 1024
+EGG_MESSAGE_ID = 3
+RELIABLE_FLAG = 1
+CONSOLE_HOST_ID = 1
+APPLICATION_CLIENT_ID = 4
+RUN_MISSION_MESSAGE_ID = 5
+LAUNCH_SETTLE_SECONDS = 20.0
+LAUNCH_STEP_SECONDS = 4.0
+STUCK_AFTER = float(sys.argv[4]) if len(sys.argv) > 4 else 12.0
+
+def run_mission_packet():
+    pkt = struct.pack(PKT_FMT_HDR, APPLICATION_CLIENT_ID, 0, CONSOLE_HOST_ID, 0)
+    pkt += struct.pack("<IiI", 12, RUN_MISSION_MESSAGE_ID, RELIABLE_FLAG)
+    return pkt
+
+def egg_wire_bytes(b):
+    return b"".join(l + b"\0" for l in b.replace(b"\r\n", b"\n").split(b"\n"))
+
+def egg_packets(egg_bytes):
+    total = len(egg_bytes); seq = 0; off = 0
+    while off < total:
+        chunk = egg_bytes[off:off+CHUNK]
+        pkt = struct.pack(PKT_FMT_HDR, 0, 0, CONSOLE_HOST_ID, 0)
+        pkt += struct.pack(PKT_FMT_MSG, MESSAGE_LENGTH, EGG_MESSAGE_ID, RELIABLE_FLAG, seq, total, len(chunk))
+        pkt += chunk.ljust(CHUNK, b"\0")
+        yield pkt
+        off += len(chunk); seq += 1
+
+def serve_pod(hostport, egg_bytes):
+    host, port = hostport.rsplit(":", 1); port = int(port); name = f"{host}:{port}"
+    while True:
+        try:
+            s = socket.create_connection((host, port), timeout=5); break
+        except OSError as e:
+            print(f"[{name}] waiting for pod ({e})", flush=True); time.sleep(1)
+    print(f"[{name}] connected; streaming egg ({len(egg_bytes)} bytes)", flush=True)
+    for pkt in egg_packets(egg_bytes):
+        s.sendall(pkt)
+    print(f"[{name}] egg sent; launch in {LAUNCH_SETTLE_SECONDS}s", flush=True)
+    s.settimeout(2.0)
+    launch_at = time.time() + LAUNCH_SETTLE_SECONDS
+    launches_sent = 0
+    stuck_at = None
+    while True:
+        if launches_sent < 2 and time.time() >= launch_at:
+            s.sendall(run_mission_packet()); launches_sent += 1
+            launch_at = time.time() + LAUNCH_STEP_SECONDS
+            print(f"[{name}] RunMission #{launches_sent} sent", flush=True)
+            if launches_sent == 2:
+                stuck_at = time.time() + STUCK_AFTER
+        # *** THE FAILURE: once mission is running, stop draining after STUCK_AFTER ***
+        if stuck_at is not None and time.time() >= stuck_at:
+            print(f"[{name}] *** GOING STUCK: socket stays OPEN, no more recv() ***", flush=True)
+            while True:
+                time.sleep(60)   # hold the socket open, never drain it again
+        try:
+            data = s.recv(4096)
+            if not data:
+                print(f"[{name}] pod closed the console socket", flush=True); return
+        except socket.timeout:
+            continue
+        except OSError as e:
+            print(f"[{name}] socket error {e!r}", flush=True); return
+
+def main():
+    egg_path = sys.argv[1]
+    pods = sys.argv[2:4]
+    with open(egg_path, "rb") as f:
+        egg = egg_wire_bytes(f.read())
+    ts = [threading.Thread(target=serve_pod, args=(p, egg), daemon=True) for p in pods]
+    for t in ts: t.start()
+    while True: time.sleep(1)
+
+if __name__ == "__main__":
+    main()