# Phase 0 — Lab proof plan Drafted 2026-07-10. Goal: prove every load-bearing SiteLink assumption **with zero cockpit hardware and zero travel**, using VMs that simulate two sites + the hub. Everything marked "verify" in [BRAINSTORM.md](BRAINSTORM.md) gets an experiment here; each experiment names its pass criteria and the open question it closes. ## 1. Objectives / exit criteria Phase 0 is done when we can state, with captures/logs in hand: 1. A TeslaConsole at site A commands a pod (vPOD) at site B over routed WireGuard (RPC 53290) — including under realistic WAN latency. *(OQ: console over WAN)* 2. A Firestorm client at site B joins a session hosted at site A (or the hub) by directed IP, across subnets, no broadcast required — and we know the exact port set on the wire. *(OQ 1, 2)* 3. We have a measured latency/jitter/loss **cliff table** for Firestorm, and know what a mid-match tunnel drop does. *(OQ 4, 10)* 4. BT411's console can push an egg to pods on two subnets and the pod mesh forms (or we know exactly why not). *(OQ 3)* 5. The hub concept is rehearsed: camera-ship-as-host confirmed (or refuted, with the dedicated-server fallback exercised), output captured and streamed. *(OQ 5 + hub/Live Cam design)* 6. An event-day dry run passes end to end: collect `.siteconfig`s → `SiteConfigMerge` exits 0 → central console sees and commands both sites' pods. Deferred past Phase 0: TeslaRel410/NetNub cross-subnet testing (OQ 11) — waits for the emulator's own networking phase (its PLAN.md Phase 6). ## 2. Lab topology Mirrors the production design 1:1, just smaller: two sites of the *minimum* real bay shape (console + 2 cockpits) plus the hub. ``` "WAN" (host-only net, 192.168.77.0/24) netem applied here = simulated internet ┌──────────────────┬──────────────────┬─────────────────┐ │ │ │ ┌────┴────┐ ┌────┴────┐ ┌────┴────┐ │ gw-a │ │ hub-gw │ │ gw-b │ │ Linux │◄──WG──►│ Linux │◄──WG──►│ Linux │ WG overlay 10.255.0.0/24 └────┬────┘ └────┬────┘ └────┬────┘ hub-and-spoke (as production) │ │ │ site A LAN hub LAN 10.0.0.0/24 site B LAN 10.0.1.0/24 ┌──────────────┐ 10.0.2.0/24 ┌────────────┐ │ hub-fs (Win) │ ┌────────────┐ │ console-a │ │ FS host / LC │ │ console-b │ │ pod-a1 │ │ + OBS stream │ │ pod-b1 │ │ pod-a2 │ └──────────────┘ │ pod-b2 │ └────────────┘ └────────────┘ ``` ### Lab IP conventions (strawman — align with real bays before Phase 1) | Host | Address | Notes | |------|---------|-------| | Site gateways | `10.0..254` | Linux, WireGuard + nftables + netem | | Cockpits | `10.0..1–.8`, `.11–.18` | per the historical seat map (`ctcl-game.ini`) | | Camera / Live Cam | `10.0..9` | historical | | Mission Review | `10.0..10` | proposed | | Site console | `10.0..100` | proposed | | Hub WG | `10.0.0.1` | hub-gw | | Hub FS host / LC / MR | `10.0.0.20` | hub-fs | | Hub services (PDF share, later) | `10.0.0.10` | | | WG overlay | `10.255.0.0/24` | hub `.1`, gw-a `.2`, gw-b `.3` | **Open item for Phase 1:** confirm the canonical last-octet map for console/MR against how real bays are actually numbered; the pod seat octets are historical fact, the rest above is proposal. ## 3. VM inventory ### Full lab (9 VMs) | VM | OS | vCPU / RAM / disk | Role | |----|----|-------------------|------| | gw-a, gw-b, hub-gw | Debian 12 (or Alpine) | 1 / 512 MB / 4 GB | routing, WireGuard, nftables, netem | | console-a, console-b | Windows 10 22H2 | 2 / 4 GB / 60 GB | TeslaConsole, SiteConfigMerge, btconsole.py | | pod-a1, pod-a2, pod-b1 (+pod-b2 optional) | Windows 10 22H2 | 2 / 4 GB / 60 GB | TeslaLauncher + vPOD; MW4 deploy for match tests | | hub-fs | Windows 10 22H2 | 4 / 8 GB / 80 GB, **3D accel ON** | FS host/camera ship, OBS, later MR instance | ### Minimum viable lab (5 VMs) — start here gw-a, gw-b (one of them doubling as hub-gw), console-a, pod-b1, hub-fs. That's enough for experiments E1–E3 and E6; grow toward the full set as experiments demand. ## 4. VM management recommendations **Hypervisor.** The dev box runs **Windows 11 Home — Hyper-V is not available**. Recommendation: - **First pass: VMware Workstation Pro 17** on the dev box — free for personal use since 2024, best-in-class old-DirectX 3D acceleration among desktop hypervisors (matters for MW4 in a VM), proper snapshot trees, **linked clones** (one Windows gold image, thin per-VM deltas), and per-VMnet host-only networks that map exactly to the topology above (VMnet2 = site A, VMnet3 = site B, VMnet4 = hub LAN, VMnet5 = WAN transit; **DHCP off** on all of them — static IPs per the plan, which also mirrors the bays' air-gapped discipline). - **Persistent lab (recommended once Phase 0 proves out): Proxmox VE on a spare box.** Pod operators tend to have spare hardware; a single Proxmox host gives the lab a permanent home with a web UI, scheduled snapshots/backups, Linux bridges per site LAN, and it can later graduate into the *real* hub staging machine. 16 GB RAM hosts the minimum lab; 32 GB hosts the full set comfortably. - VirtualBox works as a fallback but its 3D path for 1999-era DirectDraw is the weakest of the three — expect to lean harder on the physical-host fallback for game VMs. **Images and clones.** - Build **one Windows 10 gold image** (22H2, VMware Tools, updates frozen, Defender real-time off *in the lab only*, Windows Firewall configured per experiment — never just disabled, since firewall behavior is part of what Phase 0 tests). Clone everything Windows from it (linked clones). Keep one **Linux gold** the same way. - Windows licensing for the lab: 90-day Enterprise eval ISOs are fine; snapshots + re-arm cover Phase 0's lifetime. Nothing in the lab needs activation. - Name VMs exactly as in §3 (`gw-a`, `pod-b1`, …) — captures, logs, and notes all key off those names. **Snapshots as method.** Take a `baseline` snapshot of every VM the moment its role software is installed and verified idle-healthy. **Snapshot before every experiment** (`E3-pre`), and roll back rather than un-configuring. The experiment log (§6) records which snapshot each result came from. Export the gold images as OVAs to backup storage once — everything else is reproducible from them. **Config as code, in this repo.** Everything text lands under `lab/`: `lab/gw/` (wg configs with lab keys, nftables rules, `wan.sh`), `lab/checklists/`, `lab/results/` (experiment logs + pcap summaries; raw pcaps stay out of git, keep them on the lab host). Lab WireGuard keys are throwaway and may be committed; production keys never (per `.gitignore` policy). **netem control.** One knob, on both site gateways' WAN egress (half the RTT each side, symmetric): ```sh # lab/gw/wan.sh — usage: wan.sh e.g. wan.sh 30ms 5ms 0.1% tc qdisc replace dev eth0 root netem delay $1 $2 distribution normal loss $3 ``` Standard sweep points for every latency-sensitive experiment: LAN-like (0/0/0) → regional (15ms/2ms/0) → cross-country (40ms/5ms/0.1%) → bad day (80ms/15ms/0.5%) → hostile (150ms/30ms/1%) → find the cliff. ## 5. Experiments Each: goal → method → pass criteria. Run in order; later ones reuse earlier setup. - **E1 — Routed fabric + console RPC over WAN.** Bring up hub-and-spoke WireGuard, static routes, nftables allowlist per the ecosystem port map. From console-a, provision nothing (provisioning stays local by design) but command a **vPOD** on pod-b1: status, Install Product, launch. Sweep netem. *Pass:* RPC 53290 works at every sweep point up to "bad day"; note where timeouts start (feeds the WAN-tolerant-timeout to-do). - **E2 — Broadcast locality sanity.** Confirm SecureConfig UDP beacons and DirectPlay LAN browse do *not* cross the tunnel (expected, by design) and that nothing else in the console/pod bring-up secretly depends on broadcast. *Pass:* pod provisioned locally works remotely thereafter; no cross-site flow ever relied on broadcast (pcap-verified). - **E3 — Firestorm directed join across subnets.** MW4 host on console-a (or hub-fs), client on pod-b1. Set the `DirectPlayPort` registry value; join by IP (`TryToJoinASpecificGame` path via the ConLobby/CTCL flow, not the LAN browser). Wireshark both gateways. *Pass:* client joins and plays; complete port matrix documented (the firewall allowlist becomes fact instead of DX7 documentation); confirms whether the fixed port carries all session traffic. **Closes OQ 1 + 2.** - **E4 — Firestorm latency cliff.** With E3 running, walk the netem sweep during actual play (movement + weapons). Record subjective playability + any desync or disconnect per point. *Pass:* a written cliff table ("clean ≤ X ms RTT, degraded at Y, breaks at Z"). **Closes OQ 4 for FS.** - **E5 — WAN-drop behavior.** Kill the tunnel mid-match (down the WG interface); restore after 10s / 60s / 5min. *Pass:* documented behavior per flow (game session, console RPC, vPOD state) and confirmation each site's bay-local operation is unaffected. **Closes OQ 10.** - **E6 — MTU/fragmentation.** DF-bit probing host-to-host across the tunnel (WG MTU 1420); watch E3's DirectPlay UDP for fragmentation; test an MSS clamp on the gateways. *Pass:* no silent blackholing; a stated MTU/clamp recommendation for production gateways. - **E7 — BT411 cross-subnet egg push.** `btconsole.py MP.EGG 10.0.1.x:1501 10.0.2.x:1501` with btl4 instances at both sites. Inspect how the `[pilots]` mesh addresses peers (read `L4NET.CPP` alongside the capture). Netem sweep. Also observe the console-disconnect quirk over a flaky tunnel. *Pass:* mesh forms across subnets (or root cause written up); latency tolerance noted. **Closes OQ 3.** - **E8 — Hub host + broadcast rehearsal.** On hub-fs: run the FS host in camera-ship role (validate camera-ship-as-DirectPlay-host against the CTCL flow); capture with OBS → SRT to both consoles as stand-in "Live Cam screens". Build/run check of `mw4dedicatedui` as the headless fallback. If VM rendering is unusable, rerun on a physical host (see risks). *Pass:* one match hosted at the hub with both sites joined, stream watched at both consoles. **Closes OQ 5 and validates the hub/Live Cam design.** - **E9 — Event-day dry run.** Simulate the full authority-handover ceremony: `console-a` and `console-b` each export `.siteconfig` → transfer to hub → `SiteConfigMerge merge` (must exit 0) → central console (hub or console-a wearing the hat) loads `master.siteconfig` and commands vPODs at **both** sites → restore site configs afterward. *Pass:* scripted checklist completes without manual surgery; becomes the seed of the production event-day runbook. ## 6. Results discipline One markdown file per experiment in `lab/results/` (`E3-fs-directed-join.md`): date, snapshot names, netem settings, what happened, pcap filenames (pcaps stay on the lab host), verdict against pass criteria. Findings that change the design get folded back into BRAINSTORM/ECOSYSTEM in the same commit — the docs stay truthful. ## 7. Risks / known unknowns | Risk | Mitigation | |------|------------| | MW4 rendering inside VMs (DX7/DirectDraw + `DWM8And16BitMitigation` shim, keyed on exe path) | VMware 3D accel + DDrawCompat if needed; windowed mode; **fallback: run game instances on physical hosts** while gateways/consoles stay virtual — the network fabric under test doesn't care where the game runs | | vPOD fidelity limits | vPOD impersonates launcher + Munga control, **not** DirectPlay gameplay — E3/E4/E8 need real MW4 instances; don't over-conclude from vPOD-only runs | | Windows Firewall/Defender masking network results | Firewall rules are explicit per experiment, never blanket-off; Defender off only in lab gold image, noted in every result | | netem placement asymmetry | Apply on both gateways' egress; sanity-check RTT with ping before each run | | CTCL flow surprises (host IP propagation) | E3 exercises the *real* ConLobby/CTCL join, not just raw DirectPlay — that's the point | ## 8. Sequencing / effort (evenings-and-weekends scale) - **Weekend 1:** gold images, minimum lab (5 VMs), E1 + E2. - **Week 2:** E3 + E6 (join + ports + MTU), start E4. - **Week 3:** E4 + E5 (cliff table, drop behavior). - **Week 4:** E7 (BT411) and E8 (hub/stream rehearsal). - **Wrap:** E9 dry run, fold results into the docs, go/no-go for Phase 1.