# SiteLink — design brainstorm

Working record, started 2026-07-10. Companion to
[PODBAY-ECOSYSTEM.md](PODBAY-ECOSYSTEM.md), which holds the verified facts this
brainstorm builds on.

## 0. Goals (proposed — confirm/edit)

1. **Cross-site play**: cockpits at site A and site B in the same match (Firestorm first;
   BT411/RP as they mature).
2. **Preserve the air gap**: the linked fleet is one private network. No bay machine ever
   gets a route to the general internet.
3. **Fleet visibility/command**: an operator (eventually a master console) can see — and
   where appropriate command — pods at every site.
4. **Shared front-of-house**: cross-site events share a queue, callsigns, scores, voice.
5. **Zero changes to cockpit hardware**, minimal changes to bay-local operation: a bay
   must keep working stand-alone when the link is down.

Non-goals (for now): public matchmaking, spectating from home, per-player home clients.

## 1. The seed proposal

> Each site gets its own `10.0.y.x` subnet (we get `10.0.1.x`, you get `10.0.2.x`, …),
> VPN them all together into a single /16, collect every site's `.siteconfig`,
> concatenate them, and give the result to a master console that commands the fleet.

**Verdict: sound skeleton.** The three pieces (site-numbered subnets, VPN mesh into
`10.0.0.0/16`, merged fleet config) survive contact with the code. The refinements below
are about *how*, not *whether*.

## 2. Addressing plan

- **`10.0.<siteID>.0/24` per site**, keeping each bay's existing last-octet conventions
  (pods `.1–.8`, `.11–.18`, camera `.9`, …) so per-site configs become a template
  stamped with a site ID. Legacy `200.0.0.x` bays get renumbered on joining — that
  space is public internet space and must not exist inside the linked fleet.
- **Reserve `10.0.0.0/24` for shared infrastructure**: VPN hub, master console, central
  PQS, Mumble, NTP, a neutral dedicated game host. Nothing site-specific lives there.
- **Site ID registry lives in this repo** (`sites/` — one small file per site: ID, name,
  operator, subnet, contact). First allocation: `10.0.1.0/24` = Fallout Shelter Arcade;
  `10.0.2.0/24` = next site. 254 sites max — plenty.
- **Netmask choice is the real design fork** (see §3): hosts configured `/24` + gateway
  = routed model; hosts configured `/16` = requires a bridged L2 overlay.
- Renumbering mechanics: TeslaConsole already owns pod IP config (provisioning writes
  `mIPAddress/mGateway/mDns/mSubnet`), and the CTCL `[teslas]` inis are trivially
  regenerated from a template. So renumbering is a console-driven afternoon, not a
  reimage.

## 3. Topology: routed L3 vs bridged L2

### Option A — Routed L3 (recommended default)
Each site keeps its `/24`; a per-site VPN gateway (its *only* new hardware) routes
between sites over **WireGuard** tunnels.

- ✅ Works because Firestorm's join path is **directed by IP** (`TryToJoinASpecificGame`,
  see ecosystem doc) and every console→pod protocol (53290 RPC, Munga 1501, BT egg
  push) is directed TCP. Nothing in the *match-play* path needs broadcast.
- ✅ Clean failure isolation, no cross-WAN broadcast chatter from era Windows boxes,
  easy per-flow firewalling at each gateway.
- ✅ Hub-and-spoke (hub on shared infra, e.g. alongside mysticmachines.com) or full mesh —
  WireGuard does either; start hub-and-spoke, it's simpler and the hub is also where
  shared services live.
- ⚠️ Breaks broadcast-dependent conveniences: DirectPlay "browse LAN games" listing and
  SecureConfig first-boot beacons don't cross sites. Both are acceptable: game joins are
  console-directed anyway, and provisioning is an on-site act by design.

### Option B — Bridged L2 overlay (fallback / experiment)
One virtual Ethernet across all sites (ZeroTier, VXLAN/EoIP between gateways, tinc
switch mode); hosts could then even use a flat `/16`.

- ✅ Everything behaves like one big LAN — broadcast discovery, any undiscovered
  broadcast assumption in 1996–2009 era code, zero renumbering logic beyond uniqueness.
- ❌ One broadcast domain of ancient unpatched Windows across the WAN (NetBIOS chatter,
  broadcast storms), MTU headaches, harder to reason about, and failure at one site can
  be noisy everywhere.
- Position: **keep in the back pocket.** If Phase-0 testing (vPOD + two subnets) turns up
  a flow that genuinely requires broadcast, bridge *only* that (e.g. a targeted UDP
  broadcast relay) before going full L2.

### VPN tech choice
WireGuard first (kernel-fast, tiny config, UDP hole-punch friendly, runs on a $50 box or
the site router). Tailscale/NetBird/ZeroTier are managed alternates if key/peer
management becomes a burden — but note ZeroTier is the L2 option, Tailscale/NetBird are
L3. The bay machines themselves never run VPN software; only the per-site gateway does.
**Bay boxes keep no default route** — gateways carry routes for `10.0.0.0/16` only.

## 4. Fleet configuration: master console vs federation

The seed proposal: merge all `.siteconfig` files → one master console commands the fleet.

**Reality check on "concatenate":** `local.siteconfig` is a BinaryFormatter object graph
(Squad → Pods), not a text file — merging means a small tool that deserializes N
siteconfigs and emits one multi-squad config. Straightforward since TeslaConsole is
rebuilt from source; the natural mapping is **one Squad per site** ("FSA-bay1",
"Pharaoh-bay1", …). TeslaConsole's UI already organizes pods by squad.

**Security consequence:** a pod's `mKey` in the siteconfig *is* command authority over
that pod. Shipping your siteconfig to a master console = granting fleet-wide control.
Fine between trusted operators, but do it deliberately:

- Siteconfigs travel operator-to-operator over a secure channel; **never through this
  repo** (repo carries the merge tool + templates only).
- A master console with all keys is a single high-value target — it lives on shared
  infra (`10.0.0.0/24`), not on someone's laptop.

**Three architectures, in order of effort:**

1. **Merged-config master console (the seed).** One TeslaConsole instance with all
   squads/keys. Cheapest to reach; needs the merge tool + WAN-tolerant RPC timeouts.
   Risks: WAN blip mid-install, and two consoles (site + master) commanding the same pod
   concurrently — the launcher has no arbitration. Convention needed: master commands
   only during cross-site events, site console otherwise.
2. **Hybrid (probably the sweet spot).** Site consoles stay authoritative for
   provisioning/installs; the master console gets *visibility everywhere* +
   *game-orchestration rights* during events. Could be as simple as (1) plus agreed
   scope, or a read-only fleet mode in TeslaConsole.
3. **Federation service.** A SiteLink coordinator each site console registers with; the
   coordinator brokers cross-site game setup, no raw pod keys leave a site. Cleanest
   trust story, most new code. Later phase, if the fleet grows past a handful of
   trusted operators.

## 5. Linking the games themselves

### Firestorm (first target)
- **Session model:** one DirectPlay host; everyone joins directed-by-IP. Options for
  host placement:
  - a) Host at one site's console (simplest; other site eats the WAN RTT; O(n²) load on
    that console's uplink);
  - b) **Neutral dedicated host** on shared infra (`mw4dedicatedui` exists!) —
    symmetric latency for both sites, best uplink, and the camera/review stations at
    *each* site can join as local spectators. Needs validation that the dedicated
    server builds/runs (it's in the solution but unexercised in the modern work).
- **Player counts:** the golden first milestone is **8v8 across two sites (8 cockpits
  each) — inside the stock 16-player cap, zero engine changes.** Full 16v16 site-vs-site
  needs the already-drafted 16→32 plan in `firestorm\CLAUDE.md` (compiled defaults +
  drop-zone authoring per map + lobby UI polish).
- **Latency budget:** the netcode shipped for dial-up (150–300 ms era). Inter-city
  internet (20–80 ms) + WireGuard (~1 ms) should be comfortable. Measure, don't assume:
  put netem delay/jitter/loss on a test gateway and find the cliff (see Phase 0).
- **Port hygiene:** set the `DirectPlayPort` registry value at every site so game traffic
  is a single known port for the gateways' firewalls, instead of DirectPlay's default
  47624 + 2300–2400 spread.
- **Fixed-port + MTU checks:** DirectPlay UDP datagrams over a 1420-byte WireGuard MTU —
  verify no fragmentation weirdness (era stacks are fragile here).

### BT411 (classic BattleTech)
- The console already launches multi-pod games by address list (`btconsole.py MP.EGG
  10.0.1.11:1501 10.0.2.11:1501` *is* a cross-site launch). Two unknowns to resolve:
  1. Does the pod↔pod mesh derived from the egg `[pilots]` list carry literal IPs
     (routes fine) or assume same-subnet?
  2. The 1996 sim's latency tolerance — lockstep-ish engines of that era can be
     unforgiving. netem test early.
- Console-must-stay-connected quirk means the WAN link drop kills the match — argues for
  running the BT console *at the site with the most pods*, or fixing the engine bug.

### Red Planet (RP411 reconstruction)
- Same console-directed model (Munga TCP 1501). Park until BT411 learnings land — shared
  engine lineage means most answers transfer.

### Original Tesla 4.10 games under emulation (TeslaRel410)
- When the DOSBox-X/VPX emulator matures, the *original* BT/RP binaries land on
  current pod hardware speaking their original network stack: **WATTCP (real
  TCP/IP) via NetNub**, surfaced through NE2000-emulation bridged onto the bay LAN
  via pcap. Because it's plain IP with the emulated pod holding a real bay address,
  the routed-VPN model extends to them *in principle* unchanged.
- Site-local pieces: BOOTP (broadcast) needs an answerer or static config per site —
  same "provision locally, play routably" pattern as everything else.
- Unknowns to test (cheap — emulator instances run on any PC, no cockpit needed):
  NetNub discovery/mesh addressing across subnets, and the latency tolerance of a
  30 Hz 1996 lockstep-era sim. Fold into the Phase 0 netem lab once the emulator
  reaches its networking phase (PLAN.md Phase 6).
- Strategic note: BT411/RP411 (native, netcode we can patch) and TeslaRel410
  (faithful, netcode frozen in 1996) are complementary. If the original protocol
  proves WAN-hostile, the answer may be "emulated originals for local play, native
  reconstructions for cross-site" — SiteLink shouldn't promise WAN play for the
  emulated path until measured.

## 6. Shared services (the `10.0.0.0/24` rack)

| Service | Notes |
|---------|-------|
| WireGuard hub | The rendezvous point; a VPS (could sit near the existing mysticmachines.com infra) or a box at the best-connected site |
| Mumble | Cross-site voice day one — the FS507D release already integrated Mumble on the bay side. Channel per team, cross-team lobby channel |
| Central PQS (event mode) | Cross-site events need one queue/roster. Options: (a) one central PQS all consoles poll — simplest, DB schema grows a `site` column; (b) per-site PQS + sync — only if WAN-down resilience of the local queue matters during events. Callsign uniqueness becomes fleet-wide (global registration table) |
| Neutral game host | Firestorm dedicated server for site-vs-site matches |
| NTP | One clock for scores, logs, and replay/debrief alignment |
| Fleet monitoring | VncThumbnailViewer pointed across the VPN; later, master-console status board |

Local-only forever: printer, RIO/cockpit I/O, Mission Review & Live Cam *stations*
(they join the match as spectators over the VPN, but the screens/printer serve the local
audience). Cross-site match results reach both printers via PQS/debrief distribution —
worth a small design of its own later.

## 7. Security posture

- **The fleet stays air-gapped as a whole.** Only gateways touch the internet, only to
  carry WireGuard to known peers. Bay machines: no default route, no DNS egress, static
  `10.0.0.0/16` routes via the gateway.
- **Gateway firewall = allowlist by flow**, per the port map in the ecosystem doc.
  Suggested split:
  - *Game plane* (DirectPlay fixed port, Munga 1501, Mumble): site↔site and site↔shared.
  - *Management plane* (TCP 53290 launcher RPC, VNC 5900): only site-console↔own-pods
    and master-console↔pods — other sites' consoles have no business on your 53290.
- **Secrets:** siteconfigs (pod keys) and WireGuard private keys never enter this repo.
  Repo holds templates, tools, and *public* site registry data (subnets, endpoints'
  hostnames, WireGuard public keys).
- Era-Windows reality: assume every bay box is compromised-if-reachable. The allowlist
  above is the actual security boundary; the VPN is transport, not trust.

## 8. Open questions

1. **FS end-to-end join over routed subnets** — directed-join API confirmed in source;
   prove the whole ConLobby/CTCL flow passes the host IP (vPOD + two subnets, or two
   VMs). Any broadcast dependency hiding in the lobby flow?
2. **`DirectPlayPort` in practice** — is it set at existing sites, and does a fixed port
   carry all session traffic (or only enumeration) under DirectPlay Protocol?
3. **BT411 `[pilots]` mesh** — IPs or subnet assumptions? (Read `L4NET.CPP`
   StartConnecting path.)
4. **Latency cliffs per title** — netem sweep: at what RTT/jitter/loss does each engine
   degrade (rubber-banding, desync, disconnect)?
5. **Dedicated Firestorm host** — does `mw4dedicatedui` build & run in the modern tree,
   and can camera/MR spectators join it cleanly?
6. **PQS multi-site schema** — site column vs event database; global callsign policy.
7. **Master-console arbitration** — what happens when site + master consoles command the
   same pod; do we need a soft lock ("bay is in fleet mode")?
8. **Who hosts shared infra** — VPS vs best-connected site; bandwidth math for a
   32-player O(n²) host (~upstream estimate needed from real packet captures).
9. **Site count ambitions** — 2 sites soon, how many eventually? Affects hub sizing and
   whether federation (§4.3) is ever needed.
10. **WAN-drop behavior** — for each linked flow, what breaks when the tunnel drops
    mid-match, and does the bay cleanly fall back to stand-alone?
11. **NetNub over routed subnets (TeslaRel410)** — how do the original 4.10 games
    discover/address each other (read `CODE\*\MUNGA_L4\NETNUB\` + `L4NET` lineage)?
    Broadcast-dependent or console-directed like their descendants? And what RTT does
    the 30 Hz 1996 sim tolerate?

## 9. Phased roadmap (strawman)

- **Phase 0 — Lab proof (no hardware, no travel).** Two "bays" as VMs/vPOD instances on
  two subnets with a WireGuard/netem gateway between them. Prove: console→remote-pod
  RPC (53290), a 2-client Firestorm match across subnets, latency sweep, BT411
  cross-subnet egg push. *Everything above that's marked "verify" gets verified here.*
- **Phase 1 — First real link.** Two sites, gateways, tunnel, renumber to
  `10.0.1.x`/`10.0.2.x`. Mumble + fleet VNC + remote Mission-Review spectating. First
  cross-site Firestorm match at ≤16 total players (8v8) on the stock exe.
- **Phase 2 — Fleet operations.** siteconfig merge tool → master console visibility
  (hybrid model, §4.2). PQS event mode with shared queue/callsigns. Debrief/scoresheet
  distribution to both printers.
- **Phase 3 — Full site-vs-site.** Execute the firestorm 16→32 plan (code defaults +
  drop zones), neutral dedicated host, 16v16 events.
- **Phase 4 — More titles, more sites.** BT411/RP cross-site as those engines mature;
  3rd+ site onboarding kit (gateway image + site-ID allocation + config templates);
  revisit federation and the GUN/GameSpy-style fleet game browser.