Files
RP412/MUNGA/SIMULATE.h
T
CydandClaude Fable 5 7d485c9672 The wire keeps what it could not send
Cyd asked for an analysis of the networking stack and what would make the
simulation feel better over the internet. The analysis found something
more urgent than latency: the transport has been losing data silently
since the arcade, and nothing in the game could see it happen.

Every send result was discarded - L4NET, the console, all of it. On the
1ms arcade LAN the socket buffer never filled, so it never mattered. Over
the internet it matters twice. A peer stalled in its own 10-30 second
mission load stops reading, its window closes, and our nonblocking send
starts answering would-block, which threw the message away; or worse,
answering a PARTIAL count, and since framing on that stream is recovered
purely from each message's length prefix, the bytes that never followed
sheared it for good. Both are reachable in an ordinary race, because
every race has a load in it.

So sends go through a bounded per-connection queue now. What the wire
will not take is kept, byte-exact, and retried at three flush points -
before the render (the present blocks on vsync, and this frame's state
should be travelling while it does), at the top of the receive pump, and
before a connect sequence. Nothing is ever dropped from the middle: these
are reliable ordered messages carrying entity creation, damage and race
control, so a queue that overflows its 256K declares the connection dead
and lets the disconnect path run rather than quietly desyncing the
stream. RP412NETSENDQ=0 restores the old behaviour and still logs what it
would have lost, which is the honest way to A/B it. On Steam the same
queue finally surfaces k_EResultLimitExceeded, which the old code
collapsed into -1 and discarded - that was backpressure, unlogged.

The receive side gained the check the release build never had. The length
prefix is untrusted input; Verify() compiles away in release, so a
corrupt one went to memmove as a negative, or copied 4096 bytes of
assembled packet into a 1600-byte stack buffer, or named a size the pad
could never complete and wedged the connection forever. It is now
validated against the same bounds the sender works to, and a stream that
fails them is dropped like any other lost peer.

And a fry that never ends: drop zones are map entities dealt round-robin
at load, ownership transfer is not implemented, so a leaver's pads stay
in the DropZones group. The respawn request dispatched to one goes to a
host that is gone - dropped at the send, the 'no host N in the table'
path - and the two-second retry re-dispatches to the same dead owner
forever. The pad scan now skips zones whose owner has left, and
re-validates one assigned earlier before reusing it.

The rest is measurement, because the symptoms this work exists to chase
are all reported in prose and none of them are in any log. Sixteen logs
from the six-player night contain zero player-facing latency lines. A
race now ends with a NetLog summary: per remote pod, how many updates
arrived and how evenly (median and p95 out of a log2 histogram), the
widest gap, how many gaps were long enough to mean a quiet sender versus
short enough to mean OUR loop stalled, how often its motion snapped
instead of blending, and how far arriving updates moved it. Per peer,
whether the clock alignment ever had to step mid-race - which is the
input for deciding if it needs slewing, rather than guessing. The mission
t0 tick goes in the log too, alongside the console's per-pod RunMission
send ticks, because nothing has ever measured how far apart the machines
actually start; the clockwork doors inherit that skew directly.

RP412NETSTATS adds the transport's own view - per connection: messages,
bytes, wire writes, partials, refusals, how much sat queued - and on
Steam the first read this codebase has ever taken of GetConnectionRealTime
Status. Ping, quality, pending and unacked bytes, and one route
description per connection at teardown. The API was vendored and never
called; there was no RTT number anywhere in the game.

Finally, rpl4opt -spoolstats reads any recording offline. The data was
already in every spool ever made and nothing read it that way: the
recorder restamps each packet with local arrival time while the update
records inside keep the sender's sim-grid stamp, so the difference is
clock offset plus one-way delay, and the same running-minimum estimator
the game runs live separates them. It prints delay above the per-host
minimum, and decomposes each entity's gaps into sender pacing versus
delivery jitter - which no live counter can do. It lives in the game exe
rather than RPL4TOOL because the tool is deliberately not /Zp1 and would
misread every struct in the file.

Verified on the two-pod loopback harness: mesh up, egg fed, 60s raced,
stopped on command, scores collected, and both summaries reading exactly
what a pair of PARKED pods should read - heartbeat cadence, one snap per
heartbeat, sub-quarter-metre corrections, no clock steps. The t0 ticks
and the netclock offsets agree with each other to the two seconds the
pods launched apart.

The latency tier is deliberately NOT here. TCP_NODELAY, the Steam
NoNagle flag, per-frame coalescing and the pre-sim receive drain are all
scoped and all wait on this build's numbers, because the point of
shipping measurement first is to find out whether the thing we would fix
is the thing that hurts. Nagle is still on. Interest management is still
inert. The wire format is untouched, so this build and the last one still
race each other.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 12:59:25 -05:00

474 lines
12 KiB
C++

#pragma once
#include "state.h"
#include "receiver.h"
#include "time.h"
#include "resource.h"
#include "hostid.h"
//##########################################################################
//########################### Net clock ##############################
//##########################################################################
//
// Aligning a peer's clock with ours, so a replicant is dead-reckoned from
// when its update was SENT rather than when it happened to arrive.
//
// Every update record carries the sender's own timestamp. The receiver
// used to throw it away and stamp lastUpdate with its own Now() - the
// original code says so: "HACK - should be based upon message->timeStamp".
// The dead reckoner then extrapolates over (lastPerformance - lastUpdate),
// so starting that clock at ARRIVAL rather than at SEND leaves every
// remote vehicle exactly one network latency behind where it should be.
// On the 1 ms arcade LAN that was invisible. Over Steam Datagram Relay it
// is a constant 50-150 ms of positional lag - a bias, not jitter.
//
// The timestamp cannot be used raw: two machines' clocks share no epoch,
// both being QueryPerformanceCounter since their own boot. So we estimate
// the offset per peer. Each arriving record gives
//
// sample = ourNow - theirStamp = trueOffset + oneWayLatency
//
// and since latency is never negative, the SMALLEST sample seen is the
// closest to the true offset. Taking a minimum over a short rolling
// window tracks crystal drift and re-adapts when the route changes,
// instead of being pinned forever by one lucky packet.
//
// RP412NETCLOCK=0 turns the whole thing off and restores the arrival-time
// behaviour, so a test machine can A/B it without a rebuild.
//
void NetClock_BeginUpdate(HostID sender); // around one message's records
void NetClock_EndUpdate();
void NetClock_Reset(); // forget every peer (new mission)
void NetClock_ReportRaceStats(); // RP412NETLOG race-end peer lines
class Simulation__SharedData;
class Simulation__IndexData;
struct Simulation__IndexEntry;
class Simulation__AttributeIndexSet;
class MemoryStream;
//##########################################################################
//################# Simulation::UpdateRecord #########################
//##########################################################################
struct Simulation__UpdateRecord
{
public:
size_t recordLength;
Word subsystemID;
Word recordID;
Time timeStamp;
Enumeration simulationState;
};
//##########################################################################
//####################### Simulation #################################
//##########################################################################
class Simulation:
public Receiver
{
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Shared Data Support
//
public:
typedef Simulation__SharedData SharedData;
SharedData*
GetSharedData();
static Derivation *GetClassDerivations();
static SharedData DefaultData;
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Construction and Destruction Support
//
protected:
Simulation(
ClassID class_ID,
SharedData &shared_data
);
public:
~Simulation();
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Attribute Support
//
public:
typedef Enumeration AttributeID;
typedef Simulation__IndexData IndexData;
typedef Simulation__IndexEntry IndexEntry;
typedef Simulation__AttributeIndexSet AttributeIndexSet;
typedef int Simulation::*AttributePointer;
enum {
AnyAttributeID = 0,
SimulationStateAttributeID,
NextAttributeID
};
static const AttributePointer NullAttribute;
void*
GetAttributePointer(AttributeID attribute);
void*
GetAttributePointer(const char* attribute_name);
private:
static const IndexEntry AttributePointers[];
protected:
//static AttributeIndexSet AttributeIndex
static AttributeIndexSet& GetAttributeIndex();
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Simulation Support
//
public:
typedef void
(Simulation::*Performance)(Scalar time_slice);
typedef void
(Simulation::*Encore)();
typedef Simulation__UpdateRecord UpdateRecord;
void
SetPerformance(Performance performance)
{Check(this); activePerformance = performance;}
void
Perform(Scalar time_slice)
{
Check(this);
(this->*activePerformance)(time_slice);
}
virtual void
PerformAndWatch(
const Time& till,
MemoryStream *update_stream
);
//
// The two halves of PerformAndWatch, so an ENTITY can interleave its
// subsystems' physics with its own, step by step, and still run the
// watchers and the update stream once per frame. PerformTo advances
// the simulation to the given time - in fixed steps when
// RP412PHYSICSHZ names a rate, in one variable slice otherwise.
//
void
PerformTo(const Time& till);
void
WatchAndWrite(MemoryStream *update_stream);
//
// Called by the entity interleave at the TOP of every fixed step,
// before any subsystem adds its forces for that step. Per-frame set-up
// work - clearing a force accumulator, deriving local velocity from
// world state - belongs here when the fixed step is on, because "once
// per frame" is a wall-clock cadence and the whole point is that wall
// clock no longer reaches the physics. Default: nothing.
//
virtual void
BeginStep();
//
// Render interpolation hooks, called by PerformTo around the fixed
// step. They live HERE rather than on the entity interleave because a
// REPLICANT never runs that interleave - it reaches PerformTo through
// Simulation::PerformAndWatch instead - and a replicant is exactly what
// every remote pod is. Hanging the snapshot off BeginStep left the
// watched car uninterpolated while the camera watching it was smooth,
// which is most of the way to nowhere.
//
// Defaults do nothing; Entity overrides them because it owns the
// origin. See Entity::GetRenderToWorld.
//
virtual void
SnapshotRenderOrigin();
virtual void
SetRenderStepFraction(Scalar fraction);
//
// The fixed step in seconds, 0 when frame-coupled. Global on purpose:
// a mixed-rate simulation would be a worse bug than either mode.
//
static Scalar
FixedStep();
void
DoNothingOnce(Scalar time_slice);
void
DoNothing(Scalar time_slice);
void
SetLastPerformance(const Time& when)
{Check(this); Check(&when); lastPerformance = when;}
const Time&
GetLastPerformance() const
{Check(this); return lastPerformance;}
void
RequestEncore(Encore encore);
virtual void
ReadUpdateRecord(UpdateRecord *message);
virtual void
WriteUpdateRecord(
UpdateRecord *message,
int update_model
);
void
WriteSimulationUpdate(MemoryStream *update_stream);
enum {
DefaultUpdateModelBit=0,
NextUpdateModelBit
};
enum {
DefaultUpdateModelFlag = 1<<DefaultUpdateModelBit
};
void
ForceUpdate(Word model=DefaultUpdateModelFlag)
{Check(this); updateModel |= model;}
protected:
Time
lastPerformance;
Time
lastUpdate;
Word
updateModel;
Performance
activePerformance;
//##########################################################################
// Flag Support
//
public:
enum {
DelayWatchersBit,
DontExecuteBit,
NextBit
};
enum {
DelayWatchersFlag = 1<<DelayWatchersBit,
DontExecuteFlag = 1<<DontExecuteBit
};
LWord simulationFlags;
void
SetWatcherDelay()
{Check(this); simulationFlags |= DelayWatchersFlag;}
void
ClearWatcherDelay()
{Check(this); simulationFlags &= ~DelayWatchersFlag;}
Logical
AreWatchersDelayed()
{Check(this); return (simulationFlags & DelayWatchersFlag) != 0;}
void
NeverExecute()
{Check(this); simulationFlags |= DontExecuteFlag;}
void
ExecuteOnUpdate()
{Check(this); simulationFlags |= DontExecuteFlag;}
void
AlwaysExecute()
{Check(this); simulationFlags &= ~DontExecuteFlag;}
Logical
IsReplicantExecutable()
{
Check(this);
return
(simulationFlags&DontExecuteFlag) == 0
|| lastUpdate >= lastPerformance;
}
Logical
IsNonReplicantExecutable()
{Check(this); return (simulationFlags&DontExecuteFlag) == 0;}
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// State support
//
public:
enum {
DefaultState = 0,
StateCount
};
unsigned
GetSimulationState()
{Check(this); return simulationState.GetState();}
unsigned
GetOldSimulationState()
{Check(this); return simulationState.GetOldState();}
void
SetSimulationState(unsigned new_state)
{Check(this); simulationState.SetState(new_state);}
StateIndicator
simulationState;
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Watcher Support
//
public:
void
AddAudioWatcher(Component *watcher)
{Check(&audioWatcherSocket);audioWatcherSocket.Add(watcher);}
void
AddVideoWatcher(Component *watcher)
{Check(&videoWatcherSocket);videoWatcherSocket.Add(watcher);}
void
AddGaugeWatcher(Component *watcher)
{Check(&gaugeWatcherSocket);gaugeWatcherSocket.Add(watcher);}
void
AddEffectWatcher(Component *watcher)
{Check(&effectWatcherSocket); effectWatcherSocket.Add(watcher);}
void
ExecuteWatchers();
private:
SChainOf<Component*>
audioWatcherSocket;
SChainOf<Component*>
videoWatcherSocket;
SChainOf<Component*>
gaugeWatcherSocket;
SChainOf<Component*>
effectWatcherSocket;
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Test Support
//
public:
Logical
TestInstance() const;
static Logical
TestClass();
};
//##########################################################################
//################### Simulation::IndexEntry #########################
//##########################################################################
struct Simulation__IndexEntry
{
Enumeration
entryID;
const char *
entryName;
Simulation::AttributePointer
entryAddress;
};
#define ATTRIBUTE_ENTRY(class,name,attribute)\
{\
class::name##AttributeID,\
#name,\
(Simulation::AttributePointer) &class::attribute\
}
//##########################################################################
//################# Simulation::AttributeIndexSet ####################
//##########################################################################
class Simulation__AttributeIndexSet:
public Receiver::InheritanceSet
{
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Construction and Destruction
//
public:
Simulation__AttributeIndexSet(
Simulation::AttributeID count,
const Simulation::IndexEntry index_table[],
const Simulation::AttributeIndexSet &inheritance
)
{Build(count, index_table, &inheritance);}
Simulation__AttributeIndexSet(
Simulation::AttributeID count,
const Simulation::IndexEntry index_table[]
)
{Build(count, index_table, NULL);}
Simulation__AttributeIndexSet()
{attributeIndex = NULL; entryCount = 0;}
~Simulation__AttributeIndexSet();
protected:
void
Build(
Simulation::AttributeID count,
const Simulation::IndexEntry index_table[],
const Simulation::AttributeIndexSet *inheritance
);
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// AttributeIndexSet Functionality
//
protected:
Simulation::IndexEntry
*attributeIndex;
public:
Simulation::AttributePointer
Find(Simulation::AttributeID attribute) const
{
Check(this);
Verify(attribute > 0);
if (attribute<=entryCount)
return attributeIndex[attribute-1].entryAddress;
else
return Simulation::NullAttribute;
}
Simulation::AttributePointer
Find(const char* attribute_name) const;
const Simulation::IndexEntry*
FindEntry(const char* attribute_name) const;
static const Simulation::AttributeIndexSet
NullSet;
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Test Support
//
public:
static Logical
TestClass();
};
//##########################################################################
//################### Simulation::SharedData #########################
//##########################################################################
class Simulation__SharedData:
public Receiver::SharedData
{
public:
Simulation__SharedData(
Derivation* derivation,
Receiver::MessageHandlerSet &message_handlers,
Simulation::AttributeIndexSet &attribute_index,
int state_count
):
Receiver::SharedData(derivation, message_handlers),
activeAttributeIndex(&attribute_index),
stateCount(state_count)
{}
Simulation::AttributeIndexSet* activeAttributeIndex;
int stateCount;
};
inline Simulation::SharedData*
Simulation::GetSharedData()
{return Cast_Object(SharedData*,sharedData);}