Files
RP412/MUNGA/NETWORK.h
T
CydandClaude Fable 5 7d485c9672 The wire keeps what it could not send
Cyd asked for an analysis of the networking stack and what would make the
simulation feel better over the internet. The analysis found something
more urgent than latency: the transport has been losing data silently
since the arcade, and nothing in the game could see it happen.

Every send result was discarded - L4NET, the console, all of it. On the
1ms arcade LAN the socket buffer never filled, so it never mattered. Over
the internet it matters twice. A peer stalled in its own 10-30 second
mission load stops reading, its window closes, and our nonblocking send
starts answering would-block, which threw the message away; or worse,
answering a PARTIAL count, and since framing on that stream is recovered
purely from each message's length prefix, the bytes that never followed
sheared it for good. Both are reachable in an ordinary race, because
every race has a load in it.

So sends go through a bounded per-connection queue now. What the wire
will not take is kept, byte-exact, and retried at three flush points -
before the render (the present blocks on vsync, and this frame's state
should be travelling while it does), at the top of the receive pump, and
before a connect sequence. Nothing is ever dropped from the middle: these
are reliable ordered messages carrying entity creation, damage and race
control, so a queue that overflows its 256K declares the connection dead
and lets the disconnect path run rather than quietly desyncing the
stream. RP412NETSENDQ=0 restores the old behaviour and still logs what it
would have lost, which is the honest way to A/B it. On Steam the same
queue finally surfaces k_EResultLimitExceeded, which the old code
collapsed into -1 and discarded - that was backpressure, unlogged.

The receive side gained the check the release build never had. The length
prefix is untrusted input; Verify() compiles away in release, so a
corrupt one went to memmove as a negative, or copied 4096 bytes of
assembled packet into a 1600-byte stack buffer, or named a size the pad
could never complete and wedged the connection forever. It is now
validated against the same bounds the sender works to, and a stream that
fails them is dropped like any other lost peer.

And a fry that never ends: drop zones are map entities dealt round-robin
at load, ownership transfer is not implemented, so a leaver's pads stay
in the DropZones group. The respawn request dispatched to one goes to a
host that is gone - dropped at the send, the 'no host N in the table'
path - and the two-second retry re-dispatches to the same dead owner
forever. The pad scan now skips zones whose owner has left, and
re-validates one assigned earlier before reusing it.

The rest is measurement, because the symptoms this work exists to chase
are all reported in prose and none of them are in any log. Sixteen logs
from the six-player night contain zero player-facing latency lines. A
race now ends with a NetLog summary: per remote pod, how many updates
arrived and how evenly (median and p95 out of a log2 histogram), the
widest gap, how many gaps were long enough to mean a quiet sender versus
short enough to mean OUR loop stalled, how often its motion snapped
instead of blending, and how far arriving updates moved it. Per peer,
whether the clock alignment ever had to step mid-race - which is the
input for deciding if it needs slewing, rather than guessing. The mission
t0 tick goes in the log too, alongside the console's per-pod RunMission
send ticks, because nothing has ever measured how far apart the machines
actually start; the clockwork doors inherit that skew directly.

RP412NETSTATS adds the transport's own view - per connection: messages,
bytes, wire writes, partials, refusals, how much sat queued - and on
Steam the first read this codebase has ever taken of GetConnectionRealTime
Status. Ping, quality, pending and unacked bytes, and one route
description per connection at teardown. The API was vendored and never
called; there was no RTT number anywhere in the game.

Finally, rpl4opt -spoolstats reads any recording offline. The data was
already in every spool ever made and nothing read it that way: the
recorder restamps each packet with local arrival time while the update
records inside keep the sender's sim-grid stamp, so the difference is
clock offset plus one-way delay, and the same running-minimum estimator
the game runs live separates them. It prints delay above the per-host
minimum, and decomposes each entity's gaps into sender pacing versus
delivery jitter - which no live counter can do. It lives in the game exe
rather than RPL4TOOL because the tool is deliberately not /Zp1 and would
misread every struct in the file.

Verified on the two-pod loopback harness: mesh up, egg fed, 60s raced,
stopped on command, scores collected, and both summaries reading exactly
what a pair of PARKED pods should read - heartbeat cadence, one snap per
heartbeat, sub-quarter-metre corrections, no clock steps. The t0 ticks
and the netclock offsets agree with each other to the two seconds the
pods launched apart.

The latency tier is deliberately NOT here. TCP_NODELAY, the Steam
NoNagle flag, per-frame coalescing and the pre-sim receive drain are all
scoped and all wait on this build's numbers, because the point of
shipping measurement first is to find out whether the thing we would fix
is the thing that hurts. Nagle is still on. Interest management is still
inert. The wire format is untouched, so this build and the last one still
race each other.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 12:59:25 -05:00

292 lines
5.6 KiB
C++

#pragma once
#include "receiver.h"
#include "time.h"
#include "hostid.h"
//WinSock support :ADB 01/06/07
#include <Winsock2.h>
#include <Ws2tcpip.h>
class Mission;
class NetworkManager;
class NotationFile;
//
//---------------------------------------------------
// Support types for interest manager
//---------------------------------------------------
//
typedef Enumeration InterestZoneID;
const InterestZoneID NullInterestZoneID = 0;
//~~~~~~~~~~~~~~~~~~~~~~~~~~ NetworkAddress ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
typedef LWord NetworkAddress;
extern const NetworkAddress NullNetworkAddress;
//~~~~~~~~~~~~~~~~~~~~~~~~~~~ NetworkClient ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
class NetworkPacket;
class NetworkClient:
public Receiver
{
friend class NetworkManager;
//##########################################################################
// Shared Data support
//
public:
static Derivation *GetClassDerivations();
static SharedData DefaultData;
//##########################################################################
// Construction and destruction support
//
public:
enum ClientID {
NetworkManagerClientID = 0,
EntityManagerClientID = 1,
HostManagerClientID = 2,
InterestManagerClientID = 3,
ApplicationClientID = 4,
ConsoleClientID = 5,
IcomManagerClientID = 6
};
private:
ClientID
clientID;
protected:
NetworkClient(
ClassID class_ID,
SharedData &virtual_data,
ClientID clientID
);
public:
~NetworkClient();
Logical
TestInstance() const;
//##########################################################################
// Message packet receiving
//
public:
virtual void
ReceiveNetworkPacket(
NetworkPacket *packet,
Receiver::Message *packet_message
);
};
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Network ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#define NETWORKMANAGER_BUFFER_SIZE 1600
class NetworkManager__ReceiveEggFileMessage;
class NetworkManager:
public NetworkClient
{
//##########################################################################
// Shared Data support
//
public:
static Derivation *GetClassDerivations();
static SharedData DefaultData;
public:
typedef Enumeration GameID;
//WinSock support :ADB 01/06/07
//const NetworkAddress&
// GetAddress()
NetworkAddress* GetAddress()
{return addresses;}
NetworkManager(SharedData &shared_data);
~NetworkManager();
void
SetGameID(GameID new_id)
{gameID = new_id;}
virtual void
Send(
Message *what,
ClientID to,
HostID host_id
);
void
Broadcast(
Message *what,
ClientID to
);
virtual void
ExclusiveBroadcast(
Message *what,
ClientID to
);
Logical
RoutePacket();
virtual Logical
ExecuteBackground();
//
// Retry sends a connection could not take earlier. The application
// calls this at its flush points; the base engine has no queues, so
// the default is a no-op (L4 forwards it to the wire transport).
//
virtual void
FlushSends()
{
}
NetworkClient*
GetNetworkClientPointer(ClientID client_id);
virtual void
StartConnecting(Mission *mission);
virtual Logical
Shutdown();
virtual void
Marker(char *) {};
enum NetworkMode
{
ReliableMode,
UnreliableMode
};
virtual void
Mode(NetworkMode) {};
// protected: //GY for test only
virtual Logical
CheckBuffers(NetworkPacket*);
virtual void
RemovePacket(NetworkPacket *packet);
GameID
gameID;
//WinSock support :ADB 01/06/07
//NetworkAddress address;
NetworkAddress* addresses;
int num_addresses;
//struct addrinfo* address;
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Message Support
//
public:
//
// Message IDs
//
enum
{
ReceiveEggFileMessageID = NetworkClient::NextMessageID,
NextMessageID
};
//
// Message types
//
typedef NetworkManager__ReceiveEggFileMessage
ReceiveEggFileMessage;
//
// Message table
//
static const HandlerEntry
MessageHandlerEntries[];
//static MessageHandlerSet MessageHandlers;
static MessageHandlerSet& GetMessageHandlers();
void
ReceiveEggFileMessageHandler(ReceiveEggFileMessage* EggMessage);
protected:
char
*eggTempBuffer;
NotationFile
*networkEggNotationFile;
long
eggTempNext;
};
//~~~~~~~~~~~~~~~~~~~~~~~~~~ NetworkManager messages ~~~~~~~~~~~~~~~~~~~~~~~~~
// NetworkManager__ReceiveEggFileMessage
// NOTE: this message should be setup as variable length rather than fixing the
// size at 1000, I wasn't sure how to do this and there wasn't anyone around
// to ask at the time so I did it this way temporarily.
//
class NetworkManager__ReceiveEggFileMessage:
public Receiver__Message
{
public:
NetworkManager__ReceiveEggFileMessage(
int sequence_number,
int total_file_length,
char* notation_data,
int length
):
Receiver__Message(
NetworkManager::ReceiveEggFileMessageID,
sizeof(NetworkManager__ReceiveEggFileMessage)
),
sequenceNumber(sequence_number),
notationFileLength(total_file_length),
thisMessageLength(length)
{Mem_Copy(notationData,notation_data,length,sizeof(notationData));}
int
sequenceNumber;
int
notationFileLength;
int
thisMessageLength;
char
notationData[1000];
};
//~~~~~~~~~~~~~~~~~~~~~~~~ NetworkPacketHeader ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
class NetworkPacketHeader
{
public:
NetworkClient::ClientID
clientID;
NetworkManager::GameID
gameID;
HostID
fromHost;
Time
timeStamp;
};
//~~~~~~~~~~~~~~~~~~~~~~~~~~~ NetworkPacket ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
class NetworkPacket:
public NetworkPacketHeader
{
public:
Receiver::Message
messageData;
};
inline bool operator==(SOCKADDR_IN &address1, SOCKADDR_IN &address2)
{
return (address1.sin_family == address2.sin_family &&
address1.sin_addr.S_un.S_addr == address2.sin_addr.S_un.S_addr &&
address1.sin_port == address2.sin_port);
}