Commit Graph
1 Commits
Author SHA1 Message Date
Joe DiPrimaandClaude Opus 5 c791fae0f8 KB: new topic test-harness.md -- the bench machinery + the verification doctrine
User mandate (2026-08-02): "do tests like this from now on" -- after the #110
grind bench, where the field composition (peer fire destroying an arm over the
wire) replaced the constructed proxy that had let #86 be called fixed while
players kept hitting it.

The topic carries two halves on purpose:

DOCTRINE -- what counts as VERIFIED:
  1. reproduce the REPORTED scenario, not a convenient adjacent one
  2. scalpel hooks (BT_SELF_DAMAGE_ZONE / BT_KILL_SUBSYS / BT_FORCE_*) locate
     defects; they support a "fixed" claim only with proven path-identity to
     the field composition -- and the field composition still gets run
  3. MP symptom -> two-node proof (master-side correctness says nothing about
     what a peer sees)
  4. visual symptom -> pixel proof (gotcha 23)
  5. coverage claims need the axis enumerated and measured (all gates, all
     chassis), because per-chassis behaviour lives in authored data
  6. an unexplained extra effect in a passing run means the run has not passed

MACHINERY -- the bench_common.sh contract (summarized, file = source of truth),
single-node and two-node skeletons (relay, ports, affinity, fire cadence,
GOTO_STOP standoff), process hygiene (stale-node taskkill first, never
double-background, teardown kill order, stale-exe tells), and log-reading rules
(capped diagnostics are not evidence of absence; alarm lines are not trends;
name the actor at every refusal; field logs have no gates set -- spawn-time
summaries ungated, per-frame traces gated).

Routed: Quick Lookup row, CLAUDE.md reasoning step 4, build-and-run parity
section, reconstruction-method Key Relationships.  checkctx CLEAN.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 09:17:01 -05:00