test(render): add the Campaign V offline pixel gate

Seven of the campaign slices (V2, V4a-V4g) are renderer ports whose entire
acceptance criterion is that no pixel changed, and the existing connected
lifecycle route needs both a live ACE server and the user watching. That would
have made the campaign advance only when someone is at the keyboard.

The client renders the world from the DATs without ACDREAM_LIVE, so the existing
UI automation probe can capture a settled frame with no session created and no
ACE state to disturb. The gate wraps that: capture at the parent commit, capture
at slice HEAD, compare through the existing compare-screenshots CLI at the
project's pinned tolerance 2 / 0.001.

Determinism was measured rather than assumed, and the first measurement failed:
two captures at the same commit differed in 0.29% of pixels. The differences
were confined to the top ~180 rows, which is correct behavior rather than a bug
- the sky animates and the Dereth clock advances with wall time, so two launches
cannot agree there. Below the horizon everything was stable. Masking the top 280
rows brings two independent same-commit pairs to 15 and 17 differing pixels out
of 563,200 compared, a ~33x margin under the threshold. Masking the animated
band keeps the rest a strict identity check; relaxing the tolerance instead
would have hidden real regressions everywhere else.

Covers terrain and blending, scenery, static meshes, water, fog, and the whole
retained UI. Does not cover sky (masked), EnvCell interiors, particles, or the
paperdoll viewports, since the offline scene is a fixed outdoor view - so V4e,
V4f and V4g keep a user visual gate on top of this one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Erik 2026-07-27 15:19:43 +02:00
parent 4f94ad7ddd
commit 86afbe2ecd
2 changed files with 273 additions and 4 deletions

View file

@ -391,10 +391,45 @@ a loud `Debug.Assert` in Debug builds. A Debug run therefore shows one failure
that is neither a regression nor yours. The V0 baseline is **3,785 passed /
3 skipped** in Release (3,763 pre-campaign plus 22 contract tests).
"Pixel gate" means: capture the deterministic checkpoints from the parent
commit's build, capture again at slice HEAD, compare with the
`compare-screenshots` CLI at tolerance 2 / fraction 0.001, MSAA off,
`ACDREAM_DAY_GROUP` pinned.
### 5.1 The offline pixel gate
"Pixel gate" means `tools/run-offline-pixel-gate.ps1`: capture at the parent
commit, capture at slice HEAD, compare with the `compare-screenshots` CLI at
tolerance 2 / fraction 0.001.
The client is launched **without `ACDREAM_LIVE`**, so it renders the world
straight from the DATs. No session is created and no ACE state can be disturbed,
which means this gate runs unattended — it needs neither the live server nor the
user. That matters: seven slices (V2, V4aV4g) are renderer ports whose whole
acceptance criterion is "no pixel changed."
```
tools/run-offline-pixel-gate.ps1 -Out artifacts/gate-base # at the parent commit
tools/run-offline-pixel-gate.ps1 -Out artifacts/gate-head -Baseline artifacts/gate-base
```
**Determinism was measured, not assumed.** Two captures at the same commit
initially differed in 0.29% of pixels — far above the 0.001 threshold. The
differences were confined to the top ~180 rows: the sky animates (clouds scroll,
the sun moves) and the Dereth clock advances with wall time, so two launches can
never agree there. Everything below the horizon was stable. With the top 280 rows
masked, two independent same-commit pairs differ by **15 and 17 pixels out of
563,200 compared** — a fraction of 0.000027, roughly a 33× margin under the
threshold. The gate is a strict identity check on everything it covers, rather
than a loose tolerance that would hide real regressions.
**Coverage.** Terrain and terrain blending, scenery, static world meshes, water,
fog, and the entire retained UI (vitals, spell bar, toolbar, chat, radar).
**Not covered — these still need a user visual gate:** sky (masked), EnvCell
interiors, particles, and the paperdoll/appraisal viewports, because the offline
scene is a fixed outdoor view with no camera control. Slices **V4e** (particles),
**V4f** (sky), and **V4g** (viewports) therefore carry a user gate in addition to
their automated one.
MSAA is left at the quality preset for GL-versus-GL self-differentials, where it
is deterministic. The V7 GL-versus-Vulkan differential must force MSAA off,
because sample positions are not specified across implementations.
| Slice | Scope | Gate |
|---|---|---|