test(render): add the Campaign V offline pixel gate
Seven of the campaign slices (V2, V4a-V4g) are renderer ports whose entire acceptance criterion is that no pixel changed, and the existing connected lifecycle route needs both a live ACE server and the user watching. That would have made the campaign advance only when someone is at the keyboard. The client renders the world from the DATs without ACDREAM_LIVE, so the existing UI automation probe can capture a settled frame with no session created and no ACE state to disturb. The gate wraps that: capture at the parent commit, capture at slice HEAD, compare through the existing compare-screenshots CLI at the project's pinned tolerance 2 / 0.001. Determinism was measured rather than assumed, and the first measurement failed: two captures at the same commit differed in 0.29% of pixels. The differences were confined to the top ~180 rows, which is correct behavior rather than a bug - the sky animates and the Dereth clock advances with wall time, so two launches cannot agree there. Below the horizon everything was stable. Masking the top 280 rows brings two independent same-commit pairs to 15 and 17 differing pixels out of 563,200 compared, a ~33x margin under the threshold. Masking the animated band keeps the rest a strict identity check; relaxing the tolerance instead would have hidden real regressions everywhere else. Covers terrain and blending, scenery, static meshes, water, fog, and the whole retained UI. Does not cover sky (masked), EnvCell interiors, particles, or the paperdoll viewports, since the offline scene is a fixed outdoor view - so V4e, V4f and V4g keep a user visual gate on top of this one. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
4f94ad7ddd
commit
86afbe2ecd
2 changed files with 273 additions and 4 deletions
|
|
@ -391,10 +391,45 @@ a loud `Debug.Assert` in Debug builds. A Debug run therefore shows one failure
|
|||
that is neither a regression nor yours. The V0 baseline is **3,785 passed /
|
||||
3 skipped** in Release (3,763 pre-campaign plus 22 contract tests).
|
||||
|
||||
"Pixel gate" means: capture the deterministic checkpoints from the parent
|
||||
commit's build, capture again at slice HEAD, compare with the
|
||||
`compare-screenshots` CLI at tolerance 2 / fraction 0.001, MSAA off,
|
||||
`ACDREAM_DAY_GROUP` pinned.
|
||||
### 5.1 The offline pixel gate
|
||||
|
||||
"Pixel gate" means `tools/run-offline-pixel-gate.ps1`: capture at the parent
|
||||
commit, capture at slice HEAD, compare with the `compare-screenshots` CLI at
|
||||
tolerance 2 / fraction 0.001.
|
||||
|
||||
The client is launched **without `ACDREAM_LIVE`**, so it renders the world
|
||||
straight from the DATs. No session is created and no ACE state can be disturbed,
|
||||
which means this gate runs unattended — it needs neither the live server nor the
|
||||
user. That matters: seven slices (V2, V4a–V4g) are renderer ports whose whole
|
||||
acceptance criterion is "no pixel changed."
|
||||
|
||||
```
|
||||
tools/run-offline-pixel-gate.ps1 -Out artifacts/gate-base # at the parent commit
|
||||
tools/run-offline-pixel-gate.ps1 -Out artifacts/gate-head -Baseline artifacts/gate-base
|
||||
```
|
||||
|
||||
**Determinism was measured, not assumed.** Two captures at the same commit
|
||||
initially differed in 0.29% of pixels — far above the 0.001 threshold. The
|
||||
differences were confined to the top ~180 rows: the sky animates (clouds scroll,
|
||||
the sun moves) and the Dereth clock advances with wall time, so two launches can
|
||||
never agree there. Everything below the horizon was stable. With the top 280 rows
|
||||
masked, two independent same-commit pairs differ by **15 and 17 pixels out of
|
||||
563,200 compared** — a fraction of 0.000027, roughly a 33× margin under the
|
||||
threshold. The gate is a strict identity check on everything it covers, rather
|
||||
than a loose tolerance that would hide real regressions.
|
||||
|
||||
**Coverage.** Terrain and terrain blending, scenery, static world meshes, water,
|
||||
fog, and the entire retained UI (vitals, spell bar, toolbar, chat, radar).
|
||||
|
||||
**Not covered — these still need a user visual gate:** sky (masked), EnvCell
|
||||
interiors, particles, and the paperdoll/appraisal viewports, because the offline
|
||||
scene is a fixed outdoor view with no camera control. Slices **V4e** (particles),
|
||||
**V4f** (sky), and **V4g** (viewports) therefore carry a user gate in addition to
|
||||
their automated one.
|
||||
|
||||
MSAA is left at the quality preset for GL-versus-GL self-differentials, where it
|
||||
is deterministic. The V7 GL-versus-Vulkan differential must force MSAA off,
|
||||
because sample positions are not specified across implementations.
|
||||
|
||||
| Slice | Scope | Gate |
|
||||
|---|---|---|
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue