acdream/tools/run-offline-pixel-gate.ps1
Erik 122fe8a7e2 feat(render): Campaign V slice V10 — Vulkan becomes the default backend
THIS CUTOVER AWAITS THE USER'S VISUAL SIGN-OFF. It is not complete. Section 7
of the campaign plan names the V10 sign-off as the only required user stop
besides gate failures, and it has not been given. This commit flips the default
and runs the battery so that the sign-off has evidence in front of it.

ROLLBACK, one line: `git revert` of this commit. It restores the GL default,
the pre-V10 escape-hatch polarity and the gate scripts' inherited backend
together; nothing else has to move with it.

An unset, empty or unrecognised ACDREAM_RENDER_BACKEND now yields
RenderBackendKind.Vulkan. Only `gl` or `opengl`, case-insensitive, selects
OpenGL. The polarity of the typo case flipped with the default and on purpose:
before V10 an unrecognised token had to land on GL because Vulkan was dark and a
typo must never silently start a backend that cannot draw; after V10 it has to
land on Vulkan for the same reason read the other way, because GL is the backend
V11 deletes. `opengl` is honoured beside `gl` because an escape hatch exists to
be found.

Three gate scripts follow the flip. run-offline-pixel-gate.ps1 gains -Backend
(default vulkan) and now FORCES all four determinism levers — backend, day
group, world day fraction, sky phase — plus ACDREAM_MSAA_SAMPLES=0, instead of
inheriting any of them. run-repeat-connected-gate.ps1 and
run-connected-world-lifecycle-gate.ps1 CLEAR ACDREAM_RENDER_BACKEND rather than
setting it, so what they exercise is the process default and an ambient override
in a caller's shell cannot make a GL run wear the default's report.

TEST PIN UPDATED, flagged as required: RenderBackend_DefaultsToGl becomes
RenderBackend_DefaultsToVulkan, and RenderBackend_AnythingElseStaysOnGl splits
into RenderBackend_SelectsGlOnlyForTheEscapeHatchTokens and
RenderBackend_AnythingElseStaysOnVulkan. Five cases replace two. No other test
is touched, weakened or deleted.

AD-46's divergence-register row moves from "dormant until the V10 cutover" to
live, in this commit, per the same-commit register rule.

Battery, all on the new default:

  complete Release suite    9,222 passed / 5 skipped / 0 failed (9 projects)
                            +5 against the pre-flip 9,217; the +5 are this
                            slice's own escape-hatch cases
  #250 family, singly       4/4 pass (none failed in the whole-suite run)
  repeat connected gate     PASS 3/3 on both columns
  world-lifecycle route     PASS, 0 failures, both sessions graceful at exit 0
  validation layer          inserted at instance AND device level by the loader,
                            zero errors and zero warnings, real frame captured
  GL escape hatch           verified by two offline launches: 4.3.0 Core Profile
                            Context, bindless present, exit 0

Every connected launch in the battery reached Vulkan with no environment
variable set, which is the flip itself under test rather than an assertion
about it.

THE PIXEL GATE IS NOT MET, AND WAS NOT RELAXED. Vulkan against a GL-era capture
taken at this commit through the escape hatch, MSAA off and both clocks pinned:
1.099e-03 masked / 3.764e-02 whole-frame, against a 0.001 threshold. 97.9% of
the difference is in the treeline band, and the masked residual of 619 px — set
against a same-backend control of 10 px — sits entirely on the silhouettes of
distant alpha-blended scenery. That is AD-46's registered population; section
5.5.19 measured the same quantity at 497 px / 8.8e-04. Below the band the two
backends are photometrically identical: mean luminance differs by 0.01 of 255.
No baseline was regenerated and no mask or tolerance was widened.

Two instrument findings are recorded in section 5.5.23. The offline gate's sky
mask is still load-bearing — this slice tried retiring it on the reasoning that
V7's clock pins had made it obsolete, and the control refuted that: two launches
of the same binary still differ by 1,011 px on GL and 482 px on Vulkan, almost
all of it in the band. The default went back to 280 with the measurement written
into the script's help. And the repeat gate's desktop witness needs an
uncontested primary monitor: a first attempt reported 1/3, and the two failing
grabs turn out to be a web browser and Discord composited over the client rect,
not a blank frame — the client's Vulkan capture rendered in all six runs.

Nothing GL, ImGui or Studio is deleted. That is V11's scope and it is untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 22:32:25 +02:00

330 lines
14 KiB
PowerShell

<#
.SYNOPSIS
Campaign V self-differential pixel gate. Captures deterministic offline
screenshots and optionally compares them against a baseline capture.
.DESCRIPTION
Every Campaign V renderer-port slice must prove it changed no pixels. The
connected lifecycle gate needs a live ACE server and the user's attention;
this gate needs neither. The client is launched WITHOUT ACDREAM_LIVE, so it
renders the world straight from the DATs, no session is created, and no ACE
state can be disturbed.
Coverage: terrain, terrain blending, scenery, static world meshes, sky and
fog, and the full retained UI (vitals, spell bar, toolbar, chat, radar).
It does NOT cover EnvCell interiors, particles, or the paperdoll viewport —
those still need the connected route, so slices touching them (V4e, V4g)
must additionally be gated by the user.
Determinism levers, all four forced here rather than left to the ambient
environment, because a capture that inherits any of them is not comparable
against one that did not:
* ACDREAM_RENDER_BACKEND -- see -Backend. Since slice V10 flipped the
process default, "whatever the environment says" is no longer a stable
description of what got captured.
* ACDREAM_DAY_GROUP -- the sky day group, so weather and the keyframe
do not drift between runs.
* ACDREAM_WORLD_TIME -- the Dereth day fraction. This moves the SUN, so
it moves every lit surface, not just the sky. Slice V7 measured 22.3% of
the frame differing between two captures 45 s apart at one stop on one
backend with this unpinned.
* ACDREAM_SKY_PHASE_SECONDS -- the cloud sheet's UV scroll, which runs on
wall time and is independent of the world clock by design.
ACDREAM_MSAA_SAMPLES is forced to 0 for the same reason the backend
differential forces it: multisample resolve positions are explicitly
unspecified across implementations, and at 4x they swamp the threshold
ninety-fold with two drivers' sample patterns. See plan section 5.5.16.
A fixed warm-up lets streaming settle before the frame is captured.
.PARAMETER Out
Directory to write this capture into. Created if absent, emptied if present.
.PARAMETER Baseline
Optional. A directory from a previous capture. When supplied, every PNG is
compared against its namesake and the script fails on any mismatch.
.PARAMETER Backend
Which backend to capture with. DEFAULT 'vulkan', following slice V10's
cutover: the gate must capture what the client actually ships.
Pass 'gl' to capture through the escape hatch -- which is also how this
script produces the GL-era left-hand side of a cross-backend comparison, and
how the escape hatch itself gets verified. Pass '' to let the process default
decide, which is only useful for proving what that default is.
.PARAMETER WarmupMs
Milliseconds to let the world stream and settle before capturing. Default
12000, which reliably reaches a fully populated view on the reference machine.
.PARAMETER DayGroup
Sky day-group index to pin. Default 0.
.PARAMETER Tolerance
Per-channel absolute difference allowed. Default 2, matching the project's
pinned screenshot rule.
.PARAMETER MaxDifferentFraction
Fraction of differing pixels allowed. Default 0.001, likewise pinned.
.PARAMETER WorldDayFraction
Dereth day fraction pinned on the launch, in [0, 1). Default 0.5 -- noon, the
brightest and flattest lighting the day has, and the value the backend
differential pins.
.PARAMETER SkyPhaseSeconds
Elapsed-seconds value the sky's UV scroll is pinned to. Default 0, the cloud
sheet's authored origin.
.PARAMETER MsaaSamples
MSAA sample count forced on the launch. Default 0 -- off, which is what plan
section 2's pixel-acceptance contract specifies. Pass -1 to leave the quality
preset alone.
.PARAMETER MaskTopPixels
Height in pixels of the sky and treeline band excluded from comparison.
Default 280.
This is not a fudge factor, and slice V10 re-measured it rather than assuming
the V7 clock pins had retired it. Two captures at the SAME commit once
differed in 0.29% of pixels, essentially all of it in the top rows, because
the sky legitimately animates: clouds scroll on wall time and the Dereth
clock moves the sun, and therefore every lit surface. Pinning both clocks --
which this script now does -- shrinks that but does NOT remove it. Measured
at V10, two launches of the same binary with all four levers pinned:
whole frame, GL vs GL 1,011 px of 921,600 (1.10e-03)
whole frame, VK vs VK 482 px of 921,600 (5.23e-04)
top 280 masked, GL vs GL 10 px of 563,200 (1.78e-05)
top 280 masked, VK vs VK 8 px of 563,200 (1.42e-05)
So the band still moves between launches on both backends -- enough that an
unmasked self-differential on GL fails the 0.001 threshold on noise alone --
and everything below it is bit-stable to about ten pixels. Masking the band
is what makes the rest a strict identity check instead of forcing a loose
tolerance that would hide real regressions everywhere else.
Consequence, unchanged: this gate does NOT cover sky rendering.
Do NOT reach for a LARGER value to make a failing gate pass: plan section 7.1
rule 2. Its legitimate use is diagnostic -- reporting a masked/unmasked split
so a known, registered band (for instance AD-46's treeline) can be quantified
separately from the rest of the frame.
.PARAMETER SkipBuild
Skip the Release build (use when the caller already built).
.EXAMPLE
# Capture a baseline at the parent commit, then gate the slice:
git stash
tools/run-offline-pixel-gate.ps1 -Out artifacts/gate-base
git stash pop
tools/run-offline-pixel-gate.ps1 -Out artifacts/gate-head -Baseline artifacts/gate-base
.EXAMPLE
# Cross-backend: the GL-era capture, then Vulkan against it.
tools/run-offline-pixel-gate.ps1 -Out artifacts/v10-gl -Backend gl
tools/run-offline-pixel-gate.ps1 -Out artifacts/v10-vk -Baseline artifacts/v10-gl
#>
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)][string]$Out,
[string]$Baseline,
[string]$Backend = 'vulkan',
[int]$WarmupMs = 12000,
[int]$DayGroup = 0,
[double]$WorldDayFraction = 0.5,
[double]$SkyPhaseSeconds = 0,
[int]$MsaaSamples = 0,
[int]$Tolerance = 2,
[double]$MaxDifferentFraction = 0.001,
[int]$MaskTopPixels = 280,
[switch]$SkipBuild
)
$ErrorActionPreference = 'Stop'
$repo = Split-Path -Parent $PSScriptRoot
$exe = Join-Path $repo 'src\AcDream.App\bin\Release\net10.0\AcDream.App.exe'
$cli = Join-Path $repo 'src\AcDream.Cli\bin\Release\net10.0\AcDream.Cli.dll'
function Write-Step($message) { Write-Host "[pixel-gate] $message" }
# --- 1. Build -----------------------------------------------------------------
if (-not $SkipBuild) {
Write-Step 'building Release'
& dotnet build (Join-Path $repo 'AcDream.slnx') -c Release --nologo -v q | Out-Null
if ($LASTEXITCODE -ne 0) { throw "Release build failed with exit code $LASTEXITCODE." }
}
if (-not (Test-Path $exe)) { throw "Client not found at $exe. Build Release first." }
# --- 2. Prepare the capture directory and probe script ------------------------
if (Test-Path $Out) { Remove-Item -Recurse -Force $Out }
New-Item -ItemType Directory -Force -Path $Out | Out-Null
$probe = Join-Path $Out 'offline.probe.txt'
# The script runner reads one command per line. A single settled capture is the
# whole gate: a second stop would need camera movement, which offline has no
# deterministic way to drive.
Set-Content -Encoding utf8 -Path $probe -Value @"
sleep $WarmupMs
screenshot world-offline 30000
sleep 500
"@
$log = Join-Path $Out 'client.log'
# --- 3. Launch offline --------------------------------------------------------
$previousLive = $env:ACDREAM_LIVE
Remove-Item Env:\ACDREAM_LIVE -ErrorAction SilentlyContinue
$env:ACDREAM_DAT_DIR = Join-Path $env:USERPROFILE "Documents\Asheron's Call"
$env:ACDREAM_NO_AUDIO = '1'
$env:ACDREAM_RETAIL_UI = '1'
$env:ACDREAM_DAY_GROUP = "$DayGroup"
$env:ACDREAM_UI_PROBE_SCRIPT = $probe
$env:ACDREAM_AUTOMATION_ARTIFACT_DIR = $Out
# The determinism pins, forced rather than inherited. See .DESCRIPTION.
$invariant = [System.Globalization.CultureInfo]::InvariantCulture
$env:ACDREAM_WORLD_TIME = $WorldDayFraction.ToString($invariant)
$env:ACDREAM_SKY_PHASE_SECONDS = $SkyPhaseSeconds.ToString($invariant)
if ($MsaaSamples -ge 0) { $env:ACDREAM_MSAA_SAMPLES = "$MsaaSamples" }
else { Remove-Item Env:\ACDREAM_MSAA_SAMPLES -ErrorAction SilentlyContinue }
if ($Backend) { $env:ACDREAM_RENDER_BACKEND = $Backend }
else { Remove-Item Env:\ACDREAM_RENDER_BACKEND -ErrorAction SilentlyContinue }
$backendLabel = if ($Backend) { $Backend } else { 'process default' }
Write-Step "launching offline client (backend $backendLabel, warmup ${WarmupMs}ms, day group $DayGroup, day fraction $WorldDayFraction, sky phase $SkyPhaseSeconds, MSAA $MsaaSamples)"
$proc = Start-Process -FilePath $exe -RedirectStandardOutput $log `
-RedirectStandardError "$log.err" -PassThru -WindowStyle Minimized
try {
$shots = Join-Path $Out 'screenshots'
$deadline = (Get-Date).AddMilliseconds($WarmupMs + 60000)
$captured = $false
while ((Get-Date) -lt $deadline) {
if ((Test-Path $shots) -and (Get-ChildItem $shots -Filter *.png -ErrorAction SilentlyContinue)) {
$captured = $true
break
}
if ($proc.HasExited) { break }
Start-Sleep -Milliseconds 1000
}
if (-not $captured) {
Write-Host (Get-Content $log -Tail 30 -ErrorAction SilentlyContinue)
throw 'No screenshot was captured before the deadline.'
}
# Let the probe script finish its trailing sleep so the PNG is fully flushed.
Start-Sleep -Milliseconds 1500
}
finally {
# Graceful close: WM_CLOSE runs the shutdown path, so the ownership ledger
# converges the way the lifecycle tests expect. No ACE session exists here,
# but keeping the habit means this script is safe to point at a live run too.
$app = Get-Process -Name AcDream.App -ErrorAction SilentlyContinue
if ($app) {
$app.CloseMainWindow() | Out-Null
if (-not $app.WaitForExit(10000)) {
Write-Step 'WM_CLOSE timed out; forcing'
$app | Stop-Process -Force
}
}
Remove-Item Env:\ACDREAM_RENDER_BACKEND -ErrorAction SilentlyContinue
Remove-Item Env:\ACDREAM_MSAA_SAMPLES -ErrorAction SilentlyContinue
Remove-Item Env:\ACDREAM_WORLD_TIME -ErrorAction SilentlyContinue
Remove-Item Env:\ACDREAM_SKY_PHASE_SECONDS -ErrorAction SilentlyContinue
if ($previousLive) { $env:ACDREAM_LIVE = $previousLive }
}
$captures = Get-ChildItem (Join-Path $Out 'screenshots') -Filter *.png
Write-Step "captured $($captures.Count) screenshot(s) into $Out"
# The offline window is minimised but still focusable, so a stray scroll or key
# press from whoever is at the keyboard can move the camera mid-capture. That
# produces two screenshots of the same scene from different camera positions —
# a huge, entirely spurious pixel difference. This actually happened during
# slice V4b. Detect it and fail loudly rather than let a comparison be
# interpreted as a rendering regression.
$cameraInput = Select-String -Path $log -Pattern 'ScrollUp|ScrollDown|ZoomIn|ZoomOut|CameraZoom' `
-CaseSensitive -ErrorAction SilentlyContinue
if ($cameraInput) {
Write-Host ''
Write-Host '[pixel-gate] ABORTED: camera-affecting input reached the capture window.' -ForegroundColor Red
Write-Host ' The capture is not comparable. Re-run without touching the machine.' -ForegroundColor Red
$cameraInput | Select-Object -First 5 | ForEach-Object { Write-Host " $($_.Line)" -ForegroundColor Red }
exit 2
}
# --- 4. Compare against the baseline -----------------------------------------
if (-not $Baseline) {
Write-Step 'no baseline supplied; capture only'
exit 0
}
if (-not (Test-Path $cli)) { throw "AcDream.Cli not found at $cli." }
# Build the sky mask. The comparer skips any pixel whose mask alpha is non-zero,
# so the band we exclude is opaque and everything compared is transparent.
$maskPath = $null
if ($MaskTopPixels -gt 0) {
Add-Type -AssemblyName System.Drawing
$probeImage = [System.Drawing.Bitmap]::FromFile($captures[0].FullName)
$width = $probeImage.Width
$height = $probeImage.Height
$probeImage.Dispose()
$mask = New-Object System.Drawing.Bitmap($width, $height, [System.Drawing.Imaging.PixelFormat]::Format32bppArgb)
$graphics = [System.Drawing.Graphics]::FromImage($mask)
$graphics.Clear([System.Drawing.Color]::FromArgb(0, 0, 0, 0))
$opaque = New-Object System.Drawing.SolidBrush ([System.Drawing.Color]::FromArgb(255, 255, 0, 255))
$graphics.FillRectangle($opaque, 0, 0, $width, [Math]::Min($MaskTopPixels, $height))
$graphics.Dispose()
$opaque.Dispose()
$maskPath = Join-Path $Out 'sky-mask.png'
$mask.Save($maskPath, [System.Drawing.Imaging.ImageFormat]::Png)
$mask.Dispose()
Write-Step "masking the top $MaskTopPixels rows (animated sky and treeline)"
}
$failed = @()
foreach ($shot in $captures) {
$expected = Join-Path $Baseline "screenshots\$($shot.Name)"
if (-not (Test-Path $expected)) {
$failed += "$($shot.Name): no baseline counterpart"
continue
}
$report = Join-Path $Out "compare-$($shot.BaseName).json"
if ($maskPath) {
& dotnet $cli compare-screenshots $expected $shot.FullName $report $Tolerance $MaxDifferentFraction $maskPath | Out-Null
}
else {
& dotnet $cli compare-screenshots $expected $shot.FullName $report $Tolerance $MaxDifferentFraction | Out-Null
}
$verdict = Get-Content $report -Raw | ConvertFrom-Json
$fraction = $verdict.differentPixelFraction
if ($null -eq $fraction) { $fraction = $verdict.DifferentPixelFraction }
$passed = $verdict.passed
if ($null -eq $passed) { $passed = $verdict.Passed }
if ($passed) {
Write-Step "PASS $($shot.Name) (differing fraction $fraction)"
}
else {
$failed += "$($shot.Name): differing fraction $fraction exceeds $MaxDifferentFraction (report: $report)"
}
}
if ($failed.Count -gt 0) {
Write-Host ''
Write-Host '[pixel-gate] FAILED:' -ForegroundColor Red
$failed | ForEach-Object { Write-Host " - $_" -ForegroundColor Red }
exit 1
}
Write-Step 'all screenshots match the baseline'
exit 0