diff --git a/docs/plans/2026-07-27-vulkan-campaign.md b/docs/plans/2026-07-27-vulkan-campaign.md index 250d4df1..e7066614 100644 --- a/docs/plans/2026-07-27-vulkan-campaign.md +++ b/docs/plans/2026-07-27-vulkan-campaign.md @@ -391,10 +391,45 @@ a loud `Debug.Assert` in Debug builds. A Debug run therefore shows one failure that is neither a regression nor yours. The V0 baseline is **3,785 passed / 3 skipped** in Release (3,763 pre-campaign plus 22 contract tests). -"Pixel gate" means: capture the deterministic checkpoints from the parent -commit's build, capture again at slice HEAD, compare with the -`compare-screenshots` CLI at tolerance 2 / fraction 0.001, MSAA off, -`ACDREAM_DAY_GROUP` pinned. +### 5.1 The offline pixel gate + +"Pixel gate" means `tools/run-offline-pixel-gate.ps1`: capture at the parent +commit, capture at slice HEAD, compare with the `compare-screenshots` CLI at +tolerance 2 / fraction 0.001. + +The client is launched **without `ACDREAM_LIVE`**, so it renders the world +straight from the DATs. No session is created and no ACE state can be disturbed, +which means this gate runs unattended — it needs neither the live server nor the +user. That matters: seven slices (V2, V4a–V4g) are renderer ports whose whole +acceptance criterion is "no pixel changed." + +``` +tools/run-offline-pixel-gate.ps1 -Out artifacts/gate-base # at the parent commit +tools/run-offline-pixel-gate.ps1 -Out artifacts/gate-head -Baseline artifacts/gate-base +``` + +**Determinism was measured, not assumed.** Two captures at the same commit +initially differed in 0.29% of pixels — far above the 0.001 threshold. The +differences were confined to the top ~180 rows: the sky animates (clouds scroll, +the sun moves) and the Dereth clock advances with wall time, so two launches can +never agree there. Everything below the horizon was stable. With the top 280 rows +masked, two independent same-commit pairs differ by **15 and 17 pixels out of +563,200 compared** — a fraction of 0.000027, roughly a 33× margin under the +threshold. The gate is a strict identity check on everything it covers, rather +than a loose tolerance that would hide real regressions. + +**Coverage.** Terrain and terrain blending, scenery, static world meshes, water, +fog, and the entire retained UI (vitals, spell bar, toolbar, chat, radar). + +**Not covered — these still need a user visual gate:** sky (masked), EnvCell +interiors, particles, and the paperdoll/appraisal viewports, because the offline +scene is a fixed outdoor view with no camera control. Slices **V4e** (particles), +**V4f** (sky), and **V4g** (viewports) therefore carry a user gate in addition to +their automated one. + +MSAA is left at the quality preset for GL-versus-GL self-differentials, where it +is deterministic. The V7 GL-versus-Vulkan differential must force MSAA off, +because sample positions are not specified across implementations. | Slice | Scope | Gate | |---|---|---| diff --git a/tools/run-offline-pixel-gate.ps1 b/tools/run-offline-pixel-gate.ps1 new file mode 100644 index 00000000..75e80f1c --- /dev/null +++ b/tools/run-offline-pixel-gate.ps1 @@ -0,0 +1,234 @@ +<# +.SYNOPSIS + Campaign V self-differential pixel gate. Captures deterministic offline + screenshots and optionally compares them against a baseline capture. + +.DESCRIPTION + Every Campaign V renderer-port slice must prove it changed no pixels. The + connected lifecycle gate needs a live ACE server and the user's attention; + this gate needs neither. The client is launched WITHOUT ACDREAM_LIVE, so it + renders the world straight from the DATs, no session is created, and no ACE + state can be disturbed. + + Coverage: terrain, terrain blending, scenery, static world meshes, sky and + fog, and the full retained UI (vitals, spell bar, toolbar, chat, radar). + It does NOT cover EnvCell interiors, particles, or the paperdoll viewport — + those still need the connected route, so slices touching them (V4e, V4g) + must additionally be gated by the user. + + Determinism levers: ACDREAM_DAY_GROUP pins the sky day-group so weather and + lighting do not drift between runs, and a fixed warm-up lets streaming settle + before the frame is captured. + +.PARAMETER Out + Directory to write this capture into. Created if absent, emptied if present. + +.PARAMETER Baseline + Optional. A directory from a previous capture. When supplied, every PNG is + compared against its namesake and the script fails on any mismatch. + +.PARAMETER WarmupMs + Milliseconds to let the world stream and settle before capturing. Default + 12000, which reliably reaches a fully populated view on the reference machine. + +.PARAMETER DayGroup + Sky day-group index to pin. Default 0. + +.PARAMETER Tolerance + Per-channel absolute difference allowed. Default 2, matching the project's + pinned screenshot rule. + +.PARAMETER MaxDifferentFraction + Fraction of differing pixels allowed. Default 0.001, likewise pinned. + +.PARAMETER MaskTopPixels + Height in pixels of the sky band excluded from comparison. Default 280. + + This is not a fudge factor. Two captures at the SAME commit were measured to + differ in 0.29% of pixels, and the differences were confined to the top ~180 + rows: the sky legitimately animates (clouds scroll, the sun moves) and the + Dereth clock advances with wall time, so the sky cannot be identical between + two launches. Everything below the horizon — terrain, blending, scenery, + static meshes, water, and the whole retained UI — was bit-stable. Masking the + animated band is what makes the rest a strict identity check rather than + forcing a loose tolerance that would hide real regressions everywhere else. + + Consequence: this gate does NOT cover sky rendering. Slice V4f must be gated + by the user instead. + +.PARAMETER SkipBuild + Skip the Release build (use when the caller already built). + +.EXAMPLE + # Capture a baseline at the parent commit, then gate the slice: + git stash + tools/run-offline-pixel-gate.ps1 -Out artifacts/gate-base + git stash pop + tools/run-offline-pixel-gate.ps1 -Out artifacts/gate-head -Baseline artifacts/gate-base +#> +[CmdletBinding()] +param( + [Parameter(Mandatory = $true)][string]$Out, + [string]$Baseline, + [int]$WarmupMs = 12000, + [int]$DayGroup = 0, + [int]$Tolerance = 2, + [double]$MaxDifferentFraction = 0.001, + [int]$MaskTopPixels = 280, + [switch]$SkipBuild +) + +$ErrorActionPreference = 'Stop' +$repo = Split-Path -Parent $PSScriptRoot +$exe = Join-Path $repo 'src\AcDream.App\bin\Release\net10.0\AcDream.App.exe' +$cli = Join-Path $repo 'src\AcDream.Cli\bin\Release\net10.0\AcDream.Cli.dll' + +function Write-Step($message) { Write-Host "[pixel-gate] $message" } + +# --- 1. Build ----------------------------------------------------------------- +if (-not $SkipBuild) { + Write-Step 'building Release' + & dotnet build (Join-Path $repo 'AcDream.slnx') -c Release --nologo -v q | Out-Null + if ($LASTEXITCODE -ne 0) { throw "Release build failed with exit code $LASTEXITCODE." } +} +if (-not (Test-Path $exe)) { throw "Client not found at $exe. Build Release first." } + +# --- 2. Prepare the capture directory and probe script ------------------------ +if (Test-Path $Out) { Remove-Item -Recurse -Force $Out } +New-Item -ItemType Directory -Force -Path $Out | Out-Null + +$probe = Join-Path $Out 'offline.probe.txt' +# The script runner reads one command per line. A single settled capture is the +# whole gate: a second stop would need camera movement, which offline has no +# deterministic way to drive. +Set-Content -Encoding utf8 -Path $probe -Value @" +sleep $WarmupMs +screenshot world-offline 30000 +sleep 500 +"@ + +$log = Join-Path $Out 'client.log' + +# --- 3. Launch offline -------------------------------------------------------- +$previousLive = $env:ACDREAM_LIVE +Remove-Item Env:\ACDREAM_LIVE -ErrorAction SilentlyContinue +$env:ACDREAM_DAT_DIR = Join-Path $env:USERPROFILE "Documents\Asheron's Call" +$env:ACDREAM_NO_AUDIO = '1' +$env:ACDREAM_RETAIL_UI = '1' +$env:ACDREAM_DAY_GROUP = "$DayGroup" +$env:ACDREAM_UI_PROBE_SCRIPT = $probe +$env:ACDREAM_AUTOMATION_ARTIFACT_DIR = $Out + +Write-Step "launching offline client (warmup ${WarmupMs}ms, day group $DayGroup)" +$proc = Start-Process -FilePath $exe -RedirectStandardOutput $log ` + -RedirectStandardError "$log.err" -PassThru -WindowStyle Minimized + +try { + $shots = Join-Path $Out 'screenshots' + $deadline = (Get-Date).AddMilliseconds($WarmupMs + 60000) + $captured = $false + while ((Get-Date) -lt $deadline) { + if ((Test-Path $shots) -and (Get-ChildItem $shots -Filter *.png -ErrorAction SilentlyContinue)) { + $captured = $true + break + } + if ($proc.HasExited) { break } + Start-Sleep -Milliseconds 1000 + } + + if (-not $captured) { + Write-Host (Get-Content $log -Tail 30 -ErrorAction SilentlyContinue) + throw 'No screenshot was captured before the deadline.' + } + # Let the probe script finish its trailing sleep so the PNG is fully flushed. + Start-Sleep -Milliseconds 1500 +} +finally { + # Graceful close: WM_CLOSE runs the shutdown path, so the ownership ledger + # converges the way the lifecycle tests expect. No ACE session exists here, + # but keeping the habit means this script is safe to point at a live run too. + $app = Get-Process -Name AcDream.App -ErrorAction SilentlyContinue + if ($app) { + $app.CloseMainWindow() | Out-Null + if (-not $app.WaitForExit(10000)) { + Write-Step 'WM_CLOSE timed out; forcing' + $app | Stop-Process -Force + } + } + if ($previousLive) { $env:ACDREAM_LIVE = $previousLive } +} + +$captures = Get-ChildItem (Join-Path $Out 'screenshots') -Filter *.png +Write-Step "captured $($captures.Count) screenshot(s) into $Out" + +# --- 4. Compare against the baseline ----------------------------------------- +if (-not $Baseline) { + Write-Step 'no baseline supplied; capture only' + exit 0 +} + +if (-not (Test-Path $cli)) { throw "AcDream.Cli not found at $cli." } + +# Build the sky mask. The comparer skips any pixel whose mask alpha is non-zero, +# so the band we exclude is opaque and everything compared is transparent. +$maskPath = $null +if ($MaskTopPixels -gt 0) { + Add-Type -AssemblyName System.Drawing + $probeImage = [System.Drawing.Bitmap]::FromFile($captures[0].FullName) + $width = $probeImage.Width + $height = $probeImage.Height + $probeImage.Dispose() + + $mask = New-Object System.Drawing.Bitmap($width, $height, [System.Drawing.Imaging.PixelFormat]::Format32bppArgb) + $graphics = [System.Drawing.Graphics]::FromImage($mask) + $graphics.Clear([System.Drawing.Color]::FromArgb(0, 0, 0, 0)) + $opaque = New-Object System.Drawing.SolidBrush ([System.Drawing.Color]::FromArgb(255, 255, 0, 255)) + $graphics.FillRectangle($opaque, 0, 0, $width, [Math]::Min($MaskTopPixels, $height)) + $graphics.Dispose() + $opaque.Dispose() + + $maskPath = Join-Path $Out 'sky-mask.png' + $mask.Save($maskPath, [System.Drawing.Imaging.ImageFormat]::Png) + $mask.Dispose() + Write-Step "masking the top $MaskTopPixels rows (animated sky)" +} + +$failed = @() +foreach ($shot in $captures) { + $expected = Join-Path $Baseline "screenshots\$($shot.Name)" + if (-not (Test-Path $expected)) { + $failed += "$($shot.Name): no baseline counterpart" + continue + } + + $report = Join-Path $Out "compare-$($shot.BaseName).json" + if ($maskPath) { + & dotnet $cli compare-screenshots $expected $shot.FullName $report $Tolerance $MaxDifferentFraction $maskPath | Out-Null + } + else { + & dotnet $cli compare-screenshots $expected $shot.FullName $report $Tolerance $MaxDifferentFraction | Out-Null + } + $verdict = Get-Content $report -Raw | ConvertFrom-Json + + $fraction = $verdict.differentPixelFraction + if ($null -eq $fraction) { $fraction = $verdict.DifferentPixelFraction } + $passed = $verdict.passed + if ($null -eq $passed) { $passed = $verdict.Passed } + + if ($passed) { + Write-Step "PASS $($shot.Name) (differing fraction $fraction)" + } + else { + $failed += "$($shot.Name): differing fraction $fraction exceeds $MaxDifferentFraction (report: $report)" + } +} + +if ($failed.Count -gt 0) { + Write-Host '' + Write-Host '[pixel-gate] FAILED:' -ForegroundColor Red + $failed | ForEach-Object { Write-Host " - $_" -ForegroundColor Red } + exit 1 +} + +Write-Step 'all screenshots match the baseline' +exit 0