a475732587
The offline capture window is minimised but still focusable, so a scroll or key press from whoever is at the keyboard can move the camera mid-capture. That yields two screenshots of the same scene from different camera positions and an enormous, entirely spurious pixel difference - which happened during slice V4b and was correctly discarded rather than interpreted. The gate now detects camera-affecting input in the client log and exits 2, so a perturbed run cannot be mistaken for a rendering regression in either direction. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
250 lines
10 KiB
PowerShell
250 lines
10 KiB
PowerShell
<#
|
|
.SYNOPSIS
|
|
Campaign V self-differential pixel gate. Captures deterministic offline
|
|
screenshots and optionally compares them against a baseline capture.
|
|
|
|
.DESCRIPTION
|
|
Every Campaign V renderer-port slice must prove it changed no pixels. The
|
|
connected lifecycle gate needs a live ACE server and the user's attention;
|
|
this gate needs neither. The client is launched WITHOUT ACDREAM_LIVE, so it
|
|
renders the world straight from the DATs, no session is created, and no ACE
|
|
state can be disturbed.
|
|
|
|
Coverage: terrain, terrain blending, scenery, static world meshes, sky and
|
|
fog, and the full retained UI (vitals, spell bar, toolbar, chat, radar).
|
|
It does NOT cover EnvCell interiors, particles, or the paperdoll viewport —
|
|
those still need the connected route, so slices touching them (V4e, V4g)
|
|
must additionally be gated by the user.
|
|
|
|
Determinism levers: ACDREAM_DAY_GROUP pins the sky day-group so weather and
|
|
lighting do not drift between runs, and a fixed warm-up lets streaming settle
|
|
before the frame is captured.
|
|
|
|
.PARAMETER Out
|
|
Directory to write this capture into. Created if absent, emptied if present.
|
|
|
|
.PARAMETER Baseline
|
|
Optional. A directory from a previous capture. When supplied, every PNG is
|
|
compared against its namesake and the script fails on any mismatch.
|
|
|
|
.PARAMETER WarmupMs
|
|
Milliseconds to let the world stream and settle before capturing. Default
|
|
12000, which reliably reaches a fully populated view on the reference machine.
|
|
|
|
.PARAMETER DayGroup
|
|
Sky day-group index to pin. Default 0.
|
|
|
|
.PARAMETER Tolerance
|
|
Per-channel absolute difference allowed. Default 2, matching the project's
|
|
pinned screenshot rule.
|
|
|
|
.PARAMETER MaxDifferentFraction
|
|
Fraction of differing pixels allowed. Default 0.001, likewise pinned.
|
|
|
|
.PARAMETER MaskTopPixels
|
|
Height in pixels of the sky band excluded from comparison. Default 280.
|
|
|
|
This is not a fudge factor. Two captures at the SAME commit were measured to
|
|
differ in 0.29% of pixels, and the differences were confined to the top ~180
|
|
rows: the sky legitimately animates (clouds scroll, the sun moves) and the
|
|
Dereth clock advances with wall time, so the sky cannot be identical between
|
|
two launches. Everything below the horizon — terrain, blending, scenery,
|
|
static meshes, water, and the whole retained UI — was bit-stable. Masking the
|
|
animated band is what makes the rest a strict identity check rather than
|
|
forcing a loose tolerance that would hide real regressions everywhere else.
|
|
|
|
Consequence: this gate does NOT cover sky rendering. Slice V4f must be gated
|
|
by the user instead.
|
|
|
|
.PARAMETER SkipBuild
|
|
Skip the Release build (use when the caller already built).
|
|
|
|
.EXAMPLE
|
|
# Capture a baseline at the parent commit, then gate the slice:
|
|
git stash
|
|
tools/run-offline-pixel-gate.ps1 -Out artifacts/gate-base
|
|
git stash pop
|
|
tools/run-offline-pixel-gate.ps1 -Out artifacts/gate-head -Baseline artifacts/gate-base
|
|
#>
|
|
[CmdletBinding()]
|
|
param(
|
|
[Parameter(Mandatory = $true)][string]$Out,
|
|
[string]$Baseline,
|
|
[int]$WarmupMs = 12000,
|
|
[int]$DayGroup = 0,
|
|
[int]$Tolerance = 2,
|
|
[double]$MaxDifferentFraction = 0.001,
|
|
[int]$MaskTopPixels = 280,
|
|
[switch]$SkipBuild
|
|
)
|
|
|
|
$ErrorActionPreference = 'Stop'
|
|
$repo = Split-Path -Parent $PSScriptRoot
|
|
$exe = Join-Path $repo 'src\AcDream.App\bin\Release\net10.0\AcDream.App.exe'
|
|
$cli = Join-Path $repo 'src\AcDream.Cli\bin\Release\net10.0\AcDream.Cli.dll'
|
|
|
|
function Write-Step($message) { Write-Host "[pixel-gate] $message" }
|
|
|
|
# --- 1. Build -----------------------------------------------------------------
|
|
if (-not $SkipBuild) {
|
|
Write-Step 'building Release'
|
|
& dotnet build (Join-Path $repo 'AcDream.slnx') -c Release --nologo -v q | Out-Null
|
|
if ($LASTEXITCODE -ne 0) { throw "Release build failed with exit code $LASTEXITCODE." }
|
|
}
|
|
if (-not (Test-Path $exe)) { throw "Client not found at $exe. Build Release first." }
|
|
|
|
# --- 2. Prepare the capture directory and probe script ------------------------
|
|
if (Test-Path $Out) { Remove-Item -Recurse -Force $Out }
|
|
New-Item -ItemType Directory -Force -Path $Out | Out-Null
|
|
|
|
$probe = Join-Path $Out 'offline.probe.txt'
|
|
# The script runner reads one command per line. A single settled capture is the
|
|
# whole gate: a second stop would need camera movement, which offline has no
|
|
# deterministic way to drive.
|
|
Set-Content -Encoding utf8 -Path $probe -Value @"
|
|
sleep $WarmupMs
|
|
screenshot world-offline 30000
|
|
sleep 500
|
|
"@
|
|
|
|
$log = Join-Path $Out 'client.log'
|
|
|
|
# --- 3. Launch offline --------------------------------------------------------
|
|
$previousLive = $env:ACDREAM_LIVE
|
|
Remove-Item Env:\ACDREAM_LIVE -ErrorAction SilentlyContinue
|
|
$env:ACDREAM_DAT_DIR = Join-Path $env:USERPROFILE "Documents\Asheron's Call"
|
|
$env:ACDREAM_NO_AUDIO = '1'
|
|
$env:ACDREAM_RETAIL_UI = '1'
|
|
$env:ACDREAM_DAY_GROUP = "$DayGroup"
|
|
$env:ACDREAM_UI_PROBE_SCRIPT = $probe
|
|
$env:ACDREAM_AUTOMATION_ARTIFACT_DIR = $Out
|
|
|
|
Write-Step "launching offline client (warmup ${WarmupMs}ms, day group $DayGroup)"
|
|
$proc = Start-Process -FilePath $exe -RedirectStandardOutput $log `
|
|
-RedirectStandardError "$log.err" -PassThru -WindowStyle Minimized
|
|
|
|
try {
|
|
$shots = Join-Path $Out 'screenshots'
|
|
$deadline = (Get-Date).AddMilliseconds($WarmupMs + 60000)
|
|
$captured = $false
|
|
while ((Get-Date) -lt $deadline) {
|
|
if ((Test-Path $shots) -and (Get-ChildItem $shots -Filter *.png -ErrorAction SilentlyContinue)) {
|
|
$captured = $true
|
|
break
|
|
}
|
|
if ($proc.HasExited) { break }
|
|
Start-Sleep -Milliseconds 1000
|
|
}
|
|
|
|
if (-not $captured) {
|
|
Write-Host (Get-Content $log -Tail 30 -ErrorAction SilentlyContinue)
|
|
throw 'No screenshot was captured before the deadline.'
|
|
}
|
|
# Let the probe script finish its trailing sleep so the PNG is fully flushed.
|
|
Start-Sleep -Milliseconds 1500
|
|
}
|
|
finally {
|
|
# Graceful close: WM_CLOSE runs the shutdown path, so the ownership ledger
|
|
# converges the way the lifecycle tests expect. No ACE session exists here,
|
|
# but keeping the habit means this script is safe to point at a live run too.
|
|
$app = Get-Process -Name AcDream.App -ErrorAction SilentlyContinue
|
|
if ($app) {
|
|
$app.CloseMainWindow() | Out-Null
|
|
if (-not $app.WaitForExit(10000)) {
|
|
Write-Step 'WM_CLOSE timed out; forcing'
|
|
$app | Stop-Process -Force
|
|
}
|
|
}
|
|
if ($previousLive) { $env:ACDREAM_LIVE = $previousLive }
|
|
}
|
|
|
|
$captures = Get-ChildItem (Join-Path $Out 'screenshots') -Filter *.png
|
|
Write-Step "captured $($captures.Count) screenshot(s) into $Out"
|
|
|
|
# The offline window is minimised but still focusable, so a stray scroll or key
|
|
# press from whoever is at the keyboard can move the camera mid-capture. That
|
|
# produces two screenshots of the same scene from different camera positions —
|
|
# a huge, entirely spurious pixel difference. This actually happened during
|
|
# slice V4b. Detect it and fail loudly rather than let a comparison be
|
|
# interpreted as a rendering regression.
|
|
$cameraInput = Select-String -Path $log -Pattern 'ScrollUp|ScrollDown|ZoomIn|ZoomOut|CameraZoom' `
|
|
-CaseSensitive -ErrorAction SilentlyContinue
|
|
if ($cameraInput) {
|
|
Write-Host ''
|
|
Write-Host '[pixel-gate] ABORTED: camera-affecting input reached the capture window.' -ForegroundColor Red
|
|
Write-Host ' The capture is not comparable. Re-run without touching the machine.' -ForegroundColor Red
|
|
$cameraInput | Select-Object -First 5 | ForEach-Object { Write-Host " $($_.Line)" -ForegroundColor Red }
|
|
exit 2
|
|
}
|
|
|
|
# --- 4. Compare against the baseline -----------------------------------------
|
|
if (-not $Baseline) {
|
|
Write-Step 'no baseline supplied; capture only'
|
|
exit 0
|
|
}
|
|
|
|
if (-not (Test-Path $cli)) { throw "AcDream.Cli not found at $cli." }
|
|
|
|
# Build the sky mask. The comparer skips any pixel whose mask alpha is non-zero,
|
|
# so the band we exclude is opaque and everything compared is transparent.
|
|
$maskPath = $null
|
|
if ($MaskTopPixels -gt 0) {
|
|
Add-Type -AssemblyName System.Drawing
|
|
$probeImage = [System.Drawing.Bitmap]::FromFile($captures[0].FullName)
|
|
$width = $probeImage.Width
|
|
$height = $probeImage.Height
|
|
$probeImage.Dispose()
|
|
|
|
$mask = New-Object System.Drawing.Bitmap($width, $height, [System.Drawing.Imaging.PixelFormat]::Format32bppArgb)
|
|
$graphics = [System.Drawing.Graphics]::FromImage($mask)
|
|
$graphics.Clear([System.Drawing.Color]::FromArgb(0, 0, 0, 0))
|
|
$opaque = New-Object System.Drawing.SolidBrush ([System.Drawing.Color]::FromArgb(255, 255, 0, 255))
|
|
$graphics.FillRectangle($opaque, 0, 0, $width, [Math]::Min($MaskTopPixels, $height))
|
|
$graphics.Dispose()
|
|
$opaque.Dispose()
|
|
|
|
$maskPath = Join-Path $Out 'sky-mask.png'
|
|
$mask.Save($maskPath, [System.Drawing.Imaging.ImageFormat]::Png)
|
|
$mask.Dispose()
|
|
Write-Step "masking the top $MaskTopPixels rows (animated sky)"
|
|
}
|
|
|
|
$failed = @()
|
|
foreach ($shot in $captures) {
|
|
$expected = Join-Path $Baseline "screenshots\$($shot.Name)"
|
|
if (-not (Test-Path $expected)) {
|
|
$failed += "$($shot.Name): no baseline counterpart"
|
|
continue
|
|
}
|
|
|
|
$report = Join-Path $Out "compare-$($shot.BaseName).json"
|
|
if ($maskPath) {
|
|
& dotnet $cli compare-screenshots $expected $shot.FullName $report $Tolerance $MaxDifferentFraction $maskPath | Out-Null
|
|
}
|
|
else {
|
|
& dotnet $cli compare-screenshots $expected $shot.FullName $report $Tolerance $MaxDifferentFraction | Out-Null
|
|
}
|
|
$verdict = Get-Content $report -Raw | ConvertFrom-Json
|
|
|
|
$fraction = $verdict.differentPixelFraction
|
|
if ($null -eq $fraction) { $fraction = $verdict.DifferentPixelFraction }
|
|
$passed = $verdict.passed
|
|
if ($null -eq $passed) { $passed = $verdict.Passed }
|
|
|
|
if ($passed) {
|
|
Write-Step "PASS $($shot.Name) (differing fraction $fraction)"
|
|
}
|
|
else {
|
|
$failed += "$($shot.Name): differing fraction $fraction exceeds $MaxDifferentFraction (report: $report)"
|
|
}
|
|
}
|
|
|
|
if ($failed.Count -gt 0) {
|
|
Write-Host ''
|
|
Write-Host '[pixel-gate] FAILED:' -ForegroundColor Red
|
|
$failed | ForEach-Object { Write-Host " - $_" -ForegroundColor Red }
|
|
exit 1
|
|
}
|
|
|
|
Write-Step 'all screenshots match the baseline'
|
|
exit 0
|