acdream/tools/run-backend-differential-gate.ps1

<#
.SYNOPSIS
    Campaign V slice V7 instrument: the GL-versus-Vulkan connected differential.

.DESCRIPTION
    Runs the SAME connected route twice against the local ACE server -- once with
    ACDREAM_RENDER_BACKEND=gl, once with =vulkan -- pairs the screenshots by name,
    compares each pair with the `compare-screenshots` CLI at the plan's pinned
    tolerance, and prints one verdict table.

    This is the instrument for plan section 2's pixel-acceptance contract:

        compare-screenshots expected.png actual.png out.json
            channel tolerance 2, maximum differing fraction 0.001,
            MSAA off, ACDREAM_DAY_GROUP pinned,
            at every deterministic checkpoint of the connected route.

    FOUR THINGS ARE FORCED, and each is load-bearing.

    1.  ACDREAM_MSAA_SAMPLES=0 ON BOTH LAUNCHES. Multisample resolve positions
        are explicitly unspecified across implementations. Plan section 5.5.16
        MEASURED what leaving them on costs: 8.83% of the frame differed at 4x,
        81,359 pixels of 921,600, essentially all of it hugging foliage and
        silhouette edges. That is two drivers' sample patterns, not a renderer
        divergence, and it swamps the 0.001 threshold by ninety-fold. Forcing it
        off on both runs is what makes the remaining difference attributable.
        Note that it must be forced on BOTH -- matching a GL run at 4x against a
        Vulkan run at 0 would be worse than either.

    2.  ACDREAM_DAY_GROUP and ACDREAM_WORLD_TIME pinned. The Dereth clock
        advances with wall time, and it does not only move the sky: it moves the
        SUN, so every lit surface in the scene drifts with it. The route used to
        press AcdreamCycleTimeOfDay three times instead, and that was measured
        NOT to hold -- SetDebugTime is transient by design and ACE's next
        TimeSync clears it within seconds. What that cost, measured at V7 on two
        captures 45 s apart at ONE stop on ONE backend: 22.3% of the frame at
        Holtburg and 11.8% inside the Facility Hub. A cross-backend number taken
        against that noise floor means nothing, which is why the day fraction is
        now pinned by env var and the route no longer presses anything.

    3.  ACDREAM_SKY_PHASE_SECONDS pinned ON BOTH LAUNCHES. The day group and the
        time-of-day override pin ONE of the sky's two clocks -- the one that
        chooses the keyframe and the sun angle. The cloud sheet runs on the
        other: TexVelocityX/Y accumulate against real elapsed time by design,
        because retail's clouds drift independently of the date, so no amount of
        world-clock pinning makes two launches agree about where they are. The
        V6m smoke pair measured the cost of leaving it: 89% of an 18.52%
        whole-frame difference lived in the top 240 rows. Pinning the phase is
        instrument determinism -- it is off in every ordinary run, it changes one
        input to a UV offset, and it is what lets this gate keep the SKY under
        strict comparison instead of masking it, which is what -MaskTopPixels
        would have cost.

    4.  A DESKTOP WITNESS per run, with the guards the repeat-run gate
        (run-repeat-connected-gate.ps1) learned the hard way. The client's own
        capture is the differential's subject, but a blank frame reads back as a
        valid PNG (plan section 5.5.2), so a second instrument that shares
        nothing with the renderer decides whether the run is worth comparing at
        all. Its failure modes are the same three that gate documents:

          * an overlapping window or a locked screen -- CopyFromScreen grabs
            whatever is composited at those coordinates, so run this with
            nothing else on the primary monitor and the desktop unlocked;
          * stray input -- a foregrounded window takes keystrokes and scrolls,
            which moves the camera off the pinned stop and produces an enormous
            spurious difference. The client log is scanned for camera-affecting
            input afterwards and the run is aborted if any arrived;
          * a hard kill -- ACE holds the session about three minutes and the
            SECOND launch of this gate is the one that pays for it, so both runs
            close through WM_CLOSE.

.PARAMETER Out
    Directory for both captures, the comparison reports and the verdict JSON.

.PARAMETER Route
    The route both backends run. Defaults to the differential route, which is
    written for two-launch determinism -- see its header before substituting one.

.PARAMETER Smoke
    Compare only the FIRST stop. The route is truncated after its first
    `screenshot` verb, so the run is short and the artifact records exactly what
    ran. Use this to measure the distance between the backends without paying
    for the whole route; the full route is the V7 gate.

.PARAMETER Tolerance
    Per-channel absolute difference allowed. Default 2, the plan's pinned value.

.PARAMETER MaxDifferentFraction
    Fraction of differing pixels allowed. Default 0.001, likewise pinned.

.PARAMETER DayGroup
    Sky day-group index pinned on both launches. Default 0.

.PARAMETER WorldDayFraction
    Dereth day fraction pinned on both launches, in [0, 1). Default 0.5 -- noon,
    the brightest and flattest lighting the day has, which is what the route's
    three AcdreamCycleTimeOfDay presses were aiming at before it was measured
    that they do not hold. See forced item 2 above.

.PARAMETER SkyPhaseSeconds
    Elapsed-seconds value the sky's UV scroll is pinned to on both launches.
    Default 0 -- the cloud sheet's authored origin. Any finite value works: the
    offset is taken modulo 1 per axis. See forced item 3 above.

.PARAMETER MaskTopPixels
    Height in pixels of a top band excluded from every comparison. DEFAULT 0 --
    nothing is masked, and the gate is a strict identity check on the whole
    frame.

    It exists because the V6m smoke run measured what the band costs and the
    number belongs where the next reader will find it. With the world clock
    pinned by the route and MSAA off, a GL-versus-Vulkan pair at Holtburg
    differed in 18.52% of the frame -- and 89% of that difference sat in the top
    240 rows, which is the scrolling cloud sheet plus the treeline it sits
    behind. The cloud sheet advances with WALL time, not the Dereth clock, so
    two launches minutes apart can never agree there; this is the same
    population the offline pixel gate masks its top 280 rows for. Rows 300-720
    -- terrain, blending, roads, buildings, the character and the whole retained
    UI -- contributed about 9,500 differing pixels between them.

    Whether V7 accepts a mask, pins the cloud phase, or fixes something the
    smoke was actually seeing is V7's decision, not this script's, which is why
    the default is off. Do NOT reach for this to make a failing gate pass: plan
    section 7.1 rule 2.

.PARAMETER MinRenderedBytes
    Desktop-grab PNG size below which a run is called blank and the comparison
    is refused. Default 500000, the repeat gate's measured separator.

.PARAMETER CooldownSeconds
    Wait between the two launches so ACE clears the first graceful logout.
    Default 15.

.EXAMPLE
    tools/run-backend-differential-gate.ps1 -Out artifacts/v7-differential -Smoke
    tools/run-backend-differential-gate.ps1 -Out artifacts/v7-differential
#>
[CmdletBinding()]
param(
    [Parameter(Mandatory = $true)][string]$Out,
    [string]$Route,
    [switch]$Smoke,
    [int]$Tolerance = 2,
    [double]$MaxDifferentFraction = 0.001,
    [int]$DayGroup = 0,
    [double]$WorldDayFraction = 0.5,
    [double]$SkyPhaseSeconds = 0,
    [int]$MaskTopPixels = 0,
    [int]$MinRenderedBytes = 500000,
    [int]$CooldownSeconds = 15,
    [int]$RouteTimeoutSeconds = 600,
    [switch]$SkipBuild
)

$ErrorActionPreference = 'Stop'
$repo = Split-Path -Parent $PSScriptRoot
$exe = Join-Path $repo 'src\AcDream.App\bin\Release\net10.0\AcDream.App.exe'
$cli = Join-Path $repo 'src\AcDream.Cli\bin\Release\net10.0\AcDream.Cli.dll'
if (-not $Route) { $Route = Join-Path $repo 'tools\connected-backend-differential.route.txt' }

function Write-Step($message) { Write-Host "[differential] $message" }

# --- Preconditions ------------------------------------------------------------
if (Get-Process -Name AcDream.App -ErrorAction SilentlyContinue) {
    throw 'AcDream.App is already running. This gate uses the shared test account and must not steal its session.'
}
if (@(Get-NetUDPEndpoint -LocalPort 9000 -ErrorAction SilentlyContinue).Count -eq 0) {
    throw 'local ACE is not listening on UDP port 9000.'
}
if (-not (Test-Path $Route)) { throw "Route not found: $Route" }

if (-not $SkipBuild) {
    Write-Step 'building Release'
    & dotnet build (Join-Path $repo 'AcDream.slnx') -c Release --nologo -v q | Out-Null
    if ($LASTEXITCODE -ne 0) { throw "Release build failed with exit code $LASTEXITCODE." }
}
if (-not (Test-Path $exe)) { throw "Client not found at $exe." }
if (-not (Test-Path $cli)) { throw "AcDream.Cli not found at $cli." }

if (Test-Path $Out) { Remove-Item -Recurse -Force $Out }
New-Item -ItemType Directory -Force -Path $Out | Out-Null

# The route both runs execute. -Smoke truncates after the first capture; the
# truncated file is written into the artifact directory so what ran is on record.
$routeLines = Get-Content -LiteralPath $Route
if ($Smoke) {
    $firstShot = -1
    for ($i = 0; $i -lt $routeLines.Count; $i++) {
        if ($routeLines[$i].TrimStart().StartsWith('screenshot ')) { $firstShot = $i; break }
    }
    if ($firstShot -lt 0) { throw "The route has no screenshot verb to truncate at: $Route" }
    $routeLines = @($routeLines[0..$firstShot]) + @('sleep 3000')
    Write-Step "smoke mode: route truncated after its first capture ($($firstShot + 1) lines)"
}
$effectiveRoute = Join-Path $Out 'route.txt'
Set-Content -Encoding utf8 -Path $effectiveRoute -Value $routeLines

Add-Type -AssemblyName System.Drawing
Add-Type @'
using System;
using System.Runtime.InteropServices;
public static class DifferentialGateSurface {
    [DllImport("user32.dll")] public static extern bool SetForegroundWindow(IntPtr hWnd);
    [DllImport("user32.dll")] public static extern bool ShowWindow(IntPtr hWnd, int nCmdShow);
    [DllImport("user32.dll")] public static extern bool GetClientRect(IntPtr hWnd, out RECT r);
    [DllImport("user32.dll")] public static extern bool ClientToScreen(IntPtr hWnd, ref POINT p);
    [StructLayout(LayoutKind.Sequential)] public struct RECT { public int L, T, R, B; }
    [StructLayout(LayoutKind.Sequential)] public struct POINT { public int X, Y; }
}
'@

function Invoke-BackendRun([string]$Backend) {
    $dir = Join-Path $Out $Backend
    New-Item -ItemType Directory -Force -Path $dir | Out-Null
    $log = Join-Path $dir 'client.log'
    $grab = Join-Path $dir 'desktop-grab.png'

    $env:ACDREAM_DAT_DIR = Join-Path $env:USERPROFILE "Documents\Asheron's Call"
    $env:ACDREAM_LIVE = '1'
    $env:ACDREAM_TEST_HOST = '127.0.0.1'
    $env:ACDREAM_TEST_PORT = '9000'
    $env:ACDREAM_TEST_USER = 'testaccount'
    $env:ACDREAM_TEST_PASS = 'testpassword'
    $env:ACDREAM_RETAIL_UI = '1'
    $env:ACDREAM_DEVTOOLS = '0'
    $env:ACDREAM_DAY_GROUP = "$DayGroup"
    # Both launches, always. See the .DESCRIPTION notes 1 and 3.
    $env:ACDREAM_MSAA_SAMPLES = '0'
    $env:ACDREAM_SKY_PHASE_SECONDS =
        $SkyPhaseSeconds.ToString([System.Globalization.CultureInfo]::InvariantCulture)
    $env:ACDREAM_WORLD_TIME =
        $WorldDayFraction.ToString([System.Globalization.CultureInfo]::InvariantCulture)
    $env:ACDREAM_RENDER_BACKEND = $Backend
    $env:ACDREAM_UI_PROBE_SCRIPT = $effectiveRoute
    $env:ACDREAM_AUTOMATION_ARTIFACT_DIR = $dir

    Write-Step "launching $Backend (MSAA off, day group $DayGroup)"
    # Normal window style: the desktop witness needs a composited frame.
    $proc = Start-Process -FilePath $exe -WorkingDirectory $repo `
        -RedirectStandardOutput $log -RedirectStandardError "$log.err" -PassThru

    $completed = $false
    $deadline = (Get-Date).AddSeconds($RouteTimeoutSeconds)
    while ((Get-Date) -lt $deadline) {
        if (Select-String -Path $log -SimpleMatch '[UI-PROBE] UI probe script complete' `
                -ErrorAction SilentlyContinue) { $completed = $true; break }
        if ($proc.HasExited) { break }
        Start-Sleep -Milliseconds 1000
    }

    # The desktop witness, taken while the client is still in its trailing sleep
    # and therefore still rendering the last scene of the route.
    $grabBytes = 0
    if (-not $proc.HasExited) {
        try {
            $proc.Refresh()
            $h = $proc.MainWindowHandle
            if ($h -ne [IntPtr]::Zero) {
                [DifferentialGateSurface]::ShowWindow($h, 9) | Out-Null   # SW_RESTORE
                [DifferentialGateSurface]::SetForegroundWindow($h) | Out-Null
                Start-Sleep -Milliseconds 1200
                $r = New-Object DifferentialGateSurface+RECT
                [DifferentialGateSurface]::GetClientRect($h, [ref]$r) | Out-Null
                $p = New-Object DifferentialGateSurface+POINT
                [DifferentialGateSurface]::ClientToScreen($h, [ref]$p) | Out-Null
                $w = $r.R - $r.L; $ht = $r.B - $r.T
                if ($w -gt 0 -and $ht -gt 0) {
                    $bmp = New-Object System.Drawing.Bitmap $w, $ht
                    $g = [System.Drawing.Graphics]::FromImage($bmp)
                    $g.CopyFromScreen($p.X, $p.Y, 0, 0, (New-Object System.Drawing.Size $w, $ht))
                    $bmp.Save($grab, [System.Drawing.Imaging.ImageFormat]::Png)
                    $g.Dispose(); $bmp.Dispose()
                    $grabBytes = (Get-Item $grab).Length
                }
            }
        }
        catch { Write-Step "desktop grab failed on ${Backend}: $_" }
    }

    # Graceful close, always: a hard kill leaves ACE holding the session about
    # three minutes, and the second launch is the one that would pay for it.
    $app = Get-Process -Name AcDream.App -ErrorAction SilentlyContinue
    if ($app) {
        $app.CloseMainWindow() | Out-Null
        if (-not $app.WaitForExit(20000)) {
            Write-Step "WM_CLOSE timed out on $Backend; forcing"
            $app | Stop-Process -Force
        }
    }

    Remove-Item Env:\ACDREAM_RENDER_BACKEND -ErrorAction SilentlyContinue
    Remove-Item Env:\ACDREAM_MSAA_SAMPLES -ErrorAction SilentlyContinue
    Remove-Item Env:\ACDREAM_SKY_PHASE_SECONDS -ErrorAction SilentlyContinue
    Remove-Item Env:\ACDREAM_WORLD_TIME -ErrorAction SilentlyContinue

    $strayInput = @(Select-String -Path $log -Pattern 'ScrollUp|ScrollDown|ZoomIn|ZoomOut|CameraZoom' `
        -CaseSensitive -ErrorAction SilentlyContinue)
    $witness = if ($grabBytes -ge $MinRenderedBytes) { 'RENDERED' }
               elseif ($grabBytes -gt 0) { 'BLANK' }
               else { 'NO-CAPTURE' }

    return [pscustomobject][ordered]@{
        Backend = $Backend
        RouteCompleted = $completed
        GrabBytes = $grabBytes
        Witness = $witness
        StrayInputLines = @($strayInput | Select-Object -First 5 | ForEach-Object { $_.Line })
        Screenshots = Join-Path $dir 'screenshots'
        Log = $log
    }
}

$gl = Invoke-BackendRun 'gl'
Write-Step "cooldown ${CooldownSeconds}s so ACE clears the graceful logout"
Start-Sleep -Seconds $CooldownSeconds
$vk = Invoke-BackendRun 'vulkan'

# --- Sanity, before any pixel is compared ------------------------------------
$aborts = @()
foreach ($run in @($gl, $vk)) {
    if (-not $run.RouteCompleted) { $aborts += "$($run.Backend): the route did not complete" }
    if ($run.Witness -ne 'RENDERED') {
        $aborts += "$($run.Backend): desktop witness says $($run.Witness) ($($run.GrabBytes) B) -- locked screen, an overlapping window, or a blank frame"
    }
    if ($run.StrayInputLines.Count -gt 0) {
        $aborts += "$($run.Backend): camera-affecting input reached the window; the capture is not comparable"
    }
}

$pairs = @()
if ($aborts.Count -eq 0) {
    $glShots = @(Get-ChildItem $gl.Screenshots -Filter *.png -ErrorAction SilentlyContinue)
    if ($glShots.Count -eq 0) { $aborts += 'gl: no screenshots were captured' }

    # The comparer skips any pixel whose mask alpha is non-zero, so the excluded
    # band is opaque and everything compared is transparent. Same construction as
    # the offline pixel gate's sky mask; off unless -MaskTopPixels was given.
    $maskPath = $null
    if ($MaskTopPixels -gt 0 -and $glShots.Count -gt 0) {
        $probeImage = [System.Drawing.Bitmap]::FromFile($glShots[0].FullName)
        $mw = $probeImage.Width; $mh = $probeImage.Height
        $probeImage.Dispose()
        $mask = New-Object System.Drawing.Bitmap($mw, $mh, [System.Drawing.Imaging.PixelFormat]::Format32bppArgb)
        $graphics = [System.Drawing.Graphics]::FromImage($mask)
        $graphics.Clear([System.Drawing.Color]::FromArgb(0, 0, 0, 0))
        $opaque = New-Object System.Drawing.SolidBrush ([System.Drawing.Color]::FromArgb(255, 255, 0, 255))
        $graphics.FillRectangle($opaque, 0, 0, $mw, [Math]::Min($MaskTopPixels, $mh))
        $graphics.Dispose(); $opaque.Dispose()
        $maskPath = Join-Path $Out 'mask.png'
        $mask.Save($maskPath, [System.Drawing.Imaging.ImageFormat]::Png)
        $mask.Dispose()
        Write-Step "masking the top $MaskTopPixels rows of every pair"
    }

    foreach ($shot in $glShots) {
        $counterpart = Join-Path $vk.Screenshots $shot.Name
        if (-not (Test-Path $counterpart)) {
            $pairs += [pscustomobject][ordered]@{
                Stop = $shot.BaseName; Verdict = 'UNPAIRED'
                DifferentPixels = $null; Fraction = $null; MaxChannelDelta = $null
            }
            continue
        }
        $report = Join-Path $Out "compare-$($shot.BaseName).json"
        if ($maskPath) {
            & dotnet $cli compare-screenshots $shot.FullName $counterpart $report $Tolerance $MaxDifferentFraction $maskPath | Out-Null
        }
        else {
            & dotnet $cli compare-screenshots $shot.FullName $counterpart $report $Tolerance $MaxDifferentFraction | Out-Null
        }
        $verdict = Get-Content $report -Raw | ConvertFrom-Json
        $outcome = if ($verdict.passed) { 'MATCH' } else { 'DIVERGED' }
        $pairs += [pscustomobject][ordered]@{
            Stop = $shot.BaseName
            Verdict = $outcome
            DifferentPixels = $verdict.differentPixels
            Fraction = $verdict.differentPixelFraction
            MaxChannelDelta = $verdict.maximumChannelDelta
        }
    }
}

# --- Verdict ------------------------------------------------------------------
Write-Host ''
if ($aborts.Count -gt 0) {
    Write-Host '[differential] ABORTED before comparison:' -ForegroundColor Red
    $aborts | ForEach-Object { Write-Host "  - $_" -ForegroundColor Red }
}
else {
    $pairs | Format-Table -AutoSize | Out-String | Write-Host
}

$diverged = @($pairs | Where-Object Verdict -ne 'MATCH')
$report = [pscustomobject][ordered]@{
    Passed = ($aborts.Count -eq 0 -and $diverged.Count -eq 0)
    Smoke = [bool]$Smoke
    Commit = (& git -C $repo rev-parse HEAD).Trim()
    Route = $Route
    EffectiveRoute = $effectiveRoute
    ChannelTolerance = $Tolerance
    MaxDifferentPixelFraction = $MaxDifferentFraction
    MsaaSamples = 0
    DayGroup = $DayGroup
    WorldDayFraction = $WorldDayFraction
    SkyPhaseSeconds = $SkyPhaseSeconds
    MaskTopPixels = $MaskTopPixels
    Runs = @($gl, $vk)
    Pairs = @($pairs)
    Aborts = @($aborts)
}
$reportPath = Join-Path $Out 'differential.json'
$report | ConvertTo-Json -Depth 8 | Set-Content -LiteralPath $reportPath -Encoding utf8
Write-Step "report: $reportPath"

if ($aborts.Count -gt 0) { exit 2 }
if ($diverged.Count -gt 0) {
    Write-Host ("[differential] FAILED: {0}/{1} stop(s) diverged." -f $diverged.Count, $pairs.Count) -ForegroundColor Red
    exit 1
}
Write-Step ("PASS: {0}/{0} stop(s) match at tolerance {1} / fraction {2}." -f $pairs.Count, $Tolerance, $MaxDifferentFraction)
exit 0