acdream/tools/compile-shaders.ps1
Erik 777f60708d ci(render): make V9's first CI run green on both operating systems
The lavapipe job did the thing it was built to do on its first attempt.
It accepted a Cpu device at API 1.4, created a device and read pixels
back, captured a real frame, and exited 4 when a feature was forced
unsupported. Three other things were red, and none of them were the
Vulkan backend.

The shader-freshness step aborted for two separate Linux faults in the
compiler tool. Disposing the Silk.NET API container unloads the native
module, and dlclose-ing libshaderc_shared.so leaves glslang's
process-level teardown running against unmapped code. Bisected with a
four-mode probe on Ubuntu 24.04: GetApi, CompilerInitialize and
CompilerRelease each exit 0, and adding only the container Dispose turns
the exit into SIGSEGV. That is the 134 CI reported. shaderc's own handles
are still released; the container is not, because the module's lifetime
is the process's and the process is one statement from returning.
Separately, a portable dotnet build leaves the native under
runtimes/linux-x64/native/ and makes reaching it Silk.NET's probing
problem, which it solved on a local Ubuntu 24.04 and did not solve on the
runner. The script now publishes the tool for the host RID, so the native
sits beside the assembly where AppContext.BaseDirectory finds it, and
checks for it by name so a regression says which file is missing rather
than which names failed.

With both fixed, the question section 5.5.20 left open has an answer:
Linux shaderc and Windows shaderc agree byte-for-byte at the pinned Silk
2.23.0. Eighteen of eighteen .spv identical, manifest identical. The byte
comparison stays a byte comparison.

The Windows leg of portable-headless was running sudo apt-get. That step
is older than this campaign - it is red in the 2026-07-27 main run too -
and it was misplaced rather than mis-conditioned. Nothing in that job
opens a display or links GL, and the graphical jobs that do call xvfb-run
take it from the runner image, so the step is deleted rather than
guarded. Every remaining step in the two-operating-system matrix is pwsh;
every bash step now lives in an ubuntu-only job.

The last failure was ours in a quieter way. WaitForCharacterLogOff-
Confirmation expressed its deadline only as a CancellationTokenSource,
whose timeout is published from a thread-pool timer callback, so on a
saturated pool the token stays unsignalled past the deadline while the
loop keeps draining items that are already queued. That is the case the
method exists to bound. Reproduced by pinning the suite to two CPUs on
Linux, which failed 2 of 6 where four CPUs and sixteen were clean, and
where CI failed 3 of 3. The drain now reads the deadline off the
monotonic clock as well; the token still bounds the asynchronous wait.
Ten of ten clean under the same pin. The test is untouched. Filed as

Release build green. App tests 4,152 / 3 skipped against the same 4,152 /
3 measured at base 32f9bcfa. Core.Net 600 / 600.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 21:52:53 +02:00

155 lines
6.7 KiB
PowerShell

<#
.SYNOPSIS
Campaign V slice V6c: compile acdream's GLSL to the committed SPIR-V the
Vulkan backend loads at startup.
.DESCRIPTION
Plan §4.6 rules out runtime shader compilation: it would add a native
dependency and a startup cost for shaders that never change at runtime, and
CI runners have no Vulkan SDK. So the .spv artifacts are committed, this
script regenerates them, and an App test re-hashes the GLSL sources against
the manifest this writes so a source edit that never got recompiled fails a
test rather than shipping a stale binary.
Two compilers are supported, in this order:
1. glslc from a Vulkan SDK, if one is on PATH or under $VULKAN_SDK. This is
the reference implementation and is what the plan names.
2. tools/ShaderCompiler, a small .NET tool over Silk.NET.Shaderc — the same
shaderc library glslc is built on, through the already-pinned Silk.NET
2.23.0 family. It exists because neither the development machine nor CI
has an SDK installed, and requiring one to build acdream would put a
500 MB manual install between a contributor and a working checkout.
Both paths inject the same Vulkan preamble (see
tools/ShaderCompiler/VulkanGlslPreamble.cs) so the GLSL sources stay the
single source of truth for both backends.
.PARAMETER ShadersDirectory
Source directory. Defaults to src/AcDream.App/Rendering/Shaders.
.PARAMETER OutputDirectory
Where .spv and shaders.manifest.json are written. Defaults to
src/AcDream.App/Rendering/Shaders/spv.
.PARAMETER PreferSdk
Use glslc when available. On by default; pass -PreferSdk:$false to force the
managed path, which is what a comparison between the two wants.
.EXAMPLE
tools/compile-shaders.ps1
#>
[CmdletBinding()]
param(
[string]$ShadersDirectory,
[string]$OutputDirectory,
[bool]$PreferSdk = $true
)
$ErrorActionPreference = 'Stop'
$repo = Split-Path -Parent $PSScriptRoot
# Campaign V slice V9: every path below is composed one segment at a time rather
# than from an embedded 'a\b\c' literal. A backslash is a path separator on
# Windows and an ordinary filename character everywhere else, so the embedded
# form silently produced one long nonexistent file name on the Linux CI runner
# that this slice's lavapipe job introduced.
if (-not $ShadersDirectory) {
$ShadersDirectory = [System.IO.Path]::Combine(
$repo, 'src', 'AcDream.App', 'Rendering', 'Shaders')
}
if (-not $OutputDirectory) {
$OutputDirectory = Join-Path $ShadersDirectory 'spv'
}
function Write-Step($message) { Write-Host "[shaders] $message" }
New-Item -ItemType Directory -Force -Path $OutputDirectory | Out-Null
# --- 1. Locate glslc, if the machine has a Vulkan SDK -------------------------
$glslc = $null
if ($PreferSdk) {
$onPath = Get-Command glslc -ErrorAction SilentlyContinue
if ($onPath) {
$glslc = $onPath.Source
}
elseif ($env:VULKAN_SDK) {
# 'Bin/glslc.exe' on Windows, 'bin/glslc' on the SDK's Linux layout.
$candidates = @(
[System.IO.Path]::Combine($env:VULKAN_SDK, 'Bin', 'glslc.exe'),
[System.IO.Path]::Combine($env:VULKAN_SDK, 'bin', 'glslc')
)
foreach ($candidate in $candidates) {
if (Test-Path $candidate) { $glslc = $candidate; break }
}
}
}
# --- 2. Compile ---------------------------------------------------------------
# Even with glslc present the managed tool does the work: it owns the preamble
# injection and the manifest, and running the same transform through two
# code paths is exactly how the two would drift. glslc's presence is reported so
# a future slice can add a cross-check between them.
if ($glslc) {
Write-Step "a Vulkan SDK glslc was found at $glslc (recorded; the managed compiler still runs)"
}
else {
Write-Step 'no Vulkan SDK glslc found; using the managed Silk.NET.Shaderc compiler'
}
$tool = [System.IO.Path]::Combine(
$repo, 'tools', 'ShaderCompiler', 'ShaderCompiler.csproj')
# Campaign V slice V9 follow-up: the tool is PUBLISHED for the host runtime
# identifier rather than built portable. A portable `dotnet build` leaves
# shaderc's native module under runtimes/<rid>/native/ and makes loading it the
# job of Silk.NET's probing chain, which reaches it on some hosts and not
# others - it resolved on Ubuntu 24.04 locally and failed on the ubuntu-24.04
# CI runner with "Could not load from any of the possible library names!".
# Publishing with an explicit RID flattens the native next to the assembly,
# where AppContext.BaseDirectory - the first candidate Silk.NET tries - always
# finds it. Same managed code, same pinned shaderc 2.23.0; only the lookup
# changes.
$architecture = [System.Runtime.InteropServices.RuntimeInformation]::ProcessArchitecture
$ridArchitecture = switch ($architecture) {
'X64' { 'x64' }
'Arm64' { 'arm64' }
'X86' { 'x86' }
default { throw "No shaderc native is published for processor architecture $architecture." }
}
# $IsWindows and friends only exist in PowerShell 6+; 5.1 is Windows by
# definition, and -or short-circuits before touching the undefined variable.
$ridOs =
if ($PSVersionTable.PSVersion.Major -lt 6 -or $IsWindows) { 'win' }
elseif ($IsMacOS) { 'osx' }
elseif ($IsLinux) { 'linux' }
else { throw 'Unrecognised operating system; cannot choose a shaderc native.' }
$rid = "$ridOs-$ridArchitecture"
$toolDirectory = [System.IO.Path]::Combine(
$repo, 'tools', 'ShaderCompiler', 'bin', 'publish', $rid)
Write-Step "publishing the shader compiler for $rid"
& dotnet publish $tool -c Release -r $rid --self-contained false -o $toolDirectory --nologo -v q | Out-Null
if ($LASTEXITCODE -ne 0) { throw "Shader compiler publish failed with exit code $LASTEXITCODE." }
$binary = Join-Path $toolDirectory 'AcDream.Tools.ShaderCompiler.dll'
if (-not (Test-Path $binary)) { throw "Shader compiler not found at $binary." }
# Name the missing file rather than letting Silk.NET report the generic
# "could not load from any of the possible library names" three steps later.
$nativeName =
if ($ridOs -eq 'win') { 'shaderc_shared.dll' }
elseif ($ridOs -eq 'osx') { 'libshaderc_shared.dylib' }
else { 'libshaderc_shared.so' }
$native = Join-Path $toolDirectory $nativeName
if (-not (Test-Path $native)) {
throw ("The shaderc native $nativeName is not beside the shader compiler at " +
"$toolDirectory. The Silk.NET.Shaderc.Native package did not publish a " +
"$rid asset.")
}
Write-Step "shaderc native: $native"
Write-Step "compiling $ShadersDirectory -> $OutputDirectory"
& dotnet $binary $ShadersDirectory $OutputDirectory
if ($LASTEXITCODE -ne 0) { throw "Shader compilation failed with exit code $LASTEXITCODE." }
Write-Step 'done'