test: separate non-hermetic release lanes

This commit is contained in:
Erik 2026-08-18 10:49:22 +02:00
parent 52015f5052
commit 8f490240d4
17 changed files with 131 additions and 114 deletions

View file

@ -9,8 +9,12 @@ pwsh ./tools/run-release-gate.ps1
The command verifies that `AcDream.slnx` contains every `.csproj` under `src/`,
`tests/`, and `tools/`, performs a locked restore, builds that complete graph,
then discovers and runs every default test assembly once in a fresh Release
process. It does not retry failures. The graph currently contains 44 projects,
then discovers and runs every hermetic test in every default test assembly once
in a fresh Release process. It does not retry failures. Tests carrying an
explicit non-hermetic `Lane` trait (`InstalledDat`, `PreparedPackage`, `Live`,
or `Manual`), `Purpose=Diagnostic`, or `Status=KnownFailure` are excluded from
the hermetic total and run through their owned lane instead. The graph currently
contains 44 projects,
including all 13 maintained .NET tools; data-dependent tools are built but are
not executed as tests.
@ -64,9 +68,36 @@ are preserved for later use. Until then, the repository command above is the
authoritative gate. Focused portability or Vulkan jobs are not substitutes for
the complete gate.
Environment-dependent tests retain their current skip behavior and are counted
explicitly in the JSON/TRX report. Classifying or replacing those tests belongs
to R3, not to this gate checkpoint.
The JSON summary records the exact test filter. Environment-dependent,
diagnostic, manual, and known-failure results must be published as their own
lane and must never be added to the hermetic pass headline.
## Non-hermetic test lanes
Installed-DAT tests require an explicit opt-in and a retail DAT directory:
```powershell
$env:ACDREAM_RUN_INSTALLED_DAT_TESTS = '1'
$env:ACDREAM_DAT_DIR = 'C:\path\to\Asherons Call'
dotnet test tests/AcDream.App.Tests/AcDream.App.Tests.csproj -c Release `
--filter 'Lane=InstalledDat&Status!=KnownFailure&Purpose!=Diagnostic'
```
Regenerate all committed UI fixtures through the one comprehensive manual
generator (the former chat/radar-only generators were redundant):
```powershell
$env:ACDREAM_REGENERATE_UI_FIXTURES = '1'
$env:ACDREAM_DAT_DIR = 'C:\path\to\Asherons Call'
dotnet test tests/AcDream.App.Tests/AcDream.App.Tests.csproj -c Release `
--filter 'Lane=Manual'
```
Known failures (`Status=KnownFailure`) are never part of a green release total.
Run them explicitly with their prerequisite lane configured; a failure is
expected until the linked defect is fixed. Diagnostic apparatus
(`Purpose=Diagnostic`) likewise reports separately and does not inflate the
contract-test pass count.
## Updating dependencies