erik/acdream
1
0
Fork 0
Code Issues Pull requests Projects Releases 6 Packages Wiki Activity Actions

test: separate non-hermetic release lanes

Browse source
This commit is contained in:
Erik 2026-08-18 10:49:22 +02:00
parent 52015f5052
commit 8f490240d4
17 changed files with 131 additions and 114 deletions
Download patch file Download diff file Expand all files Collapse all files

59
docs/reviews/2026-08-18-r3-test-truth-ledger.md
View file

@ -32,8 +32,8 @@ The latest clean-gate TRX files account for every reported skip:
| Classification | Count | Current source | R3 decision |
|---|---:|---|---|
| Installed retail DAT lane | 73 | `InstalledDatFactAttribute` across nine files | Retain the contracts, replace the campaign-specific shared reason with prerequisite- and suite-specific identity, and report this lane separately. |
| Manual fixture generation | 2 | `ChatLayoutFixtureGenerator.GenerateChatFixture`; `RadarLayoutFixtureGenerator.GenerateRadarFixture` | Remove from default discovery and expose documented explicit generator commands. |
| Installed retail DAT lane | 73 | `InstalledDatFactAttribute` across nine files | Retain the contracts, replace the campaign-specific shared reason with prerequisite- and suite-specific identity, and report this lane separately. Batch B assigns all nine classes to `Lane=InstalledDat`. |
| Manual fixture generation | 2 | `ChatLayoutFixtureGenerator.GenerateChatFixture`; `RadarLayoutFixtureGenerator.GenerateRadarFixture` | Delete both redundant generators: the existing comprehensive `RetailLayoutFixtureGenerator` already owns their chat and radar outputs. Assign that generator to the explicit Manual lane. |
| Empty future scaffold | 1 | `PvsConformanceTests.Pvs_CottageInterior_MatchesRetailCellDrawList` | Delete the test. Preserve its retail oracle and capture recipe here and in the existing P0 plan. |
| Known product failure | 1 | `TowerAscentReplayTests.TowerAscent_StaircaseStaysConeVisible_EveryStep` | Do not pretend this is unavailable coverage. Keep the oracle and move it to an explicitly reported known-failure/non-blocking lane until #119 is fixed. |
@ -58,6 +58,21 @@ assign them to the installed-DAT, prepared-package, system-font, or other
explicit prerequisite lane; a future machine may report them as additional
skips.
The lane vocabulary is deliberately orthogonal:
- `Lane` states the execution environment (`InstalledDat`, `PreparedPackage`,
`Live`, or `Manual`);
- `Purpose=Diagnostic` marks output/investigation apparatus that cannot add to
the contract-test pass count; and
- `Status=KnownFailure` keeps a named executable oracle visible without making
a green release claim.
The default gate excludes all three dimensions. An explicitly selected lane
must fail clearly when its prerequisite is missing; it may not pass through an
empty return. The 73 custom installed-DAT facts retain discovery-time skips
when their opt-in is absent so each unavailable contract and reason remains
identifiable in the lane report.
## Finding disposition
| Finding | Current disposition | Evidence / next action |
@ -67,7 +82,7 @@ skips.
| T-003 duplicate theory row | high-confidence cleanup batch A | Remove literal `0x41000012`, which is identical to `MotionCommand.Crouch`; remove the temporary analyzer suppression. |
| T-004 warning mismatches | resolved in R2 | Clean rebuild is zero-warning with repository-wide warnings-as-errors. |
| T-005 unreachable panel stack | requires production reachability proof | Do not delete 52 meaningful tests until the dead presentation surface and any supported plugin compatibility promise are verified together. |
| T-006 misleading installed-DAT reason | open | Replace the LA8-specific shared attribute reason with truthful suite/prerequisite identity. |
| T-006 misleading installed-DAT reason | resolved in batch B | Shared opt-in is now `ACDREAM_RUN_INSTALLED_DAT_TESTS=1` (legacy switch retained), the reason names the lane, and all nine owners carry `Lane=InstalledDat`. |
| T-007 271 silent passing gates | open | Re-run the syntax inventory against current source, then convert each to a hermetic fixture, explicit prerequisite skip, or non-default lane. |
| T-008 incapable-of-failing diagnostics | batch A plus diagnostic lane work | Delete the literal wiring smoke test; repair the GPU contract tautology; later re-home output-only apparatus. |
| T-009 wall-clock double-click tests | open | Introduce a behavior-preserving injectable monotonic clock and deterministic fake time. |
@ -75,8 +90,8 @@ skips.
| T-011 51 output-only methods | open | Validate the historical 51-method list against current source; add stable oracles or move them outside default discovery. |
| T-012 source-text freezes | requires semantic replacement map | Retain whole-tree dependency rules; remove exact-text freezes only when an equivalent semantic/behavioral guard is identified. |
| T-013 controller self-comparison | high-confidence cleanup batch A | Capture the first controller next to the first body and compare every retry with that reference. |
| T-014 seven load-sensitive tests | open | Reproduce and classify each mechanism; no retries or blanket tolerances. |
| T-015 four non-prerequisite skips | batch A plus lane work | Delete the PVS scaffold now; move two generators and the known failure in later focused batches. |
| T-014 seven load-sensitive tests | active; first mechanism reproduced | `PortalProjectionTests.ClipToRegion_FrameOwnedStore_ReusesExactResultArray` reported 6,904 B only while three App test hosts ran concurrently, then passed in an isolated run. Preserve both observations and remove ambient process-allocation sensitivity rather than loosening the zero-allocation oracle. |
| T-015 four non-prerequisite skips | resolved in batches A/B | PVS scaffold deleted with rationale preserved; redundant chat/radar generators deleted in favor of the comprehensive Manual lane; tower oracle is `Status=KnownFailure`. |
| T-016 historical test taxonomy | open | Rename/re-home only after each test's durable owner and oracle are established. |
| T-017 Avalonia ownership | resolved in R2 | All seven editor variants now execute in one owned application session; 11 focused passes plus the complete gate. |
| T-018 stderr observer race | resolved in R2 | Live reader shares writes/deletes; 25 focused repetitions plus the complete gate. |
@ -130,3 +145,37 @@ The count delta is intentional and classified: Core removed the two useless
compile-only cases and the skipped PVS scaffold; App gained one case by
splitting the old mixed/tautological depth-stencil test into two independently
named contracts. Core emitted no duplicate-case discovery warning.
## Preserved rationale for removals in batch B
The chat- and radar-only fixture generators were not the sole way to recreate
their committed JSON. `RetailLayoutFixtureGenerator` already includes retail
layout IDs `0x2100006F` and `0x21000074` and writes the same
`chat_2100006f.json` and `radar_21000074.json` destinations as part of its
complete 20-layout run. Deleting the two permanently skipped duplicates removes
two false test entries without losing the regeneration recipe or retail notes;
the comprehensive generator and `docs/release-gate.md` now own that workflow.
The tower staircase oracle is not unavailable or useless. Its former static
skip documents issue #119 and the exact failing steps. Batch B keeps the
executable assertion under `Status=KnownFailure`, separates two output-only
methods with `Purpose=Diagnostic`, and makes missing installed DATs a clear lane
prerequisite failure instead of a passing early return.
## Batch B verification
The lane-classification batch was verified without changing product code:
- App Release build: 0 warnings, 0 errors;
- hermetic App lane, isolated: 5,505 passed / 0 skipped / 0 failed;
- Installed-DAT contract lane without opt-in: 3 passed / 73 individually named
skips / 0 failed (the local tower DAT prerequisite was available);
- known-failure discovery: exactly one tower staircase oracle; and
- Manual discovery: exactly one comprehensive fixture generator.
One concurrent three-host verification attempt reproduced T-014 rather than
being hidden: the hermetic host failed the portal-clipping zero-allocation test
at 6,904 bytes while the Installed-DAT and discovery hosts ran beside it. The
same unmodified hermetic command then passed 5,505/5,505 in isolation. This is
evidence for removing cross-host allocation sensitivity; it is not grounds to
weaken the assertion or claim a clean concurrent result.