test: separate diagnostic apparatus from release gates
This commit is contained in:
Erik
parent
8f490240d4
commit
c1a905004a
33 changed files with 527 additions and 24 deletions
|
|
@ -26,6 +26,50 @@ The six-case reduction from the earlier 14,748-pass headline is understood:
|
|||
seven launcher editor-focus theory rows were consolidated into one fact that
|
||||
still executes all seven variants in a single Avalonia application session.
|
||||
|
||||
## Fresh reproducible syntax inventory
|
||||
|
||||
`tools/audit-test-inventory.ps1` now regenerates the complete attributed-method
|
||||
manifest from tracked C# test sources with Roslyn:
|
||||
|
||||
```powershell
|
||||
pwsh ./tools/audit-test-inventory.ps1
|
||||
```
|
||||
|
||||
The generated JSON lives under ignored `artifacts/test-audit/`; it is not a
|
||||
second 6.8 MB checked-in source of truth. The script and this reviewed ledger
|
||||
are durable, while paths and line numbers regenerate after every batch. The
|
||||
inventory at commit `8f490240` reports:
|
||||
|
||||
| Syntax signal | Current count |
|
||||
|---|---:|
|
||||
| Tracked/parsed C# test files | 1,256 / 1,256 |
|
||||
| Attributed test methods (not expanded theory cases) | 11,455 |
|
||||
| Methods containing at least one empty `return;` | 399 |
|
||||
| Empty-return sites | 426 |
|
||||
| Prerequisite-return candidates | 282 |
|
||||
| Mechanical output-only candidates | 47 |
|
||||
| Methods directly using `Thread.Sleep` / `Task.Delay` | 20 / 23 |
|
||||
| Methods directly reading environment variables | 47 |
|
||||
| Methods directly reading `.cs` source text | 63 |
|
||||
|
||||
The 282 prerequisite-return count is intentionally a review candidate set, not
|
||||
a claim that all 399 methods are broken. It comprises 109 App, 125 Core, 29
|
||||
Content, 12 Launcher.Core, three Core.Net, three Bake, and one Headless method.
|
||||
The classifier includes external-asset, opt-in, and platform gates; behavioral
|
||||
early exits inside callback tests remain outside that candidate set. Every
|
||||
candidate still requires source inspection because `Directory.Exists`, for
|
||||
example, can also be the behavior under test.
|
||||
|
||||
The output-only classifier follows same-file helper calls, but it cannot decide
|
||||
whether an assertion merely establishes probe setup. The reviewed diagnostic
|
||||
set is therefore 51 methods: the 47 mechanical candidates plus
|
||||
`OptionsPanelLiveMountProbeTests.ProbeLiveMountShapes`,
|
||||
`DoorSetupGfxObjInspectionTests.HoltburgCottage_CellPortals_DatInspection`, and
|
||||
the two already classified tower diagnostics. Relative to T-011's historical
|
||||
51-method table, delete the removed PVS scaffold and add
|
||||
`TowerAscentReplayTests.Diagnostic_TowerAscent_PerStepTable`; all other entries
|
||||
remain. Those 51 methods expand to 70 discovered cases (28 App, 42 Core).
|
||||
|
||||
## Exact 77-skip reconciliation
|
||||
|
||||
The latest clean-gate TRX files account for every reported skip:
|
||||
|
|
@ -83,11 +127,11 @@ identifiable in the lane report.
|
|||
| T-004 warning mismatches | resolved in R2 | Clean rebuild is zero-warning with repository-wide warnings-as-errors. |
|
||||
| T-005 unreachable panel stack | requires production reachability proof | Do not delete 52 meaningful tests until the dead presentation surface and any supported plugin compatibility promise are verified together. |
|
||||
| T-006 misleading installed-DAT reason | resolved in batch B | Shared opt-in is now `ACDREAM_RUN_INSTALLED_DAT_TESTS=1` (legacy switch retained), the reason names the lane, and all nine owners carry `Lane=InstalledDat`. |
|
||||
| T-007 271 silent passing gates | open | Re-run the syntax inventory against current source, then convert each to a hermetic fixture, explicit prerequisite skip, or non-default lane. |
|
||||
| T-007 271 silent passing gates | active; current candidate set generated | Current syntax has 282 prerequisite-return candidates across seven projects. Inspect/classify from the generated manifest; the count changed with source growth and includes platform gates, so do not mechanically rewrite all 282. |
|
||||
| T-008 incapable-of-failing diagnostics | batch A plus diagnostic lane work | Delete the literal wiring smoke test; repair the GPU contract tautology; later re-home output-only apparatus. |
|
||||
| T-009 wall-clock double-click tests | open | Introduce a behavior-preserving injectable monotonic clock and deterministic fake time. |
|
||||
| T-010 two useless cases | high-confidence cleanup batch A | Delete `SmokeTest.TestProject_IsWired` and `ChaseCameraTests.ImplementsICamera`; compilation already proves both claims. |
|
||||
| T-011 51 output-only methods | open | Validate the historical 51-method list against current source; add stable oracles or move them outside default discovery. |
|
||||
| T-011 51 output-only methods | classified in batch C | The reviewed current set is 51 methods / 70 cases. All carry `Purpose=Diagnostic`, preserving the apparatus while removing it from release pass totals. Contract-shaped names remain explicitly flagged until a stable oracle exists. |
|
||||
| T-012 source-text freezes | requires semantic replacement map | Retain whole-tree dependency rules; remove exact-text freezes only when an equivalent semantic/behavioral guard is identified. |
|
||||
| T-013 controller self-comparison | high-confidence cleanup batch A | Capture the first controller next to the first body and compare every retry with that reference. |
|
||||
| T-014 seven load-sensitive tests | active; first mechanism reproduced | `PortalProjectionTests.ClipToRegion_FrameOwnedStore_ReusesExactResultArray` reported 6,904 B only while three App test hosts ran concurrently, then passed in an isolated run. Preserve both observations and remove ambient process-allocation sensitivity rather than loosening the zero-allocation oracle. |
|
||||
|
|
@ -179,3 +223,31 @@ at 6,904 bytes while the Installed-DAT and discovery hosts ran beside it. The
|
|||
same unmodified hermetic command then passed 5,505/5,505 in isolation. This is
|
||||
evidence for removing cross-host allocation sensitivity; it is not grounds to
|
||||
weaken the assertion or claim a clean concurrent result.
|
||||
|
||||
## Batch C diagnostic classification
|
||||
|
||||
Batch C assigns the reviewed 51-method diagnostic set to
|
||||
`Purpose=Diagnostic`. No body, output, historical comment, or fixture is
|
||||
removed. This resolves the misleading release accounting without pretending
|
||||
that every investigation has a stable regression oracle.
|
||||
|
||||
The 15 contract-shaped methods called out by T-011 now have a `Diagnostic_`
|
||||
prefix: four facility-hub scenarios, six stair/camera sweeps, two ambient-slot
|
||||
inventories, production-emission replication, and two drawn-polygon reports.
|
||||
Their current output and historical method references are preserved, while the
|
||||
new names no longer imply an asserted oracle. Promoting them back into the
|
||||
hermetic lane requires an asserted expected result; R3 will not invent those
|
||||
values.
|
||||
|
||||
Batch C verification:
|
||||
|
||||
- App Release build: 0 warnings, 0 errors;
|
||||
- Core Release build, serialized after App: 0 warnings, 0 errors;
|
||||
- hermetic App: 5,479 passed / 0 skipped / 0 failed;
|
||||
- hermetic Core: 4,753 passed / 0 skipped / 0 failed; and
|
||||
- diagnostic discovery: 28 App + 42 Core = 70 cases from 51 methods.
|
||||
|
||||
An attempted parallel App/Core build collided on the shared
|
||||
`AcDream.Plugin.Abstractions` intermediate DLL. The repository gate serializes
|
||||
projects, so the authoritative verification is the two isolated clean builds
|
||||
above; no code or assertion was changed in response to that tooling collision.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue