428 lines
28 KiB
Markdown
428 lines
28 KiB
Markdown
# Test-quality audit
|
|
|
|
Status: **complete at the recorded baseline**.
|
|
|
|
## Baseline
|
|
|
|
- 1,260 tracked C# files under `tests/`, totaling 402,131 lines.
|
|
- 12 primary test projects, plus fixture/helper projects.
|
|
- The reproducible per-project Release total is currently 14,747 passes and 77
|
|
reported skips. One additional duplicate-ID theory row is silently dropped
|
|
by xUnit discovery. The official all-solution process does not currently
|
|
complete reliably because of the launcher deadlock described below.
|
|
- The Release build emits 26 warnings, all in test projects. These include
|
|
nullable-flow problems, dead fixture fields, xUnit assertion-style findings,
|
|
and a duplicate `InlineData` warning.
|
|
|
|
A clean `Rebuild` classifies those 26 as: seven CS8602; three each CS0649,
|
|
CS8600, CS8604, and CS8767; two each xUnit2013 and xUnit2017; and one each
|
|
CS8625, xUnit1025, and xUnit2000. An incremental build can misleadingly print
|
|
zero because no test compiler/analyzer target reruns; the warning baseline must
|
|
therefore come from a clean/rebuild gate.
|
|
|
|
## Executable baseline by test assembly
|
|
|
|
| Assembly | Passed | Skipped | Notes |
|
|
|---|---:|---:|---|
|
|
| App | 5,510 | 76 | installed-DAT/GPU/manual/regression groups collapse into dynamic skips; many other gated facts silently no-op |
|
|
| Bake | 21 | 0 | completed |
|
|
| CLI | 4 | 0 | completed |
|
|
| Content | 154 | 0 | completed |
|
|
| Core.Net | 1,004 | 0 | three disabled live-network facts are reported as passed |
|
|
| Core | 4,797 | 1 | plus one duplicate-ID theory row dropped during discovery; many absent-DAT facts silently no-op |
|
|
| Headless | 166 | 0 | completed |
|
|
| Launcher.Core | 338 | 0 | passes alone in 48 s; deadlocks in full run |
|
|
| Launcher | 67 | 0 | completed |
|
|
| Platform | 4 | 0 | completed |
|
|
| Runtime | 1,756 | 0 | completed |
|
|
| UI.Abstractions | 926 | 0 | completed |
|
|
|
|
## Review criteria
|
|
|
|
Every test source file and parameter/data source will be checked for:
|
|
|
|
- name/behavior/assertion agreement;
|
|
- a meaningful regression signal rather than successful execution only;
|
|
- independence from reimplemented production logic;
|
|
- redundancy with stronger tests;
|
|
- stable public behavior versus implementation detail;
|
|
- deterministic time, ordering, filesystem, culture, and environment handling;
|
|
- skip/manual/live-DAT intent and discoverability;
|
|
- correct layer/project placement;
|
|
- fixtures that can actually fail for the behavior named;
|
|
- data rows that exercise distinct boundaries rather than inflating counts.
|
|
|
|
The all-file pass combines syntax-tree method inventory with a per-file signal
|
|
matrix (attributed methods, assertions/exception oracles, early returns,
|
|
environment/asset gates, sleeps/delays, source-text reads, diagnostic naming,
|
|
and helper-local assertions). It is followed by manual body/helper review for
|
|
every exception class identified by that matrix. The project totals below make
|
|
the breadth visible; counts are lexical triage signals, not quality scores:
|
|
|
|
| Test area | Files | Attributed methods | `Assert.*` calls | empty-return sites | environment reads | sleep/delay sites | file-reading heuristic |
|
|
|---|---:|---:|---:|---:|---:|---:|---:|
|
|
| App | 515 | 4,812 | 20,997 | 121 | 73 | 10 | 43 |
|
|
| Bake | 5 | 19 | 91 | 1 | 1 | 0 | 0 |
|
|
| CLI | 1 | 4 | 19 | 0 | 0 | 0 | 1 |
|
|
| Content | 29 | 134 | 716 | 25 | 6 | 1 | 0 |
|
|
| Core.Net | 115 | 867 | 3,467 | 30 | 15 | 21 | 0 |
|
|
| Core | 398 | 3,398 | 8,576 | 70 | 1 | 9 | 6 |
|
|
| Headless | 18 | 140 | 806 | 5 | 0 | 5 | 4 |
|
|
| Launcher.Core | 27 | 232 | 989 | 19 | 0 | 13 | 10 |
|
|
| Launcher | 7 | 38 | 261 | 0 | 0 | 4 | 1 |
|
|
| Platform | 2 | 4 | 17 | 0 | 0 | 0 | 0 |
|
|
| Runtime | 91 | 1,344 | 8,601 | 64 | 0 | 1 | 3 |
|
|
| UI.Abstractions | 46 | 462 | 1,228 | 1 | 0 | 5 | 2 |
|
|
|
|
Fixture/helper projects were reviewed with their consumers and contain no
|
|
attributed tests. An assertion count includes helper assertions and repeated
|
|
parameterized-oracle calls; it is not an executable-case count. The final
|
|
column is deliberately broad (it also sees fixtures and result files); T-012's
|
|
manual follow-up is the exact 30-file production-source-reading subset.
|
|
|
|
## Skip and conditional-execution inventory
|
|
|
|
| Source | Declared reason | Initial concern |
|
|
|---|---|---|
|
|
| `AcDream.App.Tests/UI/Layout/CharacterManagementLiveDatTests.cs:373-383` | installed-DAT/probe gate | dynamic skip behavior and release discoverability |
|
|
| `AcDream.App.Tests/UI/Layout/ChatLayoutFixtureGenerator.cs:29` | manual fixture generator | generator represented as a skipped test |
|
|
| `AcDream.App.Tests/UI/Layout/RadarLayoutFixtureGenerator.cs:14` | manual fixture generator | generator represented as a skipped test |
|
|
| `AcDream.App.Tests/Rendering/TowerAscentReplayTests.cs:206` | unresolved issue #119 residual | permanent red test hidden as skip |
|
|
| `AcDream.Core.Tests/Conformance/PvsConformanceTests.cs:22` | “P0 scaffold” for future capture | incomplete scaffold counted in the suite |
|
|
|
|
The custom `InstalledDatFactAttribute` is used by 73 methods. It reports a real
|
|
xUnit skip unless `ACDREAM_PROBE_LIVE_MOUNT=1`, but its single LA8-specific
|
|
reason is inherited by unrelated character creation, map, tooltip, and other
|
|
installed-DAT suites. In contrast, 294 attributed methods across 107 files
|
|
contain method-level empty `return` statements. The syntax inventory divides
|
|
the 314 return sites as follows (a method can contain more than one category):
|
|
|
|
| Gate type | Return sites | Methods | Files | Result when gate is unavailable |
|
|
|---|---:|---:|---:|---|
|
|
| DAT/package/fixture | 251 | 248 | 85 | **pass**, without exercising the named behavior |
|
|
| opt-in environment variable | 23 | 23 | 9 | **pass**, without exercising the named behavior |
|
|
| operating system | 21 | 19 | 11 | pass on the unsupported OS |
|
|
| other condition | 19 | 18 | 12 | varies; several suppress missing expected data |
|
|
|
|
The first two groups are not included in xUnit's 77 reported skips. Therefore
|
|
the executable totals above describe discovered and completed cases, not the
|
|
number of named contracts that were actually exercised.
|
|
|
|
## Confirmed test-quality findings
|
|
|
|
### T-001 — Full-suite launcher test exposes a production lock inversion
|
|
|
|
`LauncherProcessSupervisorTests.ANullStderrLogPathBehavesExactlyAsBeforeForBothChildProcessKinds`
|
|
is not itself low-value, but its unbounded `using`-scope disposal exposed a
|
|
real production deadlock. The managed stack proves a supervisor-lock / Process-
|
|
lock inversion between `LauncherProcessSupervisor.Dispose` and the process exit
|
|
callback. The test needs a deterministic race harness after the production
|
|
fix; the current timing-dependent version alternates between passing in 48 s
|
|
and hanging indefinitely.
|
|
|
|
**Post-baseline resolution checkpoint (2026-08-18, `0a934cf5`):**
|
|
`DisposeAllowsAnAlreadyCapturedExitCallbackToComplete` is now the deterministic
|
|
race harness. Explicit barriers make the fake child capture the exit delegate,
|
|
release it from inside disposal, and wait for the callback to return. The old
|
|
lock shape fails boundedly after five seconds; the fixed shape passed 25/25
|
|
fresh-process repetitions, all 22 supervisor tests, Launcher.Core 339/339, and
|
|
two serialized complete-solution runs of 14,748 passes / 77 skips. The original
|
|
real-process null-stderr test remains as end-to-end coverage rather than the
|
|
only accidental race trigger.
|
|
|
|
### T-002 — One physics assertion is a literal tautology
|
|
|
|
`Issue265SteepSlopeCaptureBisectTests.cs:920` asserts
|
|
`Assert.Equal(newModelVelocityBeforeToggle.Z > 0.01f,
|
|
newModelVelocityBeforeToggle.Z > 0.01f)`. It cannot fail and therefore proves
|
|
nothing about the comparison described by the surrounding comment. This is a
|
|
genuinely useless assertion inside an otherwise evidence-oriented test. It is
|
|
already recorded as open issue #342.
|
|
|
|
### T-003 — Duplicate theory data is silently discarded
|
|
|
|
`MotionInterpreterTests.cs:491-497` supplies both
|
|
`MotionCommand.Crouch` and literal `0x41000012u`; those values are identical.
|
|
xUnit emits duplicate case ID `6a4edd...` and skips the second row during
|
|
discovery. The test name is accurate, but the duplicate row is redundant and
|
|
the reported suite total does not reveal it. The build's xUnit1025 warning and
|
|
issue #228 already acknowledge this.
|
|
|
|
### T-004 — Test warnings include dead fixture state and contract mismatches
|
|
|
|
The current build reports 26 warnings. Examples requiring source-level review
|
|
include never-assigned `BeginTurnBlocked`/`BeginTurnUnblocked` fields in
|
|
`RemoteChaseEndToEndHarnessTests.cs:130-131`, nullable dereferences in installed-
|
|
DAT inspection tests, and nullability-mismatched fake implementations in App
|
|
composition tests. These are not all equivalent, but a release gate that
|
|
normalizes them as an old fixed warning count makes new warning regressions
|
|
invisible.
|
|
|
|
### T-005 — At least 52 tests exercise an unreachable UI presentation stack
|
|
|
|
The `IPanelRenderer` widget/menu tests and the ChatPanel layout/input/focus
|
|
tests are internally meaningful, but no production `IPanelRenderer` or
|
|
`IPanelHost` exists and no shipping code constructs `ChatPanel`, `DebugPanel`,
|
|
or `VitalsPanel`. The former ImGui implementation was deleted. These tests are
|
|
therefore obsolete release tests rather than tautologies: they can detect
|
|
changes in compiled dead code, but cannot protect user-visible retained UI.
|
|
They should be removed with that abandoned stack or moved behind a clearly
|
|
owned compatibility contract if the panel API is intentionally supported.
|
|
|
|
### T-006 — The installed-DAT attribute gives 73 unrelated tests one misleading skip identity
|
|
|
|
`InstalledDatFactAttribute` is declared inside
|
|
`CharacterManagementLiveDatTests.cs:367-383` and always says “installed-DAT
|
|
LA8 gate.” Repository-wide attribute inspection finds 73 methods using it,
|
|
including character creation and other UI/DAT areas unrelated to LA8. The
|
|
attribute does correctly produce a reported xUnit skip, but the reason does not
|
|
identify the prerequisite or contract of the test that was skipped. Split the
|
|
fixture capability from test-specific skip messages, and publish installed-DAT
|
|
gate results by suite rather than as one campaign-era bucket.
|
|
|
|
### T-007 — 271 gated facts report success without exercising their contract
|
|
|
|
A syntax-tree scan of every attributed method found 248 methods across 85
|
|
files that empty-return when DATs, a prepared package, or a replay fixture are
|
|
absent, plus 23 methods across nine files that empty-return unless an opt-in
|
|
environment variable is set. Examples include the three `LiveHandshakeTests`
|
|
(`:38`, `:115`, `:265`), whose own comment calls the return a skip although
|
|
xUnit reports all three as passed; 17 UI live-mount/powerbar probes; and broad
|
|
App/Core rendering and physics suites guarded by `datDir is null`. The pattern
|
|
also appears in all three Bake determinism facts. These names look like normal
|
|
regression contracts and their passes are included in headline totals. Use a
|
|
real skip/fail mechanism, group them under explicit installed-DAT/package/live
|
|
traits, and make release reporting state exactly which group ran.
|
|
|
|
### T-008 — Several diagnostic “tests” are deliberately incapable of failing
|
|
|
|
The whole `SmokeTest.TestProject_IsWired` body is `Assert.True(true)`.
|
|
`CellarLipWedgeTests.cs:99,305,331`,
|
|
`CellarUpTrajectoryReplayTests.cs:315,372`, and
|
|
`DoorCollisionApparatusTests.cs:238` end diagnostic output with an always-true
|
|
assertion; comments explicitly say some always pass. `GpuContractTests.cs:257`
|
|
compares the same enum constant to itself inside an otherwise useful test.
|
|
These are tools disguised as release tests. Move output-only apparatus to an
|
|
explicit diagnostic command/trait, delete the wiring smoke test, and replace
|
|
the useful diagnostics with assertions over the observed invariant.
|
|
|
|
### T-009 — Double-click tests use wall-clock sleeps instead of a controllable clock
|
|
|
|
`InputDispatcherDoubleClickTests.cs:53,73,89,106,108` sleeps for 10 or 600 ms
|
|
to cross real timing boundaries. This adds at least 640 ms to a small unit-test
|
|
class and leaves its outcome dependent on scheduler timing because production
|
|
`InputDispatcher` owns the clock. Inject `TimeProvider` (or a narrow monotonic
|
|
clock contract) and advance fake time deterministically.
|
|
|
|
### T-010 — Two entire tests have no runtime contract to protect
|
|
|
|
`SmokeTest.TestProject_IsWired` is the literal tautology described in T-008.
|
|
`ChaseCameraTests.ImplementsICamera` merely assigns a `ChaseCamera` to an
|
|
`ICamera` variable and calls `ToString`; compilation already proves the stated
|
|
interface relationship, and the call has no assertion or behavioral signal.
|
|
Both tests are useless as maintained release cases. The compile-only interface
|
|
check should be deleted or replaced by a named interface behavior test.
|
|
|
|
### T-011 — 51 release-suite methods are output-only diagnostic apparatus
|
|
|
|
A method-body and same-file helper scan found 51 attributed methods across 27
|
|
files that have no assertion, exception expectation, or other failure signal;
|
|
their observable contract is limited to diagnostic output. This count excludes
|
|
legitimate explicit no-throw cases such as no-op disposal, empty lifecycle
|
|
hooks, and secure/loopback transport construction. It also excludes tests whose
|
|
assertions live in a helper that the scanner can resolve.
|
|
|
|
Many names honestly say `Probe`, `Dump`, `Diagnostic`, `Inspection`, or
|
|
`Characterize`; these are useful investigation programs in the wrong execution
|
|
surface, not regression tests. The more serious naming problem is the subset
|
|
that reads like a behavioral contract—especially the facility-hub scenarios,
|
|
stair/camera retention sweeps, ambient-slot existence tests, production-
|
|
emission replication, and drawn-polygon comparisons—while only printing the
|
|
calculated value. Most DAT-backed examples also combine with T-007: they pass
|
|
without output when their local fixture is absent.
|
|
|
|
Exact inventory (method names in one cell belong to the named file):
|
|
|
|
| Test file | Output-only attributed methods |
|
|
|---|---|
|
|
| `App.Tests/Rendering/CornerFloodReplayTests.cs` | `Scratch_ReciprocalPrimitive_SyntheticPair` |
|
|
| `App.Tests/Rendering/Issue131SetupProbeTests.cs` | `Diagnostic_LookInFlood_AdmitsHallPorchFromCottage`; `Diagnostic_DumpOutstageCandidateSetups` |
|
|
| `App.Tests/Rendering/Issue176177FacilityHubFloodReplayTests.cs` | `ScenarioB_StairDescent_RampCellRetention`; `ScenarioC_CorridorSeamGazeSweep_Bistability`; `ScenarioE_RootLagWindow_ForwardChainRetention`; `ScenarioD_CorridorWalk_PerStepChurn` |
|
|
| `App.Tests/Rendering/Issue177StairDescentCameraFloodTests.cs` | `StairCellComposition_ShellVsStatics`; `RealStaircase_FineYawZoomSweep_FindKnifeEdge`; `FloodDepthFrom015E_VsRetail26`; `StaircaseSweep_EyeClearanceFromCeilingPortal`; `ParkedYawZoomSweep_StairAdmission`; `Descent_RealCameraSweep_StairCellRetention` |
|
|
| `App.Tests/Rendering/Issue181VisFlapReplayTests.cs` | `Diagnostic_FlappingCellViewRegion_SliverOrLarge` |
|
|
| `App.Tests/Rendering/Issue181WallPressEquilibriumTests.cs` | `Diagnostic_WallPressedCamera_EyeWanderAndViewerCellStability` |
|
|
| `App.Tests/Rendering/TowerAscentReplayTests.cs` | `Diagnostic_TopOfStairs_GateByGate` |
|
|
| `App.Tests/UI/Layout/FaPanelSlotProbeTests.cs` | `ProbePanelSlotTable` |
|
|
| `App.Tests/UI/Layout/MapHousePanelSlotProbeTests.cs` | `ProbeMapHousePanelSlot` |
|
|
| `App.Tests/UI/Layout/OptionsPanelLiveMountProbeTests.cs` | `ProbeLiveMountShapes`; `ProbeMenuPopupSizingAndTextStyle`; `ProbeFilterLabelHome` |
|
|
| `App.Tests/UI/Layout/PowerbarLayoutProbeTests.cs` | `ProbePowerbarAuthoredStrings`; `ProbeSecureTradeLayout`; `ProbeTotalItemsTemplate` |
|
|
| `App.Tests/UI/Layout/SpewBoxLayoutDumpDiagnosticTests.cs` | `SweepInstalledLayoutDescs_ForSpewBoxElementClass` |
|
|
| `Core.Tests/Audio/EnvCellSoundEmitterInventoryTests.cs` | `PortalDat_SetupsWithAmbientSlotSoundTables`; `SoundTables_WithAmbientSlots_ExistForWireBinding` |
|
|
| `Core.Tests/Conformance/CottageDoorwayCharacterizationTests.cs` | `Characterize_CottageNeighborhood_PrintStructure`; `Characterize_Doorway_FindInteriorPoints` |
|
|
| `Core.Tests/Conformance/DungeonLandblockDatProbeTests.cs` | `Probe_Dungeon0125_vs_Holtburg_A9B4` |
|
|
| `Core.Tests/Conformance/HoltburgTorchFalloffProbeTests.cs` | `Dump_Holtburg_StaticLight_Falloffs` |
|
|
| `Core.Tests/Conformance/Issue113DoorVanishDiagnosticTests.cs` | `DumpHoltburgBuildings_OrphanGeometry`; `ReplicateProductionEmission_OnPortalFills`; `DumpPortalFillSurfaceTypes`; `DumpControls_HallAndCottage` |
|
|
| `Core.Tests/Conformance/Issue113PhantomStairsDumpTests.cs` | `Dump_Cell104_ExteriorPortalPlane_Vs_GapPoint`; `DumpAAB3_Watchtower_TopDownMap`; `DumpHallModel_PolyFlagHistogram` |
|
|
| `Core.Tests/Conformance/PvsConformanceTests.cs` | `Pvs_CottageInterior_MatchesRetailCellDrawList` (also explicitly skipped and empty) |
|
|
| `Core.Tests/Conformance/ThresholdDivergenceDiagnosticTests.cs` | `Diagnose_ThresholdTransitions` |
|
|
| `Core.Tests/Physics/CameraCornerSealReplayTests.cs` | `Diagnostic_DispatchTrace_LeakPath_vs_Controls` |
|
|
| `Core.Tests/Physics/DoorSetupGfxObjInspectionTests.cs` | `HoltburgCottage_CellPortals_DatInspection`; `HoltburgLandblockStatics_DatInspection` |
|
|
| `Core.Tests/Physics/Issue137CorridorSeamInspectionTests.cs` | `CorridorSeam_FindPolygonMatchingLiveHit` |
|
|
| `Core.Tests/Physics/Issue186ConnectorCellGeometryInspectionTests.cs` | `Dump_ConnectorCells_ShellAndCollision` |
|
|
| `Core.Tests/Physics/Issue188FadingDoorMotionTableInspectionTests.cs` | `Reflect_DatReaderWriter_HookTypes`; `Dump_PedestalWeakSpot_MotionTable_HookContents` |
|
|
| `Core.Tests/Rendering/Issue176177DungeonSeamInspectionTests.cs` | `CorridorNeighborhood_CoplanarOverlappingDrawnPolyPairs`; `UnderHall_DrawnPolys_SurfaceColors`; `CellVertexNormals_SmoothOrFaceted_Dump` |
|
|
| `Core.Tests/Rendering/Issue93TownNetworkFountainRoomLightInspectionTests.cs` | `StaticObjects_SetupLightsCount_Dump` |
|
|
|
|
Move these to a separately invoked diagnostic tool/trait with artifact output,
|
|
or give each a stable oracle and assertion before retaining it in release test
|
|
counts. Contract-like names must not remain green when no comparison occurs.
|
|
|
|
### T-012 — Temporary source-text freezes became a second implementation specification
|
|
|
|
Thirty App test files locate the repository, read production `.cs` files, and
|
|
assert their literal contents or ordering. Across those files there are 560
|
|
`Assert.Contains`/`DoesNotContain`/`Matches` calls and 86 calls to custom
|
|
`AssertAppearsInOrder` helpers (each commonly checks many fragments), spread
|
|
through files containing 219 facts. Not all 219 facts use source text, and a
|
|
few whole-tree negative dependency scans are valuable architecture guards; the
|
|
problem is the large exact-fragment subset.
|
|
|
|
The clearest example is `GameWindowSlice8BoundaryTests`: its class summary
|
|
calls it a “Temporary source-shape freeze” whose assertions will be replaced by
|
|
functional tests as owners are extracted. It still contains nine facts, 73
|
|
literal fragment assertions, and 31 ordering-helper calls after the documented
|
|
Slice J/K ownership migrations. `UpdateFrameOrchestratorTests` contains another
|
|
85 literal fragment assertions and 13 ordering calls. Composition and Runtime-
|
|
ownership files repeat exact field names, constructor spellings, local-variable
|
|
names, and call text.
|
|
|
|
These cases are not useless: they did protect mechanical campaign cutovers.
|
|
They are now wrongly maintained as durable behavioral tests. Harmless renames,
|
|
formatting, extraction, or equivalent refactoring can break them, while a
|
|
semantic error that retains the expected text can pass. Keep structural rules
|
|
that encode real layer boundaries, preferably through project references,
|
|
reflection, Roslyn syntax/semantic analysis, or architecture-test tooling.
|
|
Retire campaign freezes once the named functional contract exists, and avoid
|
|
making literal method bodies a parallel architecture source of truth.
|
|
|
|
### T-013 — A retry-stability assertion compares the controller only with itself
|
|
|
|
`RuntimeLocalPlayerFirstEntryStateTests.RetryAtAwaitingActivationNeverRecommitsPublicationOrDuplicatesTheBody`
|
|
correctly captures the first `PhysicsBody` outside its three-iteration retry
|
|
loop and compares every later body with it. The adjacent controller assertion
|
|
at line 438 instead evaluates
|
|
`Assert.Same(fixture.Movement.Controller, fixture.Movement.Controller)` inside
|
|
each iteration. It can at most prove that two immediate property reads return
|
|
the same reference; it never compares the controller across retries as the
|
|
surrounding contract implies. Capture the first controller alongside `body`
|
|
and compare subsequent iterations to that reference.
|
|
|
|
### T-014 — Seven open load-sensitive tests make a green full run non-repeatable
|
|
|
|
The repository's own open issue ledger names seven distinct intermittent
|
|
tests; all passed in this audit's single per-project baseline, so none should be
|
|
silently ignored as “the known flake”:
|
|
|
|
| Issue | Test | Recorded mechanism/state |
|
|
|---|---|---|
|
|
| #302 | `PortalProjectionTests.ClipToRegion_FrameOwnedStore_ReusesExactResultArray` | exact per-thread allocation count; observed about one failure in six isolated App runs |
|
|
| #308 | `NakEmissionTests.LossSoak_TwoPercentBidirectional_ZeroMessageLoss_LedgersConverge` | virtual transport clock but real 60-second `DateTime.UtcNow` deadline and `Thread.Sleep` convergence loop |
|
|
| #321 | `DatSoundCacheTests.GetWave_ConcurrentSameId_PublishesOneCanonicalWaveAndDecodesOnce` | unresolved test-harness versus production cache race under load |
|
|
| #336 | `RuntimeCollisionReportingStateTests.WarmedSteadyContactRefreshDoesNotAllocate` | exact zero-allocation assertion; measured 2,944 bytes once under full load |
|
|
| #340 | `StreamingWorkBudgetTests.DestinationAndEmptyUnloadPriorityNeverBypassPublicationBudget` | load-sensitive, deterministic in isolation; root cause not recorded |
|
|
| #346 | `PortalProjectionTests.ProjectToClipLease_ReusesPooledWorkWithoutResultArrays` | second exact-allocation assertion in the same file; repeated full-load failures |
|
|
| #402 | `LandblockBuildFactoryTests.Build_UsesTheSuppliedSharedReaderGate` | roughly two failures in five repeated full App runs; shared-state/timing cause unresolved |
|
|
|
|
The #308 XML documentation calls the 10,000-message soak a virtual-clock test,
|
|
but its convergence actually has two real-time loops and a fixed sleep. The
|
|
names of the other six describe real contracts; they are not useless, but their
|
|
measurement mechanisms are unreliable or potentially expose real races. They
|
|
need deterministic clocks/schedulers, allocation warmup/bounds, and captured
|
|
race details before the suite can be a release gate.
|
|
|
|
### T-015 — Four explicit skips are tools, an empty scaffold, and a hidden known failure
|
|
|
|
Beyond the 73 custom installed-DAT skips, the remaining four reported skips
|
|
are not ordinary unavailable-platform tests:
|
|
|
|
- `ChatLayoutFixtureGenerator.GenerateChatFixture` and
|
|
`RadarLayoutFixtureGenerator.GenerateRadarFixture` are manual artifact
|
|
generators permanently represented as skipped facts. They have no regression
|
|
oracle and belong in a documented tool command.
|
|
- `PvsConformanceTests.Pvs_CottageInterior_MatchesRetailCellDrawList` is an
|
|
empty future scaffold. It is genuinely useless until a captured oracle and
|
|
implementation exist; a TODO/issue is clearer than a permanent green-suite
|
|
skip.
|
|
- `TowerAscentReplayTests.RetailShouldSeeTwoCells_AtFailureWindow` is a known
|
|
#119 residual deliberately disabled “until the fix.” A skipped failing
|
|
contract cannot guard against worsening or resolution; track the expected
|
|
current behavior separately, or make it an explicit non-blocking known-
|
|
failure lane with ownership and expiry.
|
|
|
|
Together with T-006, these explain all 77 reported skips. The release report
|
|
should distinguish unavailable external prerequisites from manual generators,
|
|
unfinished tests, and known product failures.
|
|
|
|
## Tests whose maintained names overclaim their behavior
|
|
|
|
This is the actionable naming subset, not a demand to rename every
|
|
campaign-prefixed test. A name is listed when its stated outcome is materially
|
|
stronger than its oracle:
|
|
|
|
| Test | Why the name is wrong or misleading | Classification |
|
|
|---|---|---|
|
|
| `PvsConformanceTests.Pvs_CottageInterior_MatchesRetailCellDrawList` (`:23`) | Empty body and permanently skipped; performs no comparison. | useless unfinished scaffold |
|
|
| `SmokeTest.TestProject_IsWired` (`:7`) | Only `Assert.True(true)`; compilation already establishes project wiring. | useless tautology |
|
|
| `ChaseCameraTests.ImplementsICamera` (`:59`) | Assignment proves only a compile-time relationship and `ToString()` is not asserted. | useless compile-only check |
|
|
| `GpuContractTests.TheDepthStencilAttachmentFormatCarriesAStencilAspect` (`:245-257`) | Checks clear/load fields, then “verifies” the format with `Assert.Equal(GpuTextureFormat.Depth24Stencil8, GpuTextureFormat.Depth24Stencil8)`; the pass description carries no format at all. | wrongly named and missing its central oracle |
|
|
| `RuntimeLocalPlayerFirstEntryStateTests.RetryAtAwaitingActivationNeverRecommitsPublicationOrDuplicatesTheBody` (`:424-438`) | The body half is checked across retries, but the adjacent controller stability assertion compares the property with itself. | partially overclaimed |
|
|
| `EnvCellSoundEmitterInventoryTests.PortalDat_SetupsWithAmbientSlotSoundTables` (`:130`) | Inventories/prints candidates without asserting that any qualifying setup exists. | output-only diagnostic named as existence contract |
|
|
| `EnvCellSoundEmitterInventoryTests.SoundTables_WithAmbientSlots_ExistForWireBinding` (`:187`) | Prints the inventory without an existence or binding assertion. | output-only diagnostic named as existence contract |
|
|
| `Issue176177FacilityHubFloodReplayTests.ScenarioB_StairDescent_RampCellRetention` (`:101`) and the `ScenarioC`/`ScenarioD`/`ScenarioE` peers (`:143,217,242`) | Calculate and print retention/churn; never assert the named retention or bistability result. | output-only replay named as contract |
|
|
| `Issue113DoorVanishDiagnosticTests.ReplicateProductionEmission_OnPortalFills` (`:194`) | Replicates and prints emission; no expected result is asserted. | diagnostic whose verb implies verified equivalence |
|
|
| `Issue176177DungeonSeamInspectionTests.CorridorNeighborhood_CoplanarOverlappingDrawnPolyPairs` (`:247`) and `UnderHall_DrawnPolys_SurfaceColors` (`:473`) | Dump calculated pairs/colors without asserting the noun phrase in the name. | inspections named as facts |
|
|
|
|
The remaining output-only cases in T-011 generally advertise `Probe`, `Dump`,
|
|
`Diagnostic`, `Inspection`, or `Characterize`; those names are honest, but the
|
|
methods are still misplaced in the default release suite. Contract-like names
|
|
in T-011 should either gain a stable oracle or be renamed and moved with the
|
|
diagnostic apparatus.
|
|
|
|
### T-016 — The test taxonomy preserves investigation history as its primary index
|
|
|
|
Forty-seven test files begin with an `Issue###` identifier, 53 filenames contain
|
|
`Probe`, `Dump`, `Diagnostic`, `Inspection`, or `Characterization`, and 17
|
|
contain `Replay`. The descriptive suffix often makes an individual file
|
|
understandable, so the issue-prefixed group is not automatically low-value.
|
|
The maintainability problem is that issue identity and one-off evidence shape
|
|
are the primary browsing taxonomy even after a behavior becomes permanent.
|
|
This couples discovery to the 19,073-line issue archive and mixes durable
|
|
regression contracts with T-011's output apparatus. When a test graduates into
|
|
a stable contract, organize it under the owning component/behavior and retain
|
|
the issue ID in a trait or comment; keep investigation programs in a separate,
|
|
non-default diagnostic project.
|
|
|
|
### T-017 — Shown Avalonia test windows escaped their owning UI session
|
|
|
|
Four `MainWindowViewTests` called `window.Show()` but never closed the window.
|
|
One default-parallel solution run then failed during Avalonia headless cleanup:
|
|
the runner/compositor touched an object from a thread other than the one that
|
|
owned it, even though the Launcher project passed alone. The tests now close
|
|
shown windows and pump dispatcher cleanup in `finally` on the owning Avalonia
|
|
session. No product code, suite serialization, or retry was added. The focused
|
|
class passes 13/13, and three fresh default-parallel complete-solution runs pass
|
|
all 12 assemblies in 56.719, 56.321, and 58.506 seconds.
|
|
|
|
### T-018 — A live stderr assertion could disable the producer it observed
|
|
|
|
`ReadFileEventuallyContainingAsync` used `File.ReadAllTextAsync` while the
|
|
ProcessStartInfo stderr callbacks were still draining. Its read handle did not
|
|
share writes. If the final callback overlapped that short read, the capture
|
|
sink saw a sharing violation, deliberately latched into its no-throw state, and
|
|
the test waited five seconds for a line it had itself caused to be dropped. The
|
|
first complete repository-gate run exposed exactly that two-of-three-lines
|
|
failure. The helper now opens a live reader with
|
|
`FileShare.ReadWrite | FileShare.Delete`, matching the production status
|
|
tailer; the expected lines and timeout are unchanged. The test subsequently
|
|
passed 25/25 fresh-process runs and Launcher.Core passed 339/339 in the final
|
|
gate.
|