diff --git a/docs/reviews/2026-08-18-r3-test-truth-ledger.md b/docs/reviews/2026-08-18-r3-test-truth-ledger.md index f3abe94a..843f71ae 100644 --- a/docs/reviews/2026-08-18-r3-test-truth-ledger.md +++ b/docs/reviews/2026-08-18-r3-test-truth-ledger.md @@ -38,7 +38,7 @@ pwsh ./tools/audit-test-inventory.ps1 The generated JSON lives under ignored `artifacts/test-audit/`; it is not a second 6.8 MB checked-in source of truth. The script and this reviewed ledger are durable, while paths and line numbers regenerate after every batch. The -inventory refreshed for Batch G reports: +inventory refreshed for Batch L reports: | Syntax signal | Current count | |---|---:| @@ -50,7 +50,9 @@ inventory refreshed for Batch G reports: | Empty-return sites | 143 | | Direct / same-file-helper prerequisite-return candidates | 2 / 1 | | Mechanical output-only candidates | 2 | -| Reviewed diagnostic methods | 51 | +| Constant-truth assertion methods / sites | 0 / 0 | +| Reviewed syntactic self-comparison methods / sites | 4 / 5 | +| Reviewed diagnostic methods | 57 | | Methods directly using `Thread.Sleep` / `Task.Delay` | 15 / 23 | | Cancellable-infinite-only / elapsed-time methods | 5 / 33 | | Methods directly reading environment variables | 47 | @@ -63,15 +65,19 @@ the inventory now fails explicitly inside its selected non-hermetic lane. The remaining 143 empty returns are behavioral control flow, callback exits, or the three platform branches; they are not missing-prerequisite passes. -The output-only classifier follows same-file helper calls, but it cannot decide -whether an assertion merely establishes probe setup. The reviewed diagnostic -set is therefore 51 methods: the 47 mechanical candidates plus -`OptionsPanelLiveMountProbeTests.ProbeLiveMountShapes`, -`DoorSetupGfxObjInspectionTests.HoltburgCottage_CellPortals_DatInspection`, and -the two already classified tower diagnostics. Relative to T-011's historical -51-method table, delete the removed PVS scaffold and add -`TowerAscentReplayTests.Diagnostic_TowerAscent_PerStepTable`; all other entries -remain. Those 51 methods expand to 70 discovered cases (28 App, 42 Core). +The output-only classifier follows same-file helper calls, so an assertion used +only while constructing a probe can prevent an otherwise diagnostic method from +appearing in its two-candidate result. The reviewed `Purpose=Diagnostic` set is +therefore the authority: Batch C's 51 methods plus six artifact/report writers +reconciled in Batch L, for 57 methods and 76 discovered cases (28 App, 48 Core). +Their investigative output and historical rationale remain executable outside +the release total. + +The five syntactic self-comparisons are reviewed, not vacuous: two call a hash +twice to assert determinism, one reads the singleton twice, and two read the +same Runtime owner's properties twice to assert stable identity. The inventory +keeps these candidates visible separately from the now-zero constant-truth +assertions. ## Exact 77-skip reconciliation @@ -130,10 +136,10 @@ identifiable in the lane report. | T-005 unreachable panel stack | requires production reachability proof | Do not delete 52 meaningful tests until the dead presentation surface and any supported plugin compatibility promise are verified together. | | T-006 misleading installed-DAT reason | resolved in batch B | Shared opt-in is now `ACDREAM_RUN_INSTALLED_DAT_TESTS=1` (legacy switch retained), the reason names the lane, and all nine owners carry `Lane=InstalledDat`. | | T-007 271 silent passing gates | resolved in batch G | All 280 directly found gates plus three later-reconciled sites now fail explicitly when their selected lane lacks its prerequisite. The only three syntax candidates left are reviewed cross-platform branch false positives. | -| T-008 incapable-of-failing diagnostics | batch A plus diagnostic lane work | Delete the literal wiring smoke test; repair the GPU contract tautology; later re-home output-only apparatus. | +| T-008 incapable-of-failing diagnostics | resolved in batches A, C, and L | The literal wiring smoke test was deleted, the GPU contract tautology was repaired, and the six remaining constant-truth diagnostic assertions were removed after their output was preserved under `Purpose=Diagnostic`. | | T-009 wall-clock double-click tests | resolved in batch E | Four sleeps were replaced by a deterministic test clock behind an internal factory overload. The production factory still reads `Environment.TickCount64` exactly as before. | | T-010 two useless cases | high-confidence cleanup batch A | Delete `SmokeTest.TestProject_IsWired` and `ChaseCameraTests.ImplementsICamera`; compilation already proves both claims. | -| T-011 51 output-only methods | classified in batch C | The reviewed current set is 51 methods / 70 cases. All carry `Purpose=Diagnostic`, preserving the apparatus while removing it from release pass totals. Contract-shaped names remain explicitly flagged until a stable oracle exists. | +| T-011 output-only methods | classified in batches C and L | The reviewed current set is 57 methods / 76 cases. All carry `Purpose=Diagnostic`, preserving the apparatus while removing it from release pass totals. Contract-shaped names remain explicitly flagged until a stable oracle exists. | | T-012 source-text freezes | replacement map complete in batch I; 46 implementation-shape facts remain staged | Twelve whole-tree architecture rules and five cross-artifact contracts stay. The other 46 are campaign-era literal implementation freezes; retire each only beside its semantic/behavioral replacement, with user approval for owner/lifecycle assertions that have no equivalent yet. | | T-013 controller self-comparison | high-confidence cleanup batch A | Capture the first controller next to the first body and compare every retry with that reference. | | T-014 seven load-sensitive tests | six mechanisms repaired in batch F; one product defect classified | Virtual/wall-clock mixing, tiered-JIT allocation noise, a live controller clock, and a ThreadPool-start timing oracle were removed without weakening behavioral contracts. `DatSoundCache` #321 is a real in-flight-entry race and now runs as `Status=KnownFailure` pending a product fix. | @@ -143,6 +149,7 @@ identifiable in the lane report. | T-018 stderr observer race | resolved in R2 | Live reader shares writes/deletes; 25 focused repetitions plus the complete gate. | | T-019 remaining explicit waits | classified in batch H; nine fixed-delay negative oracles need cleanup | Five of the 38 methods are deterministic cancellation stubs, not wall-clock tests. Preserve four intentional real-time contracts and bounded integration polling; replace the nine tests that infer “still blocked” from a fixed delay with observable synchronization state. | | T-020 exact duplicate bodies/data | resolved in batch K | Four redundant executions were removed. The remaining 11 body-equivalent groups are intentionally split theories with disjoint, meaningfully named datasets; the inventory enforces zero repeated rows within a theory or across body-equivalent theories. | +| T-021 suspicious assertion shapes | resolved in batch L | The inventory enforces zero `Assert.True(true)` / `Assert.False(false)` sites and reports syntactic self-comparisons for review. All five current self-comparisons assert meaningful determinism or stable identity. | ## Preserved rationale for removals in batch A @@ -579,3 +586,47 @@ complete 44-project Release build reports zero warnings and zero errors. The no-retry complete hermetic Release gate passed 14,387/14,387 with zero skips or failures across all 12 test assemblies. The four-case reduction is exactly the three duplicate facts and one duplicate theory row removed above. + +## Batch L vacuous diagnostic assertions + +Six investigation methods still ended in an unconditional green assertion: + +- three cellar-lip trajectory/replay artifact writers in + `CellarLipWedgeTests`; +- two cellar-ascent trajectory reports in `CellarUpTrajectoryReplayTests`; and +- the installed-DAT front-approach probe in `DoorCollisionApparatusTests`. + +These were useful tools but not regression tests. Batch L adds +`Purpose=Diagnostic`, removes the six `Assert.True(true, ...)` calls, preserves +the two reports that previously existed only as assertion messages by writing +them to captured console output, and leaves the existing `%TEMP%` artifacts and +probe logs intact. No historical investigation notes, fixtures, or probe logic +were deleted. + +The durable inventory now recognizes `File.WriteAll*` / `File.AppendAll*` as +diagnostic-output signals and reports high-confidence constant-truth assertions +separately from syntactic self-comparisons. It reports zero constant-truth sites. +The five self-comparison sites were individually reviewed and retained: + +- `Hash32Tests.Calculate_IsDeterministic` and + `PacketHeaderTests.CalculateHeaderHash32_DeterministicForSameInput` execute + the calculation twice and compare the results; +- `NullCommandBusTests.Instance_IsSingleton` reads the singleton twice; and +- `RuntimeEntityObjectLifetimeTests.Owner_ConstructsOneExactDirectoryAndObjectTablePerInstance` + verifies repeated access to both owned objects returns the same instances. + +Batch L verification: + +- all six affected diagnostic methods pass, including the installed-DAT door + probe; +- the refreshed inventory reports 11,447 attributed methods, 57 diagnostic + methods, zero constant-truth assertions, and five reviewed self-comparison + sites across four methods; +- the complete 44-project Release build reports zero warnings and zero errors; + and +- the no-retry complete hermetic Release gate passed 14,382/14,382 with zero + skips or failures across all 12 test assemblies. + +The five-case reduction from Batch K is exact: five formerly hermetic diagnostic +facts are now excluded by `Purpose=Diagnostic`; the sixth was already outside +the portable lane because its class requires installed DATs. diff --git a/tests/AcDream.Core.Tests/Physics/CellarLipWedgeTests.cs b/tests/AcDream.Core.Tests/Physics/CellarLipWedgeTests.cs index 4f837936..3d69a53c 100644 --- a/tests/AcDream.Core.Tests/Physics/CellarLipWedgeTests.cs +++ b/tests/AcDream.Core.Tests/Physics/CellarLipWedgeTests.cs @@ -71,10 +71,11 @@ public class CellarLipWedgeTests /// /// Diagnostic: drive the player off the threshold toward the cottage and - /// dump the trajectory + indoor-BSP/step probes. Always passes; the - /// captured stdout shows exactly what the engine does each tick. + /// dump the trajectory + indoor-BSP/step probes. The captured artifact + /// shows exactly what the engine does each tick; this is not a verdict test. /// [Fact] + [Trait("Purpose", "Diagnostic")] public void Diagnostic_DriveOffThreshold_DumpTrajectory() { PhysicsDiagnostics.ProbeResolveEnabled = true; @@ -96,7 +97,6 @@ public class CellarLipWedgeTests $"tick={p.Tick} pos=({p.Position.X:F4},{p.Position.Y:F4},{p.Position.Z:F4}) " + $"cell=0x{p.CellId:X8} onGround={p.IsOnGround} cpValid={p.CpValid}")) + "\n\nPROBES:\n" + probeLines); - Assert.True(true); } finally { @@ -284,9 +284,11 @@ public class CellarLipWedgeTests /// /// Diagnostic: replay every captured wedge record and report advance% — to /// confirm the lip-cell engine reproduces the live stuck (0% advance) before - /// asserting a fix. Always passes; results in the message + %TEMP% file. + /// asserting a fix. Results are written to a %TEMP% artifact; this is not + /// a verdict test. /// [Fact] + [Trait("Purpose", "Diagnostic")] public void Diagnostic_ReplayLiveWedgeRecords_Advance() { var recs = LoadWedgeRecords(); @@ -302,7 +304,6 @@ public class CellarLipWedgeTests $"res=({res.X:F2},{res.Y:F2},{res.Z:F2})"); } File.WriteAllText(Path.Combine(Path.GetTempPath(), "lip-wedge-replay.log"), string.Join("\n", lines)); - Assert.True(true, string.Join("\n", lines.Take(10))); } /// @@ -310,6 +311,7 @@ public class CellarLipWedgeTests /// probes on, capturing why the step-up fails. Output to %TEMP%/lip-wedge-stepup.log. /// [Fact] + [Trait("Purpose", "Diagnostic")] public void Diagnostic_ReplayFloorCpRecord_StepUpProbes() { var rec = LoadWedgeRecords().First(r => r.BodyBefore is not null @@ -328,7 +330,6 @@ public class CellarLipWedgeTests $"record cur=({rec.Input.CurrentPos.X:F4},{rec.Input.CurrentPos.Y:F4},{rec.Input.CurrentPos.Z:F4}) " + $"tgt=({rec.Input.TargetPos.X:F4},{rec.Input.TargetPos.Y:F4},{rec.Input.TargetPos.Z:F4}) " + $"req={req:F3} adv={adv:F3} res=({res.X:F4},{res.Y:F4},{res.Z:F4})\n\n" + sw.ToString()); - Assert.True(true); } finally { diff --git a/tests/AcDream.Core.Tests/Physics/CellarUpTrajectoryReplayTests.cs b/tests/AcDream.Core.Tests/Physics/CellarUpTrajectoryReplayTests.cs index 57852bd7..592131a2 100644 --- a/tests/AcDream.Core.Tests/Physics/CellarUpTrajectoryReplayTests.cs +++ b/tests/AcDream.Core.Tests/Physics/CellarUpTrajectoryReplayTests.cs @@ -292,6 +292,7 @@ public class CellarUpTrajectoryReplayTests : IDisposable /// what the harness is actually doing. /// [Fact] + [Trait("Purpose", "Diagnostic")] public void Harness_DiagnosticDump_FirstTenTicks() { PhysicsDiagnostics.ProbeResolveEnabled = true; @@ -308,11 +309,7 @@ public class CellarUpTrajectoryReplayTests : IDisposable string.Join("\n ", trajectory.Select(p => $"tick={p.Tick} pos=({p.Position.X:F4},{p.Position.Y:F4},{p.Position.Z:F4}) " + $"cell=0x{p.CellId:X8} onGround={p.IsOnGround} cpValid={p.CpValid}")); - - // Always pass — this is a diagnostic test; the probe - // output appears in the test runner's captured stdout - // and the trajectory in the assertion message on failure. - Assert.True(true, msg); + Console.WriteLine(msg); } finally { @@ -330,6 +327,7 @@ public class CellarUpTrajectoryReplayTests : IDisposable /// the cellar floor. /// [Fact] + [Trait("Purpose", "Diagnostic")] public void Harness_DiagnosticDump_NoBodySeed() { PhysicsDiagnostics.ProbeResolveEnabled = true; @@ -368,8 +366,7 @@ public class CellarUpTrajectoryReplayTests : IDisposable string.Join("\n ", trajectory.Select(p => $"tick={p.Tick} pos=({p.Position.X:F4},{p.Position.Y:F4},{p.Position.Z:F4}) " + $"onGround={p.IsOnGround}")); - - Assert.True(true, msg); + Console.WriteLine(msg); } finally { diff --git a/tests/AcDream.Core.Tests/Physics/DoorCollisionApparatusTests.cs b/tests/AcDream.Core.Tests/Physics/DoorCollisionApparatusTests.cs index 1271285f..e1c524b6 100644 --- a/tests/AcDream.Core.Tests/Physics/DoorCollisionApparatusTests.cs +++ b/tests/AcDream.Core.Tests/Physics/DoorCollisionApparatusTests.cs @@ -203,6 +203,7 @@ public class DoorCollisionApparatusTests /// without yielding a green/red verdict. /// [Fact] + [Trait("Purpose", "Diagnostic")] public void Apparatus_DiagnosticDump_FrontApproach() { if (!TryBuildScenario(out var ctx)) Assert.Fail("Lane=InstalledDat requires an installed retail DAT directory; see docs/release-gate.md."); @@ -236,7 +237,6 @@ public class DoorCollisionApparatusTests isOnGround = result.IsOnGround; } - Assert.True(true, "Diagnostic test — always passes; check stdout."); } finally { diff --git a/tools/audit-test-inventory.ps1 b/tools/audit-test-inventory.ps1 index 73b70ca9..e2ebbc31 100644 --- a/tools/audit-test-inventory.ps1 +++ b/tools/audit-test-inventory.ps1 @@ -8,7 +8,9 @@ records test attributes and traits, static skips, empty-return sites, prerequisite gates reached through same-file helper calls, assertion/throw signals (including same-file helper calls), diagnostic output signals, - wall-clock waits, environment-variable dependencies, and source-text reads. + high-confidence constant-truth assertions, syntactic self-comparison + candidates, wall-clock waits, environment-variable dependencies, and + source-text reads. This is a candidate generator, not a semantic proof. In particular, an output-only candidate can call a failure-capable helper declared in another @@ -182,6 +184,7 @@ function Test-DirectOutputSignal { })) { $expression = $invocation.Expression.ToString() if ($expression -match '(^|\.)(_?out|output|Console|Debug|Trace)\.(Write|WriteLine)$' -or + $expression -match '(^|\.)File\.(WriteAllText|WriteAllLines|WriteAllBytes|AppendAllText|AppendAllLines)$' -or $expression -match '(^|\.)(WriteDiagnostic|Dump|DumpSteps|Print)(\.|$)') { return $true } @@ -189,6 +192,60 @@ function Test-DirectOutputSignal { return $false } +function Get-AssertionAuditSites { + param( + [Parameter(Mandatory)]$Tree, + [Parameter(Mandatory)]$Method + ) + + $sites = [Collections.Generic.List[object]]::new() + foreach ($invocation in @($Method.DescendantNodes() | Where-Object { + $_.GetType().Name -eq 'InvocationExpressionSyntax' + })) { + $expression = $invocation.Expression.ToString() + $arguments = @($invocation.ArgumentList.Arguments) + if ($arguments.Count -eq 0) { + continue + } + + $first = $arguments[0].Expression.ToString() + $constantTruthKind = if ($expression -match '(^|\.)Assert\.True$' -and + $first -ceq 'true') { + 'Assert.True(true)' + } elseif ($expression -match '(^|\.)Assert\.False$' -and + $first -ceq 'false') { + 'Assert.False(false)' + } else { + $null + } + if ($null -ne $constantTruthKind) { + $sites.Add([ordered]@{ + Line = Get-NodeLine $Tree $invocation + Category = 'ConstantTruth' + Kind = $constantTruthKind + Invocation = [regex]::Replace($invocation.ToString(), '\s+', ' ').Trim() + }) + } + + if ($arguments.Count -lt 2 -or + $expression -notmatch '(^|\.)Assert\.(Equal|StrictEqual|Same|NotEqual|NotSame)$') { + continue + } + + $second = $arguments[1].Expression.ToString() + if ($first -cne $second) { + continue + } + $sites.Add([ordered]@{ + Line = Get-NodeLine $Tree $invocation + Category = 'SelfComparison' + Kind = "Assert.$($expression.Split('.')[-1])" + Invocation = [regex]::Replace($invocation.ToString(), '\s+', ' ').Trim() + }) + } + return @($sites) +} + function Get-WaitSites { param( [Parameter(Mandatory)]$Tree, @@ -416,6 +473,7 @@ foreach ($relativePath in $trackedFiles) { $bodyText = $method.ToString() $waitSites = @(Get-WaitSites $tree $method) + $assertionAuditSites = @(Get-AssertionAuditSites $tree $method) $environmentVariables = @([regex]::Matches( $bodyText, 'GetEnvironmentVariable\s*\(\s*"([A-Za-z0-9_]+)"') | @@ -442,6 +500,12 @@ foreach ($relativePath in $trackedFiles) { HasFailureSignal = $hasFailureSignal HasOutputSignal = $hasOutputSignal OutputOnlyCandidate = $hasOutputSignal -and -not $hasFailureSignal + ConstantTruthAssertions = @($assertionAuditSites | Where-Object { + $_.Category -eq 'ConstantTruth' + }) + SelfComparisonAssertions = @($assertionAuditSites | Where-Object { + $_.Category -eq 'SelfComparison' + }) EnvironmentVariables = $environmentVariables WaitSites = $waitSites HasThreadSleep = @($waitSites | Where-Object { @@ -543,6 +607,16 @@ $summary = [ordered]@{ $_.HelperPrerequisiteReturnCandidate }).Count OutputOnlyCandidates = @($orderedRecords | Where-Object { $_.OutputOnlyCandidate }).Count + ConstantTruthAssertionMethods = @($orderedRecords | Where-Object { + $_.ConstantTruthAssertions.Count -gt 0 + }).Count + ConstantTruthAssertionSites = @($orderedRecords | + ForEach-Object { $_.ConstantTruthAssertions }).Count + SelfComparisonAssertionMethods = @($orderedRecords | Where-Object { + $_.SelfComparisonAssertions.Count -gt 0 + }).Count + SelfComparisonAssertionSites = @($orderedRecords | + ForEach-Object { $_.SelfComparisonAssertions }).Count DiagnosticMethods = @($orderedRecords | Where-Object { $_.Traits -contains 'Purpose, Diagnostic' }).Count