erik/acdream
1
0
Fork 0
Code Issues Pull requests Projects Releases 6 Packages Wiki Activity Actions

test: remove vacuous diagnostic assertions

Browse source
This commit is contained in:
Erik 2026-08-18 12:48:07 +02:00
parent 5fa9933636
commit ea17bc8624
5 changed files with 151 additions and 28 deletions
Download patch file Download diff file Expand all files Collapse all files

77
docs/reviews/2026-08-18-r3-test-truth-ledger.md
View file

@ -38,7 +38,7 @@ pwsh ./tools/audit-test-inventory.ps1
The generated JSON lives under ignored `artifacts/test-audit/`; it is not a
second 6.8 MB checked-in source of truth. The script and this reviewed ledger
are durable, while paths and line numbers regenerate after every batch. The
inventory refreshed for Batch G reports:
inventory refreshed for Batch L reports:
| Syntax signal | Current count |
|---|---:|
@ -50,7 +50,9 @@ inventory refreshed for Batch G reports:
| Empty-return sites | 143 |
| Direct / same-file-helper prerequisite-return candidates | 2 / 1 |
| Mechanical output-only candidates | 2 |
| Reviewed diagnostic methods | 51 |
| Constant-truth assertion methods / sites | 0 / 0 |
| Reviewed syntactic self-comparison methods / sites | 4 / 5 |
| Reviewed diagnostic methods | 57 |
| Methods directly using `Thread.Sleep` / `Task.Delay` | 15 / 23 |
| Cancellable-infinite-only / elapsed-time methods | 5 / 33 |
| Methods directly reading environment variables | 47 |
@ -63,15 +65,19 @@ the inventory now fails explicitly inside its selected non-hermetic lane. The
remaining 143 empty returns are behavioral control flow, callback exits, or
the three platform branches; they are not missing-prerequisite passes.
The output-only classifier follows same-file helper calls, but it cannot decide
whether an assertion merely establishes probe setup. The reviewed diagnostic
set is therefore 51 methods: the 47 mechanical candidates plus
`OptionsPanelLiveMountProbeTests.ProbeLiveMountShapes`,
`DoorSetupGfxObjInspectionTests.HoltburgCottage_CellPortals_DatInspection`, and
the two already classified tower diagnostics. Relative to T-011's historical
51-method table, delete the removed PVS scaffold and add
`TowerAscentReplayTests.Diagnostic_TowerAscent_PerStepTable`; all other entries
remain. Those 51 methods expand to 70 discovered cases (28 App, 42 Core).
The output-only classifier follows same-file helper calls, so an assertion used
only while constructing a probe can prevent an otherwise diagnostic method from
appearing in its two-candidate result. The reviewed `Purpose=Diagnostic` set is
therefore the authority: Batch C's 51 methods plus six artifact/report writers
reconciled in Batch L, for 57 methods and 76 discovered cases (28 App, 48 Core).
Their investigative output and historical rationale remain executable outside
the release total.
The five syntactic self-comparisons are reviewed, not vacuous: two call a hash
twice to assert determinism, one reads the singleton twice, and two read the
same Runtime owner's properties twice to assert stable identity. The inventory
keeps these candidates visible separately from the now-zero constant-truth
assertions.
## Exact 77-skip reconciliation
@ -130,10 +136,10 @@ identifiable in the lane report.
| T-005 unreachable panel stack | requires production reachability proof | Do not delete 52 meaningful tests until the dead presentation surface and any supported plugin compatibility promise are verified together. |
| T-006 misleading installed-DAT reason | resolved in batch B | Shared opt-in is now `ACDREAM_RUN_INSTALLED_DAT_TESTS=1` (legacy switch retained), the reason names the lane, and all nine owners carry `Lane=InstalledDat`. |
| T-007 271 silent passing gates | resolved in batch G | All 280 directly found gates plus three later-reconciled sites now fail explicitly when their selected lane lacks its prerequisite. The only three syntax candidates left are reviewed cross-platform branch false positives. |
| T-008 incapable-of-failing diagnostics | batch A plus diagnostic lane work | Delete the literal wiring smoke test; repair the GPU contract tautology; later re-home output-only apparatus. |
| T-008 incapable-of-failing diagnostics | resolved in batches A, C, and L | The literal wiring smoke test was deleted, the GPU contract tautology was repaired, and the six remaining constant-truth diagnostic assertions were removed after their output was preserved under `Purpose=Diagnostic`. |
| T-009 wall-clock double-click tests | resolved in batch E | Four sleeps were replaced by a deterministic test clock behind an internal factory overload. The production factory still reads `Environment.TickCount64` exactly as before. |
| T-010 two useless cases | high-confidence cleanup batch A | Delete `SmokeTest.TestProject_IsWired` and `ChaseCameraTests.ImplementsICamera`; compilation already proves both claims. |
| T-011 51 output-only methods | classified in batch C | The reviewed current set is 51 methods / 70 cases. All carry `Purpose=Diagnostic`, preserving the apparatus while removing it from release pass totals. Contract-shaped names remain explicitly flagged until a stable oracle exists. |
| T-011 output-only methods | classified in batches C and L | The reviewed current set is 57 methods / 76 cases. All carry `Purpose=Diagnostic`, preserving the apparatus while removing it from release pass totals. Contract-shaped names remain explicitly flagged until a stable oracle exists. |
| T-012 source-text freezes | replacement map complete in batch I; 46 implementation-shape facts remain staged | Twelve whole-tree architecture rules and five cross-artifact contracts stay. The other 46 are campaign-era literal implementation freezes; retire each only beside its semantic/behavioral replacement, with user approval for owner/lifecycle assertions that have no equivalent yet. |
| T-013 controller self-comparison | high-confidence cleanup batch A | Capture the first controller next to the first body and compare every retry with that reference. |
| T-014 seven load-sensitive tests | six mechanisms repaired in batch F; one product defect classified | Virtual/wall-clock mixing, tiered-JIT allocation noise, a live controller clock, and a ThreadPool-start timing oracle were removed without weakening behavioral contracts. `DatSoundCache` #321 is a real in-flight-entry race and now runs as `Status=KnownFailure` pending a product fix. |
@ -143,6 +149,7 @@ identifiable in the lane report.
| T-018 stderr observer race | resolved in R2 | Live reader shares writes/deletes; 25 focused repetitions plus the complete gate. |
| T-019 remaining explicit waits | classified in batch H; nine fixed-delay negative oracles need cleanup | Five of the 38 methods are deterministic cancellation stubs, not wall-clock tests. Preserve four intentional real-time contracts and bounded integration polling; replace the nine tests that infer “still blocked” from a fixed delay with observable synchronization state. |
| T-020 exact duplicate bodies/data | resolved in batch K | Four redundant executions were removed. The remaining 11 body-equivalent groups are intentionally split theories with disjoint, meaningfully named datasets; the inventory enforces zero repeated rows within a theory or across body-equivalent theories. |
| T-021 suspicious assertion shapes | resolved in batch L | The inventory enforces zero `Assert.True(true)` / `Assert.False(false)` sites and reports syntactic self-comparisons for review. All five current self-comparisons assert meaningful determinism or stable identity. |
## Preserved rationale for removals in batch A
@ -579,3 +586,47 @@ complete 44-project Release build reports zero warnings and zero errors. The
no-retry complete hermetic Release gate passed 14,387/14,387 with zero skips or
failures across all 12 test assemblies. The four-case reduction is exactly the
three duplicate facts and one duplicate theory row removed above.
## Batch L vacuous diagnostic assertions
Six investigation methods still ended in an unconditional green assertion:
- three cellar-lip trajectory/replay artifact writers in
`CellarLipWedgeTests`;
- two cellar-ascent trajectory reports in `CellarUpTrajectoryReplayTests`; and
- the installed-DAT front-approach probe in `DoorCollisionApparatusTests`.
These were useful tools but not regression tests. Batch L adds
`Purpose=Diagnostic`, removes the six `Assert.True(true, ...)` calls, preserves
the two reports that previously existed only as assertion messages by writing
them to captured console output, and leaves the existing `%TEMP%` artifacts and
probe logs intact. No historical investigation notes, fixtures, or probe logic
were deleted.
The durable inventory now recognizes `File.WriteAll*` / `File.AppendAll*` as
diagnostic-output signals and reports high-confidence constant-truth assertions
separately from syntactic self-comparisons. It reports zero constant-truth sites.
The five self-comparison sites were individually reviewed and retained:
- `Hash32Tests.Calculate_IsDeterministic` and
`PacketHeaderTests.CalculateHeaderHash32_DeterministicForSameInput` execute
the calculation twice and compare the results;
- `NullCommandBusTests.Instance_IsSingleton` reads the singleton twice; and
- `RuntimeEntityObjectLifetimeTests.Owner_ConstructsOneExactDirectoryAndObjectTablePerInstance`
verifies repeated access to both owned objects returns the same instances.
Batch L verification:
- all six affected diagnostic methods pass, including the installed-DAT door
probe;
- the refreshed inventory reports 11,447 attributed methods, 57 diagnostic
methods, zero constant-truth assertions, and five reviewed self-comparison
sites across four methods;
- the complete 44-project Release build reports zero warnings and zero errors;
and
- the no-retry complete hermetic Release gate passed 14,382/14,382 with zero
skips or failures across all 12 test assemblies.
The five-case reduction from Batch K is exact: five formerly hermetic diagnostic
facts are now excluded by `Purpose=Diagnostic`; the sixth was already outside
the portable lane because its class requires installed DATs.

13
tests/AcDream.Core.Tests/Physics/CellarLipWedgeTests.cs
View file

@ -71,10 +71,11 @@ public class CellarLipWedgeTests
/// <summary>
/// Diagnostic: drive the player off the threshold toward the cottage and
/// dump the trajectory + indoor-BSP/step probes. Always passes; the
/// captured stdout shows exactly what the engine does each tick.
/// dump the trajectory + indoor-BSP/step probes. The captured artifact
/// shows exactly what the engine does each tick; this is not a verdict test.
/// </summary>
[Fact]
[Trait("Purpose", "Diagnostic")]
public void Diagnostic_DriveOffThreshold_DumpTrajectory()
{
PhysicsDiagnostics.ProbeResolveEnabled = true;
@ -96,7 +97,6 @@ public class CellarLipWedgeTests
$"tick={p.Tick} pos=({p.Position.X:F4},{p.Position.Y:F4},{p.Position.Z:F4}) " +
$"cell=0x{p.CellId:X8} onGround={p.IsOnGround} cpValid={p.CpValid}")) +
"\n\nPROBES:\n" + probeLines);
Assert.True(true);
}
finally
{
@ -284,9 +284,11 @@ public class CellarLipWedgeTests
/// <summary>
/// Diagnostic: replay every captured wedge record and report advance% — to
/// confirm the lip-cell engine reproduces the live stuck (0% advance) before
/// asserting a fix. Always passes; results in the message + %TEMP% file.
/// asserting a fix. Results are written to a %TEMP% artifact; this is not
/// a verdict test.
/// </summary>
[Fact]
[Trait("Purpose", "Diagnostic")]
public void Diagnostic_ReplayLiveWedgeRecords_Advance()
{
var recs = LoadWedgeRecords();
@ -302,7 +304,6 @@ public class CellarLipWedgeTests
$"res=({res.X:F2},{res.Y:F2},{res.Z:F2})");
}
File.WriteAllText(Path.Combine(Path.GetTempPath(), "lip-wedge-replay.log"), string.Join("\n", lines));
Assert.True(true, string.Join("\n", lines.Take(10)));
}
/// <summary>
@ -310,6 +311,7 @@ public class CellarLipWedgeTests
/// probes on, capturing why the step-up fails. Output to %TEMP%/lip-wedge-stepup.log.
/// </summary>
[Fact]
[Trait("Purpose", "Diagnostic")]
public void Diagnostic_ReplayFloorCpRecord_StepUpProbes()
{
var rec = LoadWedgeRecords().First(r => r.BodyBefore is not null
@ -328,7 +330,6 @@ public class CellarLipWedgeTests
$"record cur=({rec.Input.CurrentPos.X:F4},{rec.Input.CurrentPos.Y:F4},{rec.Input.CurrentPos.Z:F4}) " +
$"tgt=({rec.Input.TargetPos.X:F4},{rec.Input.TargetPos.Y:F4},{rec.Input.TargetPos.Z:F4}) " +
$"req={req:F3} adv={adv:F3} res=({res.X:F4},{res.Y:F4},{res.Z:F4})\n\n" + sw.ToString());
Assert.True(true);
}
finally
{

11
tests/AcDream.Core.Tests/Physics/CellarUpTrajectoryReplayTests.cs
View file

@ -292,6 +292,7 @@ public class CellarUpTrajectoryReplayTests : IDisposable
/// what the harness is actually doing.
/// </summary>
[Fact]
[Trait("Purpose", "Diagnostic")]
public void Harness_DiagnosticDump_FirstTenTicks()
{
PhysicsDiagnostics.ProbeResolveEnabled = true;
@ -308,11 +309,7 @@ public class CellarUpTrajectoryReplayTests : IDisposable
string.Join("\n ", trajectory.Select(p =>
$"tick={p.Tick} pos=({p.Position.X:F4},{p.Position.Y:F4},{p.Position.Z:F4}) " +
$"cell=0x{p.CellId:X8} onGround={p.IsOnGround} cpValid={p.CpValid}"));
// Always pass — this is a diagnostic test; the probe
// output appears in the test runner's captured stdout
// and the trajectory in the assertion message on failure.
Assert.True(true, msg);
Console.WriteLine(msg);
}
finally
{
@ -330,6 +327,7 @@ public class CellarUpTrajectoryReplayTests : IDisposable
/// the cellar floor.
/// </summary>
[Fact]
[Trait("Purpose", "Diagnostic")]
public void Harness_DiagnosticDump_NoBodySeed()
{
PhysicsDiagnostics.ProbeResolveEnabled = true;
@ -368,8 +366,7 @@ public class CellarUpTrajectoryReplayTests : IDisposable
string.Join("\n ", trajectory.Select(p =>
$"tick={p.Tick} pos=({p.Position.X:F4},{p.Position.Y:F4},{p.Position.Z:F4}) " +
$"onGround={p.IsOnGround}"));
Assert.True(true, msg);
Console.WriteLine(msg);
}
finally
{

2
tests/AcDream.Core.Tests/Physics/DoorCollisionApparatusTests.cs
View file

@ -203,6 +203,7 @@ public class DoorCollisionApparatusTests
/// without yielding a green/red verdict.
/// </summary>
[Fact]
[Trait("Purpose", "Diagnostic")]
public void Apparatus_DiagnosticDump_FrontApproach()
{
if (!TryBuildScenario(out var ctx)) Assert.Fail("Lane=InstalledDat requires an installed retail DAT directory; see docs/release-gate.md.");
@ -236,7 +237,6 @@ public class DoorCollisionApparatusTests
isOnGround = result.IsOnGround;
}
Assert.True(true, "Diagnostic test — always passes; check stdout.");
}
finally
{

76
tools/audit-test-inventory.ps1
View file

@ -8,7 +8,9 @@
records test attributes and traits, static skips, empty-return sites,
prerequisite gates reached through same-file helper calls, assertion/throw
signals (including same-file helper calls), diagnostic output signals,
wall-clock waits, environment-variable dependencies, and source-text reads.
high-confidence constant-truth assertions, syntactic self-comparison
candidates, wall-clock waits, environment-variable dependencies, and
source-text reads.
This is a candidate generator, not a semantic proof. In particular, an
output-only candidate can call a failure-capable helper declared in another
@ -182,6 +184,7 @@ function Test-DirectOutputSignal {
})) {
$expression = $invocation.Expression.ToString()
if ($expression -match '(^|\.)(_?out|output|Console|Debug|Trace)\.(Write|WriteLine)$' -or
$expression -match '(^|\.)File\.(WriteAllText|WriteAllLines|WriteAllBytes|AppendAllText|AppendAllLines)$' -or
$expression -match '(^|\.)(WriteDiagnostic|Dump|DumpSteps|Print)(\.|$)') {
return $true
}
@ -189,6 +192,60 @@ function Test-DirectOutputSignal {
return $false
}
function Get-AssertionAuditSites {
param(
[Parameter(Mandatory)]$Tree,
[Parameter(Mandatory)]$Method
)
$sites = [Collections.Generic.List[object]]::new()
foreach ($invocation in @($Method.DescendantNodes() | Where-Object {
$_.GetType().Name -eq 'InvocationExpressionSyntax'
})) {
$expression = $invocation.Expression.ToString()
$arguments = @($invocation.ArgumentList.Arguments)
if ($arguments.Count -eq 0) {
continue
}
$first = $arguments[0].Expression.ToString()
$constantTruthKind = if ($expression -match '(^|\.)Assert\.True$' -and
$first -ceq 'true') {
'Assert.True(true)'
} elseif ($expression -match '(^|\.)Assert\.False$' -and
$first -ceq 'false') {
'Assert.False(false)'
} else {
$null
}
if ($null -ne $constantTruthKind) {
$sites.Add([ordered]@{
Line = Get-NodeLine $Tree $invocation
Category = 'ConstantTruth'
Kind = $constantTruthKind
Invocation = [regex]::Replace($invocation.ToString(), '\s+', ' ').Trim()
})
}
if ($arguments.Count -lt 2 -or
$expression -notmatch '(^|\.)Assert\.(Equal|StrictEqual|Same|NotEqual|NotSame)$') {
continue
}
$second = $arguments[1].Expression.ToString()
if ($first -cne $second) {
continue
}
$sites.Add([ordered]@{
Line = Get-NodeLine $Tree $invocation
Category = 'SelfComparison'
Kind = "Assert.$($expression.Split('.')[-1])"
Invocation = [regex]::Replace($invocation.ToString(), '\s+', ' ').Trim()
})
}
return @($sites)
}
function Get-WaitSites {
param(
[Parameter(Mandatory)]$Tree,
@ -416,6 +473,7 @@ foreach ($relativePath in $trackedFiles) {
$bodyText = $method.ToString()
$waitSites = @(Get-WaitSites $tree $method)
$assertionAuditSites = @(Get-AssertionAuditSites $tree $method)
$environmentVariables = @([regex]::Matches(
$bodyText,
'GetEnvironmentVariable\s*\(\s*"([A-Za-z0-9_]+)"') |
@ -442,6 +500,12 @@ foreach ($relativePath in $trackedFiles) {
HasFailureSignal = $hasFailureSignal
HasOutputSignal = $hasOutputSignal
OutputOnlyCandidate = $hasOutputSignal -and -not $hasFailureSignal
ConstantTruthAssertions = @($assertionAuditSites | Where-Object {
$_.Category -eq 'ConstantTruth'
})
SelfComparisonAssertions = @($assertionAuditSites | Where-Object {
$_.Category -eq 'SelfComparison'
})
EnvironmentVariables = $environmentVariables
WaitSites = $waitSites
HasThreadSleep = @($waitSites | Where-Object {
@ -543,6 +607,16 @@ $summary = [ordered]@{
$_.HelperPrerequisiteReturnCandidate
}).Count
OutputOnlyCandidates = @($orderedRecords | Where-Object { $_.OutputOnlyCandidate }).Count
ConstantTruthAssertionMethods = @($orderedRecords | Where-Object {
$_.ConstantTruthAssertions.Count -gt 0
}).Count
ConstantTruthAssertionSites = @($orderedRecords |
ForEach-Object { $_.ConstantTruthAssertions }).Count
SelfComparisonAssertionMethods = @($orderedRecords | Where-Object {
$_.SelfComparisonAssertions.Count -gt 0
}).Count
SelfComparisonAssertionSites = @($orderedRecords |
ForEach-Object { $_.SelfComparisonAssertions }).Count
DiagnosticMethods = @($orderedRecords | Where-Object {
$_.Traits -contains 'Purpose, Diagnostic'
}).Count