erik/acdream
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

test(app): measure the warmed path, not the path being warmed (#250)

Browse source 
The zero-allocation family failed about one full-suite run in three, on
unchanged trees, and had been dismissed as inherent noise in
`GC.GetAllocatedBytesForCurrentThread` three separate times. It is not noise.
Reading the four members side by side, they share one root: **the measured
window was never the warmed path.**

  UiDatFontTests            1 warm call, then a 10,000-iteration loop inline
  RenderFrameProductTests   8 warm calls, then a 1,000-iteration loop inline
  OracleTests               1 warm call, 1 measured call
  ArchRenderSceneTests      warms Apply(registrations), measures Apply(updates)

Two mechanisms come out of that table. A test method is JIT-compiled at tier 0
like anything else, and a long-running loop in tier-0 code gets replaced
mid-flight by on-stack replacement — which compiles on the thread running the
loop, so its bookkeeping is charged to the window being measured. That is the
first two. And `ArchRenderSceneTests` warmed one arm of a switch and measured
the other, so the measured call was the first ever into `ApplyUpdate` and paid
that arm's JIT, type loads and static initialisation inside the window;
`RenderFrameProductTests` warmed 8 times, below the tier-0 call-counting
threshold of 30, so promotion was still pending when measurement began.

That also explains the signature nobody could account for. Alone, the process is
quiet and the runtime has finished before the assertion arrives. Alongside eight
other test assemblies, tier-0 compilation never stops, the call-counting delay is
re-armed continually, and the work slides into the window. Clean in isolation,
failing under load, on a tree that changed nothing.

`ZeroAllocationProbe` invokes the step many times before measuring anything, then
measures windows that run the same already-warmed loop over the same
already-taken path. Each window is a batch of 32 invocations and it reports the
minimum across 4 of them. Both halves are load-bearing: the minimum is what
excludes a one-time cost, and the batch is what keeps the assertion as strong as
the loops it replaces — minimising over *single* invocations would report zero
for a path that allocates every tenth call, which is a real regression made
invisible. I had written it that way first and the apparatus test caught it.

**The bound is untouched: exactly zero, no tolerance, no retry, no assertion
relaxed.** `ZeroAllocationProbeTests` proves the apparatus can still fail — a
step allocating every call reads above zero and does throw, a first-invocation
cost reads as zero, a cost every tenth call is caught, and the one stated limit
(the batch must cover the period) is pinned as a test rather than left as prose.
Without those, a later edit could quietly make the whole family unfailable.

Twelve further sites in this assembly still use the hand-rolled shape. None has
been observed failing, and each needs its own repeatability analysis — several
mutate state or consume monotonic sequences — so they are listed in the issue
for adoption when next touched rather than converted blind at scale.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Erik 2026-07-29 03:42:08 +02:00
parent ce9445b270
commit 1d73ce524c
7 changed files with 436 additions and 24 deletions
Download patch file Download diff file Expand all files Collapse all files

60
docs/ISSUES.md
View file

@ -702,6 +702,66 @@ the goal is to remove the measurement noise, not to loosen the bound.
**Acceptance:** twenty consecutive Release runs of the App suite with zero
failures.
### Fixed at the measurement — 2026-07-29
**The four members share one root: the measured window was never the warmed
path.** Reading them side by side makes it obvious, and it is not "allocation
measurement is inherently noisy" — it is two concrete, fixable mistakes.
| Test | Warmup | Measured window |
|---|---|---|
| `UiDatFontTests` | 1 call | a **10,000-iteration loop** written inline |
| `RenderFrameProductTests` | **8** calls | a **1,000-iteration loop** written inline |
| `CurrentRenderSceneOracleTests` | 1 call | 1 call (body is a 1,000-iteration loop) |
| `ArchRenderSceneTests` | `Apply(registrations)` | `Apply(**updates**)` — a different switch arm |
Two mechanisms follow:
1. **On-stack replacement inside the window.** A test method is JIT-compiled at
tier 0 like any other method, and a long-running loop in tier-0 code is
replaced mid-flight by OSR. OSR compiles on the thread running the loop —
the measuring thread — so its bookkeeping is charged to the window. Both
inline-loop tests measured exactly the shape that triggers it.
2. **First-call cost inside the window.** `ArchRenderSceneTests` warmed the
`ApplyRegister` arm and measured the `ApplyUpdate` arm, so the measured call
was the first ever into that code: its tier-0 JIT, type loads and static
initialisation all landed inside. `RenderFrameProductTests` warmed 8 times,
below the tier-0 call-counting threshold of 30, so promotion was still
pending when measurement began.
That also explains the signature — clean alone, failing about one full run in
three. Alone, the process is quiet and the runtime has finished before the
assertion arrives. Alongside eight other test assemblies, tier-0 compilation
never stops, the call-counting delay is re-armed continually, and the work
slides into the window.
**Fix:** `tests/AcDream.App.Tests/ZeroAllocationProbe.cs`. It invokes the step
many times before measuring anything, then measures windows that run the same
already-warmed loop over the same already-taken path. Each window is a **batch**
of 32 invocations and the probe reports the **minimum** across 4 such batches.
The minimum excludes one-time costs; the batch is what keeps the assertion as
strong as the loops it replaced, since minimising over *single* invocations
would report zero for a path that allocates every tenth call. **The bound stays
exactly zero — no tolerance, no retry, no assertion weakened.**
`ZeroAllocationProbeTests` guards the apparatus in both directions: a step that
allocates every call is reported above zero and does throw; a first-invocation
cost reads as zero; a cost every tenth call is caught; and the stated limit —
the batch must cover the period — is pinned rather than left as prose. Without
those, a later edit could quietly make the whole family unfailable.
**Not yet done:** twelve further `GetAllocatedBytesForCurrentThread` sites in
`AcDream.App.Tests` still use the hand-rolled shape (`CellViewDedupTests`,
`EquippedChildProjectionWithdrawalTests`, `PortalProjectionTests`,
`RenderFrameRouteOwnerSelectorTests`, `StaticRenderProjectionJournalTests`,
`PackedProjectionClassificationCacheTests`,
`GpuWorldStateRenderTraversalTests`, `UiTextLayoutCacheTests`,
`LiveEntityRuntimeTests`, `RetailInboundEventDispatcherTests`, and a second
site in `CurrentRenderSceneOracleTests`). None has been observed failing.
Each needs its own repeatability analysis — several mutate state or consume
monotonic sequences — so they were left alone rather than converted blind.
Adopt the probe when one is next touched, or immediately if it flakes.
---
## #249 — Bindless handles stay resident after their table slot is released