Both attempts removed from the branch by user direction after live
gates (attempt 1 culled the interior — the 'fixed' exit was a
nothing-drawn false positive; attempt 2's luminosity port produced no
visible change, theory-vs-plumbing unresolved). Patches preserved in
the session scratchpad. Next attempt starts with apparatus: tunnel
freeze probe, RenderDoc capture, ACViewer oracle, retail side-by-side.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
While a destination reservation hides the world behind the tunnel, the
frame meter runs on a hold-widened profile (2 ms -> 8 ms destination
lane, non-destination lane absolute caps unchanged), reverting the frame
the reservation ends. Measured: warm-process portal hold gate-ready
3.6 s / total 8.7 s (retail-feel band); login drip 6-7 s -> 3.0 s with
the remaining login floor now attributed to process cold-start warmup
(~8 s concurrent phase), the next #418 lever. Tunnel held 64-66 fps
through the burst; mid-game no-hold path bit-identical (test-pinned).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
While a destination reservation hides the world behind the authored
tunnel, the streaming frame meter now runs a hold-widened profile
(StreamingWorkBudget.WidenForDestinationHold): the time ceiling rises
from the authored 2 ms to an absolute 8 ms default
(ACDREAM_STREAM_WORK_HOLD_DEST_MS is a measurement-only override), every
count/byte dimension scales by the same factor so elapsed time stays the
authoritative guard (the measured binder is Time at both ceilings), and
the reserve fraction is re-derived (0.75 -> 0.9375) so the
NON-destination lane's absolute per-frame caps are unchanged. The
widening keys off the existing BeginDestinationReservation/
EndDestinationReservation bracket only, is derived per-Tick from the
CURRENT budget (mid-hold quality swaps compose), and a frame with no
reservation uses the authored budget verbatim (test-pinned). Portal
holds ride the same bracket as login holds by construction - intended,
and pinned by a kind-parity test through the real coordinator plus a
live @telepoi portal hold (kind=portal gate-ready 3589 ms).
Why: issue #418's next-hypothesis (1). Measured result: the ~5 s
publication drip collapsed to ~2 s (loaded 625/625 at ~3.0 s, tunnel at
64-66 fps), the portal-hold gate-ready fell to ~3.6 s - and login
gate-ready/total stayed at 8.4-8.8 s / 12.6-12.7 s, exposing the real
remaining pacer: the login-cold render-thread upload/registration
barrier behind GpuWorldState.IsRenderReady, which ran concurrently under
the old drip. Full attribution appended to docs/ISSUES.md #418; no
divergence-register row (the streamed result and reveal gate are
byte-identical; only the scheduling rate during a hidden hold changed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The user's benchmark (retail ~6 s at a smaller window without our
complete-at-reveal guarantee) is now the issue's done bar. Next levers
in order: hold-scoped destination-lane budget widening (predicted
gate-ready ~2-3 s), then upload/composite overlap; retail's authored
tunnel exit is not a tuning target. Miss-branch closeout per the
session goal: acceptance measured 12689/12734 ms (> 12000), attribution
complete, no blind tuning.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The flat 32 blocks/s was the metered publication arm treating Runtime's
deliberate two-poll collision activation as end-of-frame: one landblock
per 32 Hz streaming tick with the 2 ms budget ~90% idle. The metered arm
now uses the same Runtime continuation gate the synchronous path always
used; the budget is genuinely authoritative, debt semantics preserved,
streamed result byte-identical. Remaining floor fully attributed in #418
(~8 s budgeted readiness + retail's authored tunnel exit).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase-1 measurement (new [publish-timing]/[stream-tick] probe surfaces,
ACDREAM_PROBE_REVEAL_TIMING=1) refuted the ~31 ms-per-admission
hypothesis: the hold runs at ~64 fps with the streaming tick at ~32 Hz,
the whole 625-block window costs only ~500 ms of publication CPU (far
blocks ~0.17 ms, near 2-43 ms), and steady state showed ZERO meter
yields with ~0.22 ms of the 2 ms budget used - yet exactly one block
published per tick against a ~400-deep completion queue.
The real limiter: Runtime's collision-generation activation is a
deliberate two-poll transaction (the first
TryAcquireCollisionPrefixMutationPermission poll parks residents and
refuses by design), and LandblockPresentationPipeline.Advance's metered
arm returned Completed=false on ANY nonterminal commit, which
DrainAndApply treats as end-of-frame. One landblock per 32 Hz tick =
the flat 32/s, with the authored budget ~90% idle.
Fix: the metered arm now uses the same Runtime-owned gate the unmetered
arm and the synchronous CompletePublication API always used
(CanContinueMutationSynchronously). The second poll runs in the same
frame under the same meter, so the unchanged 2 ms elapsed-time ceiling
is now genuinely the authoritative per-frame bound; with any real debt
(live residents parked mid-game, pending withdrawals, dispatch backlog)
publication defers to the next frame exactly as before. No budget
values change, no reveal-gate/readiness change, and the streamed result
is byte-identical - only the frame scheduling of identical operations.
Measured A/B (this binary, two runs): totalMs 12689 / 12734 vs baseline
26728/27395/27503; loaded slope 32/s -> bursts of 100-360/s, 625/625 in
~6-7 s vs ~23 s. The remaining ~12.7 s floor is fully attributed in
docs/ISSUES.md: ~8 s of real budgeted readiness work plus retail's
authored tunnel exit (TunnelContinue 2-5 s + two 1 s fades, golden
constants), so the <12 s acceptance needs a lead decision on the
hold-time budget, not another hidden limiter.
New regression pin:
MeteredLoaded_NonterminalCommitWithoutDebt_CompletesInOneMeteredAdvance.
Gates: Release build 0 errors; App tests 5576/3 skips/0 failed;
Runtime tests 1756/0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The identical flat 32/s across the serial and pooled binaries points at a
single ~31 ms indivisible per-landblock admission on the update thread
(one admission per frame under the 2 ms floor, frame stretched to ~31 ms,
~32 fps x 1/frame = 32/s); the old serial builder producing at the same
~31 ms/block masked it. Predicts ~31 ms tunnel frame time; fix shape is
splitting/off-threading the publication cost, not budgets (disproved
twice). Verify the frame-time prediction first next session.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Login publishes the 25x25 window at a flat 32 blocks/s (~27 s in the
tunnel). The reveal-timing probe A/B (695a27b4) showed the consumer
budget env ceilings change nothing, which was read as producer-limited:
one "acdream.streaming.worker" thread, ~31 ms/block. This replaces the
single worker with min(ProcessorCount-2, 8) workers, floor 1.
Design: striped/affinity dispatch. Each worker owns one unbounded lane
channel plus its own high/low priority queues; jobs route to
lane = ((id >> 16) * 2654435761) % N (the low word of a landblock id is
constant, so the id is mixed before reduction). Striping was chosen
over a shared queue + in-flight conflict tracker because it preserves
the per-landblock contract structurally rather than by bookkeeping:
every job for one id lives on one lane, so per-id enqueue order IS
execution and completion-arrival order, and the same-landblock
supersede rules (PromoteToNear removes queued LoadFar/Unload) keep
seeing every queued job for that id. Contract, point by point:
- Per-landblock ordering: same id -> same lane -> serial FIFO.
- ClearLoads: broadcast to every lane inside the same _inboxGate lock
that serializes enqueues, so any load enqueued before
ClearPendingLoads() returns sits ahead of its lane's ClearLoads copy
in that lane's FIFO and is dropped at read time, exactly like the
single-thread path. Already-dequeued builds still complete (now up
to one per worker instead of one total); StreamingController's
SweepCollapsed already unloads those uniformly.
- Priority: per-lane high/low split unchanged. Cross-lane, priority is
not globally ordered (a lane cannot run another lane's job), which
the contract permits; near-tier jobs hash-spread across lanes and
are preferred within each.
- Outbox: SingleWriter flipped to false; nothing assumed single-writer
(PublishResult already used TryWrite + an Interlocked backlog, and
the consumer's peek->read head-stability holds because only the
single reader ever moves the head). Cross-landblock arrival order
was verified arbitrary-tolerant before relying on it:
StreamingController.AdmitCompletions classifies each result
independently into per-priority FIFOs (generation staleness +
per-landblock retirement blocking); per-landblock arrival order is
preserved by striping.
- Crash surface: per-worker. The first real crash publishes
WorkerCrashed (prefixed "worker N:" in pools > 1), sets
_workerFailure, completes every lane, and cancels the pool (a crash
still ends all processing, as before); siblings that merely observe
the closed lanes (ChannelClosedException) exit quietly instead of
reporting spurious crashes; the outbox completes only when the LAST
worker exits so no in-flight completions are dropped.
- Disposal: joins every worker under the same _disposeGate; Start
stays idempotent and dispose-serialized.
Thread-safety audit of the production build closures
(SessionPlayerComposition), per shared object:
- DatCollection (every read in LandblockBuildFactory.BuildLocked:
LandblockLoader.Load, SceneryGenerator.Generate, SetupMesh.Flatten,
CellMesh.Build, GfxObjBounds.Get, GfxObjDegradeResolver): NOT
thread-safe; already serialized under the shared _datLock, which
BuildLocked holds for the whole read transaction. Unchanged; the
probe run measured hold 0-13 ms / wait <= 12 ms during the login
window, so the lock is not the new bottleneck and the build was NOT
serialized beyond it.
- PakPreparedAssetSource / PakReader (BuildPreparedCollisionClosure,
outside the lock): immutable TOC array + read-only
MemoryMappedViewAccessor random-access reads + ConcurrentDictionary
verdict caches - safe for N concurrent readers (Slice I3 design;
the headless SharedPreparedCollisionCache wrapper is fully
lock-protected).
- LandblockMesh.Build (outside the lock): pure math over the dat
record + the composition-time height table + the immutable
TerrainBlendingContext record; the shared SurfaceCache is a
ConcurrentDictionary and BuildSurface is deterministic, so its
lookup-or-build race is last-write-wins-benign (the code already
documented exactly this).
- PhysicsDiagnostics probe statics: read-only bools + thread-safe
Console writes.
MEASURED OUTCOME (gate 4): the timing acceptance did NOT pass, and per
the task contract that is reported, not tuned around. With 8 workers
on this 16-core machine all 625 builds complete in ~203 ms
(ACDREAM_PROBE_TELEPORT BUILD lines t=3475390..3475593) - the producer
is off the critical path - but loaded= still advances at exactly
+32/1000 ms and SUMMARY totalMs measured 27395 and 27503 across two
runs (baseline 26728). The 32/s pacer is in the consumer
admission/publication path and is not governed by the
StreamingWorkBudgetOptions env ceilings. #418 stays IN-PROGRESS on the
consumer side; see docs/ISSUES.md for the evidence chain.
Tests: per-landblock ordering under 4-worker contention, cross-lane
ClearLoads drop, per-lane near-before-far preference, pool-of-1 serial
equivalence, disposal joining every worker, lane-spread guard, and
worker-count validation (LandblockStreamerPoolTests). Two existing
tests asserted a GLOBAL cross-landblock execution order - a serial
implementation detail, not the contract - and now pin workerCount: 1
with justification comments (LoadNear_OvertakesQueuedFarLoads,
TwoQueuedLoads_RetainTheirDistinctOriginAndGeneration).
Gates: Release build 0 errors; App suite 5575 passed / 3 skipped
(5568 + 7 new); Runtime suite 1756/0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
[reveal-timing] lines from the reveal coordinator: per-dimension
first-ready edges (render neighborhood, composite textures, collision,
gate, materialization), 1 Hz progress with the resident-landblock count
(new GpuWorldState.LoadedLandblockCount), and one SUMMARY line at the
viewport reveal. Measurement-first groundwork for the login-load speedup:
the readiness barrier observes its dimensions serially, so the edges give
each dimension's observed tail while the progress lines expose the
pacing shape (a budget-paced linear drip reads directly off the counts).
Probe-gated in StreamingDiagnostics per Code Structure Rules §5; no
behavior change, one branch per Evaluate poll when unset.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
