acdream/docs/plans/2026-07-24-modern-runtime-slice-e-cost-budgeted-streaming.md

# Modern Runtime Slice E — Cost-budgeted streaming and retirement

**Status:** complete — E0/E1/E2/E3/E4/E5/E6 landed and gated 2026-07-24
**Program:** `docs/plans/2026-07-24-modern-runtime-architecture.md`
**Baseline:** Slice D closeout commit `66690805`
**Behavior contract:** no visual-quality or retail-behavior reduction

## 1. Outcome

Slice E removes whole-window portal publication and retirement transactions
from a single update frame. It preserves the existing worker, canonical
`GpuWorldState`, presentation owners, retry receipts, world-reveal barrier,
and renderer resource owners.

The change is scheduling and lifecycle architecture:

- one immutable `StreamingWorkBudget` defines real per-frame limits;
- one single-writer scheduler admits and advances explicit work stages;
- destination-critical work receives reserved capacity across portal frames;
- old generations become unavailable and quiescent immediately;
- expensive owner/resource teardown resumes from exact cursors;
- publication advances through exact owner receipts under time, byte, entity,
  upload, and retirement-operation limits;
- the existing `WorldRevealCoordinator` remains the only reveal predicate.

No distance, texture, particle, cell, collision, or presentation quality is
reduced.

## 2. Retail boundary

Retail teardown is synchronous:

- `CLandBlock::destroy_static_objects @ 0x0052FA50`;
- `CLandBlock::Destroy @ 0x0052FAA0`;
- `CObjectMaint::DestroyObjects @ 0x00508C30`.

Retail portal/cell blocking freezes object maintenance, physics, landscape,
game time, and ambient sound while continuing scene/UI/event work:
`SmartBox::UseTime @ 0x00455410`.

Acdream's async adaptation may defer physical cleanup, but may not leave a
retired owner active. Full translation:
[`../research/2026-07-24-retail-streaming-retirement-pseudocode.md`](../research/2026-07-24-retail-streaming-retirement-pseudocode.md).

## 3. Existing-owner inventory

| Concern | Current owner | Current behavior | Slice E action |
|---|---|---|---|
| Worker inbox/outbox | `LandblockStreamer` | unbounded channels; near jobs prioritized | retain worker; expose bounded, allocation-free single-consumer drain and backlog facts |
| Completion apply | `StreamingController._deferredApply` | retained `List`; count-budgeted loads; unload and priority ring bypass | replace with explicit generation/priority FIFO stage queues and exact retained-byte accounting |
| Full reload | `BeginFullWindowRetirement` | snapshots and begins every resident retirement synchronously | retain stable snapshot/cursor and advance admissions under budget |
| Origin recenter | `OriginRecenterRetirement` | has a cursor but drains it to completion in one call | make the cursor the canonical budgeted admission receipt |
| Logical/spatial detach | `GpuWorldState.DetachLandblock/DetachNearLayer` | exact immediate bucket withdrawal; captures entity receipt | preserve; add destination-generation quiesce before cursored detaches |
| Retirement ledger | `LandblockRetirementCoordinator` | retryable per-stage/per-entity state, but advances a complete ticket on begin/advance | expose one bounded atomic step at a time; stable FIFO |
| Retirement effects | `LandblockPresentationRetirementOwner` | lights, translucency, plugin, terrain, physics, cells/buildings | preserve order; each entity/stage consumes measured budget |
| Publication ledger | `LandblockPresentationPipeline` | retryable stage booleans, but `Advance` runs the complete transaction | convert booleans/cursors into resumable atomic publication steps |
| Render publication | `LandblockRenderPublisher` | terrain/visibility/AABB/buildings/EnvCells whole stages | retain receipts; meter GPU bytes and split iterable registries where needed |
| Physics publication | `LandblockPhysicsPublisher` | rebuilds terrain/cells/buildings/statics/reflood in one call | cursor cell/building/static preparation and commit at safe mutation boundaries |
| Static presentation | `LandblockStaticPresentationPublisher` | allocates dictionaries/sorts arrays, then loops prior/plugin entities | prepare stable ordered arrays once; cursor cleanup/light/plugin work |
| Mesh upload | `ObjectMeshManager`/`WbMeshAdapter` | existing per-frame renderer owner/budgets | report exact upload bytes/operations to the streaming meter; never move GL ownership |
| Reveal | `WorldRevealCoordinator` + `WorldRevealReadinessBarrier` | one correct login/portal predicate | add destination generation/readiness stage facts; never duplicate predicate |
| Portal wait cue | `PortalTunnelPresentation` | authored tunnel and centered wait notice port exists | hold tunnel past five seconds and publish diagnostic; never early reveal |
| Audio | `OpenAlAudioEngine`/`AudioHookSink` | one-shot owner is not retained in source slot | tag 3-D voices by owner; stop old-generation owners at quiesce |

## 4. Work model

### 4.1 Immutable budget

The App-layer type is equivalent to:

```csharp
public readonly record struct StreamingWorkBudget(
    TimeSpan MaxUpdateTime,
    int MaxCompletionAdmissions,
    long MaxAdoptedCpuBytes,
    int MaxEntityOperations,
    long MaxGpuUploadBytes,
    int MaxGlRetireOperations,
    float DestinationReserveFraction);
```

All fields are positive and validated as one profile. The initial High/default
profile is evidence-tuned, not a reduction in quality:

- 2.0 ms update-thread streaming work;
- 64 completion admissions;
- 8 MiB retained CPU payload adoption;
- 256 entity operations;
- 8 MiB requested GPU upload;
- 64 logical GL-retirement admissions;
- 75% reserved for destination-critical work while a reveal is pending.

These are initial scheduling ceilings, not content ceilings. Work continues on
later frames until exact convergence. Connected E gates may retune the profile
as one atomic change.

The compatibility `MaxCompletionsPerFrame` setting maps to complete Low/Medium/
High/Ultra work profiles during migration. It does not remain an independent
hidden second throttle.

### 4.2 Meter

`StreamingWorkMeter` is stack/frame scoped and single-thread owned. It records:

- start/deadline timestamp;
- admitted results and retained CPU bytes;
- entity operations;
- GPU bytes requested;
- GL retirement operations;
- stage completions, yields, overruns, failures, and oldest-work age.

Reservation is checked before an atomic operation using its conservative known
cost. Elapsed time is checked after every atomic operation. A first operation
may run when a dimension is otherwise empty so progress cannot deadlock; any
single-operation overrun is named in diagnostics and becomes a required split
site.

### 4.3 Cost estimation

`LandblockStreamResultCost` is computed once from immutable payloads:

- terrain vertex/index bytes;
- entity count;
- mesh-reference count;
- EnvCell visibility/shell count;
- physics cell/building/GfxObj counts;
- prepared payload arrays retained by the completion.

It never uses process working set as a scheduling unit and never counts the
memory-mapped package's address space as adopted bytes.

## 5. Queue and ordering contract

One scheduler owns queues by `(generation, priority, stage)`.

Priorities:

1. destination collision/required Near scene;
2. visible Near;
3. ordinary Near;
4. Far/speculative;
5. retirement cleanup after immediate quiesce.

Rules:

- FIFO is exact within one generation/priority.
- Generation supersession cancels stale unpublished work and releases its
  retained-byte charge.
- Destination reserve applies only while the canonical reveal generation is
  pending.
- Unreserved capacity is work-conserving: any queue may use it.
- At least one noncritical/retirement operation is admitted when capacity
  remains, preventing starvation during a slow destination.
- Failed stages retain their exact receipt and queue position.
- Callback reentrancy appends work; it never mutates the active cursor.
- `_deferredApply`, `MaxDrainIterations`, the eager priority-radius bypass, and
  count-only `MaxCompletionsPerFrame` execution disappear at cutover.

## 6. Immediate quiesce versus deferred release

At a hard destination-generation edge:

1. The old world generation is marked unavailable to normal world drawing,
   picking, radar, status targeting, and collision queries.
2. Object/static animation, PhysicsScript, particle emission, and ambient/
   owner-tagged audio for that generation are frozen or silenced.
3. The portal tunnel/UI/event path continues, matching retail's
   `blocking_for_cells` branch.
4. Stable resident landblock IDs are captured once.
5. Per-landblock spatial detach and owner/resource release advance from cursors.

Quiesce is idempotent and generation-scoped. It does not destroy live server
identity, replay Hidden/UnHide, or clear the replacement generation.

For ordinary Near→Far demotion outside a hard portal edge, the exact landblock
detach remains the immediate visibility/collision boundary; only its expensive
owner cleanup is deferred.

## 7. Publication contract

One accepted completion progresses through:

```text
Admitted
  -> PreparedReceipt
  -> RenderPrefix
  -> PhysicsBase
  -> RenderCellsBuildings
  -> StaticEntityPresentationAndCollision
  -> SpatialCommit
  -> RenderPinsAndEnvCellReplay
  -> LiveProjectionRecovery
  -> Complete
```

Each transition owns a retry receipt and known cost. Mutation order remains the
existing retail-rooted order. A stage cannot publish partial externally visible
state unless its receipt describes and resumes that exact partial prefix.

Destination reveal consumes only `Complete`/renderer-upload facts belonging to
the active destination generation. Far-ring work may continue after reveal.

## 8. Threading

- Worker threads build immutable `LandblockBuild`/prepared payloads and enqueue
  immutable completions only.
- The update/render thread is the sole scheduler, meter, world-state,
  publication, and retirement writer.
- GL upload/non-residency/deletion remains with renderer resource owners.
- No update operation waits for the worker, a GPU fence, file I/O, or a lock
  held by render work.
- Back-pressure never blocks the update thread. Worker-side bounded queues may
  wait/cancel; update-side admission is non-blocking.

## 9. Delivery checkpoints

### E0 — Oracle, inventory, and contract

- Record named-retail teardown/blocking pseudocode.
- Inventory every current publication/retirement owner and bypass.
- Define budget dimensions, atomic work, queue ordering, quiesce, retries,
  threading, diagnostics, and acceptance.

**Complete 2026-07-24.**

### E1 — Typed budgets, meter, cost model, and diagnostics

- Add validated `StreamingWorkBudgetOptions` to `RuntimeOptions`.
- Add pure `StreamingWorkMeter` and deterministic injected-clock tests.
- Add exact immutable completion-cost estimation.
- Publish per-stage backlog, bytes, oldest age, work/yield/overrun facts through
  lifecycle artifacts.
- Preserve current execution while measuring the would-yield decisions.

**Complete 2026-07-24.** `RuntimeOptions` now owns one validated scheduling
profile (time, admissions, retained CPU bytes, entity operations, requested GPU
bytes, GL retire operations, and destination reserve). The single-thread meter
has a deterministic clock seam, first-operation progress rule, explicit
yield/oversize/failure/overrun facts, and no effect on legacy execution at this
checkpoint. Immutable completion payloads receive deterministic charges for
their exact array data and logical retained entries; this is deliberately not
misreported as CLR heap size. Lifecycle JSON now includes last-frame work,
deferred retained bytes/age, and pending publication/retirement counts.

Validation: complete App suite 3,643 passed / 3 skipped; complete solution
8,124 passed / 5 skipped; Release solution build green. Existing priority,
retry, stale-generation, and deferred-compaction tests remain green.

### E2 — Explicit admission queues

- Replace `_deferredApply` with stable generation/priority queues.
- Bound result admission by count and CPU bytes.
- Remove `MaxDrainIterations`.
- Preserve exact retry position and stale-generation rejection.
- Keep current publication atomic until E4, but execute only work admitted by
  the typed meter.

**Complete 2026-07-24.** Production now consumes the worker channel through
one allocation-free `TryPeek`/`TryRead` source, prices the exact immutable
result before adoption, and admits it through the typed completion-count and
retained-CPU dimensions. Accepted work lives in reusable stable FIFOs for
destination, control, unload, Near, and Far classes. FIFO order is exact inside
each class, a blocked publication head does not reorder its tail, callback
appends cannot invalidate an active receipt, and exact-reference compaction
cannot conflate equal records or GUID/landblock reuse.

`_deferredApply`, `MaxDrainIterations`, direct priority hunting, and the unload
budget bypass are gone. Destination and unload work win queue order but consume
the same execution meter. One indivisible publication may make progress after
admission work, but that privilege is granted only once per frame and every
oversize/overrun remains named. Stale generations consume a bounded admission
without retaining their payload. The old completion-count quality setting now
selects a complete scaled time/count/byte/operation profile rather than acting
as a hidden second throttle.

Validation: 240 focused App streaming tests and 58 focused Core streamer/
controller tests passed; the complete App suite passed 3,656 / 3 skipped, the
complete solution passed 8,138 / 5 skipped, and the Release solution build was
clean. Physical connected and visual gates are intentionally deferred until
E6; the active desktop was RDP and is not valid performance evidence.

### E3 — Quiesce and budgeted retirement

- Add destination-generation world quiesce.
- Tag/stop owner audio at the immediate edge.
- Make full-window and recenter retirement admission cursor/time/entity bounded.
- Make each retirement ticket advance by bounded atomic entity/stage work.
- Prove no old generation ticks, emits, collides, targets, or renders while its
  retained memory converges.

**Complete 2026-07-24.** `WorldGenerationQuiescence` now publishes the hard
login/portal edge through one generation-scoped availability state owned by
`WorldRevealCoordinator`. The edge withdraws world drawing, picking/radar
spatial queries, target selection, object/static animation, PhysicsScript time,
particle/effect progression, liveness, spatial reconciliation, and owner-tagged
3-D audio while network/event/command, portal presentation, UI, streaming, and
destination readiness continue. A superseding reveal remains continuously
quiescent and only its exact generation can reopen the world; canonical live
records and readiness state remain retained, so the mechanism does not
synthesize Hidden/UnHide or rebuild server identity.

Full-window and shared-origin retirement are now retained frame transactions.
Generation invalidation, worker-inbox clearing, accepted-payload release,
region clearing, stable resident capture, and landblock detachment all resume
from exact cursors through the frame's single `StreamingWorkMeter`.
`LandblockRetirementCoordinator` preserves stable FIFO order and advances one
entity or one owner stage per reservation across scripts, classifications,
lighting, translucency, plugin projection, terrain, physics, cell visibility,
building registries, and environment cells. Failures retry the exact unfinished
entity/stage without replaying committed work. A landblock detach remains one
named indivisible spatial boundary; its complete entity count is charged before
the operation.

Validation: 51 focused reveal/quiescence/retirement tests passed; the complete
App suite passed 3,666 / 3 skipped; the complete solution passed 8,148 / 5
skipped; Release solution build green. The physical connected performance and
visual gate remains deferred to E6 because the active desktop is RDP and is not
valid GPU/FPS evidence.

### E4 — Cursor-budgeted publication

- Move stable ordered arrays/cost facts into retained receipts.
- Cursor static cleanup, static light/collision/plugin publication, physics
  cell/building work, and environment-cell publication at safe boundaries.
- Feed GPU requested-byte and retirement-operation facts from renderer owners.
- Prove every failure resumes the exact unfinished operation without replay.

**Complete 2026-07-24.** One accepted completion now remains at the exact queue
head while `LandblockPresentationPipeline` advances retained render, physics,
and static receipts through the frame's single meter. Stable entity/Gfx/building
and prior-owner arrays are captured once. Terrain upload bytes are charged at
the render owner; physics cells/buildings, static collision/light/plugin work,
prior static teardown, shadow reflood, building-registry construction, and
EnvCell shell construction advance one bounded operation at a time. Building
and EnvCell replacements are prepared off-side and become visible only when
complete. Failures retain the unfinished cursor and never replay a committed
prefix. Settings/native callbacks can defer policy changes but cannot drain
publication outside `StreamingController.Tick`.

The final `GpuWorldState.MutationBatch` remains one deliberate observer-atomic
operation: bucket mutation, render-id ownership, activation, and the outer
visibility notification cannot span frames without exposing a partial spatial
identity. It is time-metered and named for E6 overrun evidence; all iterable
owner work now occurs before it.

Validation: focused render/building/publication/controller tests passed; the
complete App suite passed 3,669 / 3 skipped; the complete Release solution
passed 8,152 / 5 skipped; Release solution build green. Physical connected
performance and visual evidence remains an E6 gate.

### E5 — Destination reservation and reveal generation

- Replace eager `PriorityRadius` execution with destination reservations.
- Join scheduler destination generation to the existing reveal coordinator.
- Keep the tunnel animating and surface the retail wait cue/diagnostic beyond
  five seconds.
- Remove the last count-only execution/bypass paths.

**Complete 2026-07-24.** `WorldRevealCoordinator` now owns one exact
generation/cell/radius reservation from login or portal begin through complete
or cancel. `StreamingController` no longer exposes a mutable priority
landblock/radius pair: accepted work records the reveal generation that
classified it, stale completion cannot consume a replacement generation's
reserved lane, and stale teardown cannot clear the replacement reservation.
The frame meter records destination and non-destination use independently and,
while a reveal is active, protects the configured destination share across
wall time, completion admissions, adopted CPU bytes, entity operations,
requested GPU bytes, and GL-retirement admissions. Destination work still
uses the same global ceilings and the existing first-indivisible-operation
progress diagnostic; it does not regain a budget bypass.

The former ten-second forced materialization path is deleted. An incomplete
destination remains in the authored portal scene until the canonical render,
composite, and collision predicate is true. After five seconds the retained
gameplay UI displays retail's centered
`"In Portal Space - Please Wait..."` notice and lifecycle telemetry records the
cue once; the tunnel continues its DAT animation and repeats the notice on the
retail camera-rotation cadence. Login completion now ends its reveal
reservation/quiescence at the same auto-entry edge that exposes the ready
world.

Validation: 91 focused reservation/reveal/teleport/UI tests passed; the
complete App suite passed 3,674 / 3 skipped; the complete Release solution
passed 8,158 / 5 skipped; Release solution build green with zero warnings.
The physical-local routes and final performance/resource assertions remain
E6, not E5.

### E6 — Gates and closeout

- Deterministic pressure, reentrancy, failure, cancellation, stale-generation,
  GUID reuse, dungeon, outdoor, and session-reset suites.
- Release build and full solution tests.
- Physical-local capped/uncapped nine-stop and pinned dense-Arwic routes.
- Compare portal p99 and maximum single-frame allocation to Slice A and Slice C.
- Require zero `viewport-before-ready`, zero stranded old generation, staged
  upload, collision, effect, audio, or GPU owner.
- Fixed-camera screenshots and subjective visual gate only if pixels change.
- Update architecture, inventory, issues, roadmap, divergence register, and
  durable memory.

**Complete 2026-07-24.** Deterministic budget, pressure, cancellation,
stale-generation, retry, callback-reentrancy, GUID-reuse, quiescence, and
resource-ownership gates pass. The Release solution builds and the complete
suite passes 8,164 tests with 5 intentional skips.

The exact `91e82c3c6850fcedbf20322b2ecf4fdcd11a2b2a` Release binary passed
capped and uncapped nine-stop routes plus pinned dense Arwic on the physical
local AMD display. Every destination materialized only after the canonical
render/composite/collision predicate, every client exited gracefully, and
every canonical checkpoint ended with zero pending publication, retirement,
worker/destination/class backlog, staged upload, composite warmup, residency
staging/requested-GPU, or retiring bytes.

The gate found and corrected two connected-only ownership defects. CPU
mesh-cache hits now stage only for a live exact owner, preventing evicted work
from recreating stale uploads. Loaded spatial residency is now distinct from
world availability, allowing quiesced destination live objects to prepare
their render projection without becoming drawable, collidable, pickable,
targetable, audible, or simulated before reveal.

Relative to Slice A, capped/uncapped CPU p99 improved 34.7%/30.4% and largest
frame allocation fell 84.7%/82.2%. Relative to Slice C, CPU p99 improved
21.4%/16.1% and largest frame allocation fell 33.5%/9.7%. Full evidence:
[`../research/2026-07-24-slice-e-cost-budgeted-streaming-report.md`](../research/2026-07-24-slice-e-cost-budgeted-streaming-report.md).

## 10. Acceptance

Slice E closes only when:

1. No priority or unload path bypasses all work budgets.
2. No frame admits another operation after a configured count/byte/entity
   dimension is exhausted. The documented first-indivisible-operation rule is
   the sole oversize path and remains named.
3. Every wall-time overrun names one indivisible operation and the scheduler
   admits no later operation after its deadline. Because the meter uses elapsed
   wall time on a non-real-time OS, thread preemption can make an otherwise
   bounded atomic operation observe more than 2 ms; the physical gate therefore
   also requires materially improved route/portal p99 and no repeatable
   algorithmic tail.
4. Old generations become unavailable, tick-frozen, and audio-silent at the
   hard transition edge.
5. Retirement/publication retries resume exact cursors without replay.
6. Deferred retained CPU bytes stay within their queue budget.
7. Reveal is possible only for the active destination generation and the
   existing readiness predicate.
8. Portal-window p99 and largest frame allocation materially improve over the
   post-Slice-A physical baseline.
9. Complete connected routes end with zero pending publication, retirement,
   upload, effect, collision, audio, and stale-generation owners.
10. Visual output/range/quality and retail gameplay behavior are unchanged.