acdream/docs/plans/2026-07-24-modern-runtime-slices-f-g-render-scene.md

Modern Runtime Slices F/G — Incremental Render Scene and Delta Submission

Date: 2026-07-24

Status: approved 2026-07-24; Slice F active

Program: docs/plans/2026-07-24-modern-runtime-architecture.md

Baseline: Slice E closeout commits 91e82c3c and e7d9d6fa

Behavior contract: no change to pixels, view distance, particles, PView visibility, clipping, draw order, selection, lighting, animation, or retail-faithful gameplay

0. Execution ledger

Unit	State	Evidence / rollback
F0a — partition-input referee	complete	Diagnostic-only CurrentRenderSceneOracle; no production draw decision changed.
F0b — survivor/dispatcher/selection referee	complete	Exact PView routes, dispatcher candidates and final group payloads, material/alpha/clip/light/selection fields, and accepted picking parts. Release gate: 3,690 App tests / 3 skips and 8,174 complete-solution tests / 5 skips.
F1 — scene types and contained adapter	complete	Arch 2.1.0 is pinned in App only behind acdream contracts. Five narrow archetypes, generation/incarnation/sequence gates, borrowed-query invalidation, memory accounting, deterministic digesting, and update-thread enforcement pass 11 focused tests. Release gate: 3,701 App tests / 3 skips and 8,185 complete-solution tests / 5 skips.
F2 — static projection journal	complete	Static and EnvCell-shell projections now append ordered deltas only after the final activation receipt and exact detach receipt. Same-landblock rehydrate reconciles retained/new/omitted identities; stale Far completions are inert. Seven focused lifecycle tests pass inside the 3,708-App / 8,192-solution Release gate. The scene remains unconstructed and non-drawing in production.
F3 — live/equipped projection	complete	Exact EntityReady/resource teardown, visibility, attachment pose/removal, and active-only final-frame seams now drive generation/incarnation-gated live records. Duplicate CreateObject, pending/loaded rebucket, hidden/appearance, reentrancy, session clear, attachment, and GUID-generation replacement pass 11 focused tests. Production still constructs no render scene.
F4 — dynamic indices	complete	The contained scene now maintains outdoor/static, per-cell static/dynamic, special dynamic-route, translucent, selectable, light-candidate, and dirty indices incrementally. Final-frame live/equipped synchronization remains active-only, and active animated statics are synchronized from the scheduler's active workset rather than a resident-world scan. The production scene remains unconstructed and non-drawing.
F5	active — connected gate	Lifecycle automation now constructs the otherwise-absent non-drawing scene, drains ordered static/live deltas after post-network spatial reconciliation, compares the accepted current partition every 30 rendered frames and at every canonical checkpoint, retains only the first field-level mismatch, and publishes scene/journal/index/memory evidence in checkpoint JSON. Focused and complete App suites are green; capped/uncapped nine-stop and dense-Arwic physical gates remain.
G0–G5	pending	No production consumer has switched.

The exact pre-F/G runtime rollback anchor remains e7d9d6fa. F0 is non-drawing diagnostic infrastructure and therefore is not a visual-cutover rollback unit. Each later commit that changes the production draw source is listed in section 10 before it is offered for a connected gate.

1. Decision record

1.1 Why this work has a separate approval gate

Slices A–E optimized measurement, prepared content, residency, streaming, and retirement without replacing the renderer's scene model. Slices F/G resume the MP3 work that the user explicitly deferred on 2026-07-05:

• Slice F introduces a second, render-only projection in non-drawing shadow mode.
• Slice G makes that projection the renderer's production input and removes the old whole-world enumeration/partition path.

This is a materially larger regression surface than changing an isolated scheduler or cache. A wrong cell bucket, incarnation, transform, material variant, or draw-order classification can produce the historical stairs/doorway/player-vanish bug classes while still compiling and appearing faster. The approval gate therefore remains binding.

Approval had to be explicit and recorded as:

Approved — continue with all slices.

The user supplied that approval on 2026-07-24 after reviewing the post-E evidence and rewrite risk. Runtime work therefore proceeds from the exact pre-F/G anchor e7d9d6fa.

1.2 Post-Slice-E evidence

Physical-local Release captures on exact binary 91e82c3c6850fcedbf20322b2ecf4fdcd11a2b2a produced:

Capture	CPU p50 / p95 / p99	GPU p50 / p95 / p99
Uncapped nine-stop	1.216 / 4.216 / 5.450 ms	0.725 / 2.486 / 2.666 ms
Dense Arwic uncapped	2.652 / 4.153 / 5.736 ms	1.576 / 2.504 / 2.662 ms

Dense Arwic exercised 15,530 entities and 111 animated entities. Its CPU median is already below the provisional 3.33 ms / 300 FPS target. Therefore:

• F/G is not needed to claim that one provisional median target.
• CPU p95/p99 still exceed GPU p95/p99, leaving measurable CPU headroom.
• The current path still scales with the resident entity population rather than only with scene changes and visible buckets.
• The expected win is lower CPU utilization, better dense-scene scaling, lower tail cost, and a clean future snapshot/headless seam.
• A dramatic average-FPS increase is possible but is not promised.

The go/no-go decision is consequently about accepting architectural rewrite risk for throughput headroom and a cleaner long-term runtime boundary, not about repairing an existing visual or stability failure.

2. Current owner map

Concern	Current owner	Current cost/contract	F/G treatment
Logical live identity and accepted state	LiveEntityRuntime	sole GUID/incarnation authority	unchanged; emits exact projection facts only
Spatial landblock membership	GpuWorldState	sole loaded/pending bucket authority	unchanged; journals committed render projection deltas
Static publication/retirement	LandblockPresentationPipeline and retirement coordinator	retained exact Slice-E receipts	append render deltas at existing committed receipt boundaries
Live presentation resources	EntitySpawnAdapter	exact per-incarnation mesh/composite ownership	unchanged; appearance facts copied into render projection
Equipped-child presentation	EquippedChildRenderController	final current transform and visibility	copied into equipped projection archetype
Static animation	RetailStaticAnimatingObjectScheduler	final root transform for active animated statics	active-only synchronization into dynamic transform components
Frame camera/root/building facts	WorldRenderFrameBuilder	borrowed one-frame facts	retained
Retail cell visibility	PortalVisibilityBuilder / RetailPViewRenderer	retail PView and viewcone oracle	retained without algorithm changes
Per-frame entity split	InteriorEntityPartition	scans every resident WorldEntity	replaced by incremental scene indices in G
Visibility walk/classification	WbDrawDispatcher	walks entity/mesh pairs and rebuilds instance groups	consumes packed frame candidates and dirty instance ranges in G
Alpha order	RetailAlphaQueue	shared stable far-to-near ordering	retained
Selection	retail selection scene/sink	visible part publication and lighting	same accepted parts and transforms from frame view
Lighting	LightManager plus dispatcher selection	per-object retail light set	same algorithm over packed bounds
GL ownership	WbMeshAdapter, texture/residency owners, dispatcher buffers	render-thread only	unchanged

GpuWorldState remains the canonical spatial projection. The new render scene is a derived, disposable presentation index. It cannot answer gameplay, network, collision, inventory, interaction, radar, target, or persistence questions.

3. Target boundary

3.1 Acdream-owned API

Arch is an implementation detail in AcDream.App. No Arch type crosses an acdream interface, enters Core, or appears in a test fixture outside App.

The boundary is equivalent to:

internal interface IRenderScene
{
    RenderSceneGeneration Generation { get; }
    RenderProjectionCounts Counts { get; }

    RenderDeltaApplyResult Apply(ReadOnlySpan<RenderProjectionDelta> deltas);
    void SynchronizeDynamicSources(in DynamicProjectionSyncInput input);
    RenderSceneDigest BuildDigest(RenderSceneDigestBuffer reuse);
    RenderSceneQuery OpenQuery();
    void Clear(RenderSceneGeneration replacementGeneration);
}

The exact API may split mutation, query, and diagnostics into narrower interfaces, but these rules are fixed:

• mutation is update-thread only;
• draw receives only a borrowed read/query surface;
• draw cannot call LiveEntityRuntime, GpuWorldState, or WorldEntity dictionaries;
• Arch entities and .Value primitives stay inside the scene implementation;
• all external identities are acdream-owned wrapper types;
• session/world generations are explicit;
• stale incarnation deltas are rejected rather than applied to a reused slot.

3.2 Identity

Use distinct wrapper identities:

RenderProjectionId
RenderOwnerIncarnation
RenderSceneGeneration
RenderSpatialBucket
RenderAssetHandle

Projection keys are derived presentation identities, not a second gameplay GUID map:

• static: world generation + canonical landblock + current static entity id;
• live root: current local WorldEntity.Id + accepted instance sequence;
• equipped child: parent local id + attachment slot/child local id + incarnation;
• EnvCell: world generation + cell id.

The scene may keep a source-key-to-projection lookup solely to apply deltas. It may not expose that lookup as object identity to gameplay consumers.

3.3 Component set

Initial packed components:

RenderTransform
PreviousRenderTransform
RenderMeshSet
RenderMaterialVariant
RenderSpatialResidency
RenderWorldBounds
RenderFlags
RenderDegradeState
RenderSortKey
RenderOwnerIncarnation
RenderDirtyMask

Archetypes remain narrow:

• outdoor static;
• indoor-cell static;
• live/dynamic root;
• active animated static;
• equipped child;
• translucent/light-bearing variants only where query density justifies them.

Do not attach every component to every projection. Effects, particles, and lights remain in their existing owners during F/G unless a measured query benefit and an exact parity fixture justify a packed reference.

3.4 Delta journal

One update-thread-owned RenderProjectionJournal receives:

Register
UpdateTransform
UpdateAppearance
UpdateFlags
Rebucket
Unregister
ClearGeneration

Each delta carries:

• scene generation;
• projection source key;
• exact owner incarnation;
• monotonically increasing journal sequence;
• complete payload for the changed channel.

Rules:

• structural order is exact;
• updates may coalesce only within one host tick, after the most recent structural edge for the same projection/incarnation;
• register/unregister and generation boundaries never coalesce away;
• unregister of an old incarnation cannot remove its replacement;
• a journal drains fully at the update-to-render boundary;
• any carry-over at frame publication is an invariant failure;
• callback reentrancy appends after the current sequence and cannot invalidate active iteration;
• no bounded queue or worker thread is introduced in F/G—the accepted host thread remains the only writer.

3.5 Static versus dynamic synchronization

Static projections are change-driven:

• landblock spatial publication registers them once;
• appearance or static-animation membership changes issue explicit deltas;
• rebucket/retirement removes them once.

Dynamic root and equipped-child transforms are synchronized after animation, physics, remote motion, local-player projection, and attachment transforms have reached their existing final frame state. The synchronizer walks only the scene's dynamic/active-animated source registry, compares packed final facts, and emits updates for changed records. It never scans all resident statics.

This active-only synchronization is deliberate. Retrofitting presentation version counters into every simulation setter during F would broaden the change into gameplay/physics ownership. A later measured slice may replace the comparison with versioned producers without changing IRenderScene.

4. Referee before the match

The comparison oracle lands before Arch or shadow storage.

CurrentRenderSceneOracle captures the current accepted render input after PView construction and before dispatcher classification. Its deterministic digest contains, in stable sort order:

• source/projection identity and incarnation;
• spatial class: outdoor static, cell static, dynamic, equipped;
• canonical landblock and full cell;
• root transform and world bounds;
• mesh ids and part transforms in order;
• material/palette/texture replacement identity;
• draw/ancestor/hidden/translucency/degrade flags;
• active-animation classification;
• PView survivor class and clip-route identity;
• expected selection part ids;
• opaque/alpha classification identity.

The oracle is validated on the current production path first. It must be:

• deterministic across repeated unchanged frames;
• allocation-free when disabled;
• bounded and reusable when enabled;
• independent of dictionary enumeration order;
• continuously sampled during login, portal transit, reveal, movement, animation, appearance change, and teardown—not only at stable checkpoints.

The oracle does not decide what retail should draw. It records what the already accepted current path supplies, so F/G cannot silently redefine parity in favor of the new implementation.

5. Slice F — Non-drawing shadow render scene

F0 — Current-path oracle

1. Add acdream-owned digest/value types and reusable buffers.
2. Capture the current partition and dispatcher candidate identity without changing draw behavior.
3. Add deterministic unit fixtures for static, dynamic, animated, equipped, translucent, hidden, duplicate ids across landblocks, and GUID reuse.
4. Extend canonical lifecycle artifacts with digest generation, counts, mismatch count, journal depth, and scene bytes.
5. Run the current pipeline alone through the canonical route and establish the referee's own stability.

Commit gate: no Arch dependency; identical screenshots; no material performance regression with comparison disabled.

F1 — Scene types and contained Arch adapter

1. Verify Arch's current license and selected package API; pin one exact version in AcDream.App only.
2. Add IRenderScene, wrapper ids, component structs, query contracts, and ArchRenderScene.
3. Add an architecture test preventing Arch namespaces outside the allowed App folder and preventing primitive id extraction outside the owner assembly.
4. Add memory accounting for chunks, lookup/index storage, journal buffers, and retained synchronization sources.
5. Prove create/update/remove/clear and stale-incarnation rejection in pure App tests.

Commit gate: the scene is unbound and non-drawing; full suite green.

Landed evidence: the package/API/license decision is recorded in docs/research/2026-07-24-arch-render-scene-dependency.md. Production composition does not construct ArchRenderScene; F1 therefore changes no runtime path. Source/reflection architecture tests confine all Arch types and primitive-ID extraction to AcDream.App/Rendering/Scene/Arch. Pure App tests cover every structural/channel operation, generation replacement, stale incarnation and sequence rejection, borrowed-query invalidation, deterministic digest order, retained-memory accounting, and cross-thread mutation rejection. The Release gate is 3,701 App tests / 3 skips and 8,185 complete-solution tests / 5 skips.

F2 — Static projection journal

1. Append Register deltas from the existing final landblock spatial publication receipt.
2. Append Unregister deltas from the exact retirement/demotion receipt.
3. Preserve owner landblock identity independently of the caller's current draw landblock—the #119 cache lesson.
4. Mirror indoor cell statics, outdoor statics, scenery, building shells, and EnvCell projection identity without changing their resource ownership.
5. Make same-landblock rehydrate a reconcile operation: retained identities update, omitted identities unregister, and new identities register.
6. Clear by exact world generation on login, portal supersession, session reset, and graceful shutdown.

Commit gate: shadow static digest equals the referee over deterministic publication/retirement permutations.

F3 — Live and equipped projection journal

1. Register only after the current live record, materialized projection, and presentation resources have committed.
2. Use exact local id + instance sequence; never server GUID alone.
3. Journal loaded/pending visibility edges without treating rebucket as logical recreate.
4. Mirror appearance, mesh refs, palette/material variants, draw flags, ancestor visibility, full cell, and bounds.
5. Mirror equipped children after their current attachment transform update.
6. Unregister exactly once on pickup/parenting/withdraw/delete/session reset, rejecting callbacks from displaced generations.

Commit gate: duplicate CreateObject, reentrant callback, pending-to-loaded, loaded-to-loaded, hidden, parent/child, delete/recreate, and GUID-reuse tests have zero leaked or stale projections.

F4 — Dynamic synchronization and incremental indices

1. Synchronize final current-frame transforms for live roots, active animated statics, and equipped children.
2. Maintain outdoor-static, per-cell-static, dynamic, translucent, selectable, light-candidate, and dirty-record indices incrementally.
3. Preserve InteriorEntityPartition.IsIndoorCellId semantics.
4. Preserve retail's special dynamic routing:
   • outdoor dynamics under an interior root enter the outside stage;
   • exit-portal-straddling dynamics may enter both stages;
   • out-of-flood indoor dynamics remain members but fail the per-dynamic viewcone cull.
5. Assert index counts and membership against the current partition every sampled frame.

Commit gate: no whole-static-world synchronization in a steady frame; unchanged dynamic records emit no update.

F5 — Continuous shadow comparison

1. Apply the journal and build the shadow digest at the update/render boundary.
2. Compare continuously at a fixed diagnostic cadence and at every canonical checkpoint.
3. Record the first exact mismatch with both projections and source channel; never log an unbounded mismatch storm.
4. Run deterministic portal, churn, animation, appearance, hidden, selection, and teardown tests.
5. Run capped and uncapped nine-stop routes plus dense Arwic on the physical display.
6. Prove:
   • zero mismatches;
   • zero stale delta applications;
   • zero journal carry-over;
   • zero retained old-generation scene state;
   • bounded shadow memory;
   • no pixel changes because the shadow does not draw.

Slice F gate: all evidence above passes before any production consumer is switched. F may temporarily cost memory and a small amount of CPU in diagnostic mode; those costs are reported separately and shadow comparison is disabled by default after its gate.

6. Slice G — Frame product and production cutover

G0 — Borrowed double-buffered frame product

Add two reusable RenderFrameArena instances. Building frame N writes the arena not borrowed by draw N-1 and publishes a generation-stamped RenderFrameView.

The view contains spans/ranges for:

• visible outdoor static candidates;
• visible per-cell static candidates keyed by PView order;
• dynamic candidates;
• final root/part transforms;
• opaque and alpha classification records;
• per-object light sets;
• selection parts and lighting;
• existing PView/clip products by borrowed reference;
• diagnostic counts and source digest.

Rules:

• no extra frame of latency;
• no render thread is introduced;
• a view is invalid after its arena is reused;
• no consumer retains spans across frame completion;
• abort recycles the incomplete arena without publishing it;
• world generation and frame sequence must match at draw.

G1 — Replace whole-world partition with scene queries

1. Query outdoor statics directly.
2. Query each PView-visible cell bucket in the exact existing far-to-near order.
3. Query the dynamic set once, then apply the existing per-dynamic viewcone and outside-stage/straddle rules.
4. Preserve look-in cells as a distinct landscape-stage source.
5. Preserve particle-owner filtering against the exact mesh survivor set.
6. Compare the new candidate digest with the old partition without drawing twice.

Commit gate: exact candidate parity through indoor/outdoor transitions, door apertures, look-ins, non-flooded rooms, and portal straddles.

G2 — Packed dispatcher input

1. Introduce an acdream-owned RenderInstanceCandidate consumed by WbDrawDispatcher.
2. Copy every current WorldEntity fact the dispatcher actually uses:
   • mesh refs and ordered part transforms;
   • root transform;
   • owner landblock/cell;
   • palette and texture replacement identity;
   • hidden/draw/ancestor flags;
   • scale, bounds, source setup/gfx id;
   • selection owner identity;
   • translucency/degrade state.
3. Preserve cache key and invalidation semantics, especially owner landblock identity and incomplete-mesh retry.
4. Keep Wb/texture/selection/light/alpha owners unchanged.
5. Add old-input/new-input instance-set comparison at the dispatcher boundary.

Commit gate: identical instance keys, model matrices, material identities, clip slots, light sets, selection parts, alpha records, and draw counts.

G3 — Dirty instance ranges and reusable command storage

1. Retain persistent instance records keyed by projection + mesh part + batch.
2. Rebuild classification only for Register, Appearance, Material, Mesh, Degrade, or relevant Flags dirtiness.
3. Update transform/light/selection fields through dirty ranges.
4. Reuse MDI command, batch, sort, and alpha scratch storage.
5. Cache static command templates by scene generation and visible-cell set only after parity proves the simpler indexed path.
6. Keep resource upload and GL retirement on the accepted render thread.
7. Do not add GPU culling, GPU particles, or render-worker jobs in G.

Commit gate: warm unchanged frames rebuild no static classification and allocate no core scene/dispatcher memory.

G4 — Cutover

1. Run old and new input construction in compare-only mode; draw only the old path.
2. Fix every mismatch at its source.
3. Switch the single production draw input to RenderFrameView.
4. Run instance-set and fixed-camera image comparisons.
5. Obtain the user's connected visual confirmation.
6. Delete:
   • production InteriorEntityPartition use;
   • full LandblockEntries walk from normal world draw;
   • transition flags and dual-input compare code;
   • obsolete static classification rebuild paths.
7. Keep only reusable oracle tooling behind diagnostics/tests.

There is no permanent runtime fallback. A failed gate reverts the cutover commit; it does not ship a hidden old/new switch.

G5 — Physical performance and lifecycle gate

Run the exact Slice-E reference matrix:

• capped physical-local nine-stop;
• uncapped physical-local nine-stop;
• pinned dense-Arwic uncapped with stationary turn;
• fresh login, portal/dungeon, same-location revisit, world edge;
• appearance, hidden/unhide, equipped-child changes;
• graceful logout/reconnect;
• rapid generation replacement and GUID reuse.

Report, without changing thresholds:

• CPU/GPU p50/p95/p99/p99.9 and maximum;
• update, PView, scene query, frame build, dispatcher, alpha, and GL stages;
• frame-thread and process allocation;
• scene/chunk/index/journal bytes;
• managed heap, LOH/POH fragmentation, GC pauses;
• tracked CPU/GPU/staging/retiring bytes;
• instance/draw/triangle counts;
• screenshot and instance-set comparison.

7. Acceptance matrix

7.1 Correctness

• Exact static/live/equipped projection count and digest.
• Exact PView cell order and viewcone behavior.
• Exact outside-stage and portal-straddling dynamic behavior.
• Exact opaque/clip-map/alpha classification and stable alpha order.
• Exact root and part transforms with no one-frame lag.
• Exact hidden, NoDraw, ancestor visibility, and translucency behavior.
• Exact material/palette/texture replacement identity.
• Exact light sets and selection-part placement.
• Exact incarnation behavior across stale updates and GUID reuse.
• Zero replay of logical create/default script/effect work on rebucket.
• Zero draw calls into gameplay/spatial dictionaries.

7.2 Resource and lifecycle

• Journal empty at every frame publication.
• No old world generation in scene after replacement converges.
• Scene memory plateaus on third location visit.
• No stale Arch entity handle after removal/reuse.
• No extra mesh/texture/composite owner.
• No unbalanced reference, staged upload, dynamic light, effect, or alpha record.
• Graceful shutdown drains scene and GL retirement exactly once.

7.3 Performance

The program goals remain:

• dense uncapped p50 at or below 3.33 ms, or a newly attributed blocker;
• traversal p99 at or below 16.67 ms;
• no post-warm frame above 33.3 ms without an attributed indivisible/OS event;
• steady update p95 at or below 2 ms;
• core render-scene/dispatcher allocation p50 of zero;
• whole-client steady allocation initially at or below 4 KiB/frame;
• no known exception-as-control-flow site.

Because Slice E already satisfies the dense median target, G must additionally show one of:

• a statistically meaningful CPU p95/p99 reduction;
• lower CPU utilization at equal uncapped throughput;
• materially improved resident-entity scaling;
• or a documented result that the removed scan was no longer dominant and the next measured bottleneck belongs to Slice H.

No visual-quality reduction is an acceptable way to pass.

8. Commit and review sequence

Each sub-slice is one or more bisectable commits, but a commit may not mix shadow storage, production cutover, and old-path deletion.

For every commit:

1. inspect the complete diff for authority leakage, stale identity, lifecycle asymmetry, and allocations;
2. run focused App tests;
3. run dotnet build AcDream.slnx -c Release;
4. run the complete Release solution tests before a sub-slice gate;
5. update architecture, roadmap, issues, divergence register, and memory in the same commit when their truth changes;
6. introduce no retail-divergence row—F/G is performance-only and any visible difference is a bug.

No substantial feature body enters GameWindow. Composition changes live in FrameRootComposition; scene, journal, oracle, synchronization, and frame product owners live under focused AcDream.App/Rendering namespaces.

9. Stop/rollback conditions

Stop the current sub-slice and fix the root cause when:

• the shadow digest differs;
• source identity requires a second authoritative GUID map;
• a draw consumer must call back into LiveEntityRuntime or GpuWorldState;
• delta order cannot represent a reentrant lifecycle edge;
• the new path needs a timer, suppression flag, or retry workaround;
• a screenshot/instance difference cannot be explained;
• the frame product adds a frame of latency;
• memory fails to plateau;
• F/G performance is worse outside explicitly enabled shadow comparison.

The rollback unit is the failing sub-slice commit. Do not retain a permanent legacy fallback, dual draw, or user-facing renderer switch.

10. Approval and rollback state

This plan is approved and active.

• Pre-F/G runtime rollback anchor: e7d9d6fa.
• Every production cutover commit is added here and to claude-memory/project_render_pipeline_digest.md.
• Shadow/referee commits precede the production switch and need not be reverted for a visual mismatch unless they independently change runtime behavior.
• A failed visual gate reverts the exact cutover commit(s); it does not use git reset, discard later evidence, or enable a permanent fallback.