diff --git a/docs/plans/2026-04-11-roadmap.md b/docs/plans/2026-04-11-roadmap.md index a0e43cfa..0847808c 100644 --- a/docs/plans/2026-04-11-roadmap.md +++ b/docs/plans/2026-04-11-roadmap.md @@ -1568,6 +1568,17 @@ port in any phase — no separate listing here. > or throughput redesign. It is corrective ownership/reclamation work for a > reproduced crash-class regression in the mandatory renderer, using the current > architecture and preserving pixels/ranges. +> +> **2026-07-24 planning reconciliation:** a new connected nine-stop CPU/GPU/ +> memory audit reproduced a larger portal allocation/GC storm (approximately +> 204 MiB in one frame, 276 ms maximum frame, 38 Gen-2 collections) while +> confirming stable entity/effect/GPU plateaus and low steady GPU utilization. +> The user requested a detailed modern architecture plan. The resulting +> [`2026-07-24-modern-runtime-architecture.md`](2026-07-24-modern-runtime-architecture.md) +> unifies the unfinished MP pak/render-world work with future Track LH, preserves +> retail-shaped simulation and the N.5 renderer, and reorders execution around +> prepared content and cost-budgeted streaming before render-ECS throughput. +> This records the plan only; implementation remains not started. **Spec:** `docs/superpowers/specs/2026-07-05-modern-pipeline-design.md` (the umbrella design — read it first). **Goal:** smoothness first (no frame over diff --git a/docs/plans/2026-07-24-modern-runtime-architecture.md b/docs/plans/2026-07-24-modern-runtime-architecture.md new file mode 100644 index 00000000..8ed1299a --- /dev/null +++ b/docs/plans/2026-07-24-modern-runtime-architecture.md @@ -0,0 +1,797 @@ +# Modern Runtime Architecture — detailed execution plan + +**Date:** 2026-07-24 + +**Status:** Planning complete; implementation not started + +**Scope:** Reconcile and sequence the existing Modern Pipeline (`MP`) and +Linux/headless (`LH`) tracks using the 2026-07-24 connected performance audit. + +**Primary objective:** Minimize CPU, managed/native memory, GPU memory, +allocation, and frame-time variance without reducing view distance, particles, +world detail, or retail behavior. + +This plan refines rather than replaces +[`docs/superpowers/specs/2026-07-05-modern-pipeline-design.md`](../superpowers/specs/2026-07-05-modern-pipeline-design.md). +It preserves the shipped N.5 bindless/MDI renderer, the completed `GameWindow` +decomposition, canonical `LiveEntityRuntime` ownership, the retail PView/cell +visibility ports, and all behavior already accepted by the user. + +--- + +## 1. Why the architecture should change + +The 2026-07-24 six-minute, nine-stop connected route established: + +| Observation | Measurement | Architectural implication | +|---|---:|---| +| Steady capped update work | approximately 0.4–1.3 ms median | The gameplay/update architecture is not the steady-state crisis | +| External GPU use | 7.6% median, 15.9% p95 | The tested route was not GPU-bound | +| Portal allocation | up to approximately 204 MiB in one frame | Runtime content preparation and publication must leave the frame transaction | +| Managed allocation rate | approximately 40.8 MB/s average, 2.22 GiB/s maximum | Portal work is creating short-lived graphs and pixel/mesh arrays faster than GC can absorb | +| GC | 38 Gen-2 collections; 256 ms longest pause | Frame spikes are dominated by allocation/publication, not upload time alone | +| Longest frame | 276 ms | Landblock-count budgets are not real cost budgets | +| Final live GC heap | approximately 194 MiB | The 1–2 GiB process high-water is not an unbounded live-object leak | +| Dedicated GPU memory | 633–818 MiB | Texture arrays/caches are bounded but expensive and insufficiently observable | +| Revisited locations | entity/emitter/particle counts returned to the same plateaus | Logical lifetime is broadly convergent; optimize rather than replace it | +| Exception traffic | 4,520 invalid DAT probes plus 1,168 receive timeouts | Exceptions are being used as normal control flow | + +Allocation attribution names the runtime mesh/texture path +(`MeshExtractor.PrepareMeshData` → `PrepareGfxObjMeshData` → +`AddSurfaceToBatch`) as the largest portal source. The update-thread +publication/retirement graph then amplifies it with complete dictionaries, +sorts, effect profiles, and unmetered per-entity teardown. + +The conclusion is: + +> Keep retail-shaped gameplay and the modern GPU backend. Replace the runtime +> content, scene-projection, residency, and work-scheduling architecture that +> feeds the renderer. + +--- + +## 2. Fixed architectural decisions + +These decisions are binding unless a later measured gate disproves one. + +1. **No whole-client rewrite.** Existing retail ports, network behavior, + physics, animation, UI, plugins, and the N.5 renderer remain in place. +2. **No simulation ECS.** Retail-shaped object-oriented code remains the oracle- + traceable form for physics, motion, sequencers, combat, inventory, and magic. +3. **A data-oriented render world only.** Arch remains the selected storage + implementation, hidden behind acdream-owned interfaces so it can be replaced + without changing simulation or rendering contracts. +4. **One authoritative live-object owner.** `LiveEntityRuntime` is evolved and + later extracted; no second GUID map, gameplay world, or mirrored entity state + may be introduced during migration. +5. **Headless is not a hidden window.** A headless session constructs no + Silk.NET window, GL context, render assets, particles, audio, or gameplay UI. +6. **One content interpretation.** Bake and runtime validation use the exact + existing `AcDream.Content` interpretation code. A baked artifact must be + byte/field equivalent before runtime uses it. +7. **No visual-quality optimization by default.** View horizon, retail PView, + particles, effects, and texture pixels stay unchanged. Retail distance LOD is + a faithful mechanism and remains separately configurable. +8. **Work is budgeted by cost, not object count.** Streaming stages use elapsed + microseconds, prepared bytes, uploaded bytes, and entity operations. “Four + landblocks” is not a meaningful budget. +9. **GL remains render-thread owned.** Workers may perform I/O, parsing, + decompression, mesh preparation, and collision preparation, but never mutate + GL or live scene state. +10. **Frame products are borrowed immutable views, not object graphs.** Snapshot + boundaries use double-buffered arrays/spans and generation stamps; they do + not allocate a deep immutable tree each frame. +11. **Parallelism follows data cleanup.** Do not add a job system to compensate + for repeated decode, sorting, allocation, or whole-world scans. +12. **Every cutover deletes the replaced production path.** Temporary shadow + execution is allowed for comparison, but no permanent dual renderer, + duplicate world, or silent decode fallback remains after its gate. + +--- + +## 3. Target dependency architecture + +```text +AcDream.GraphicalHost (current AcDream.App) + ├── window/input/OpenGL/OpenAL/retail UI + ├── render scene projection + renderer + └── owns one AcDream.Runtime.GameRuntime + +AcDream.Headless + ├── deterministic host clock + ├── bot/plugin command surface + └── owns one or more AcDream.Runtime.GameRuntime instances + +AcDream.Runtime [new, extracted incrementally] + ├── session lifecycle and inbound ordering + ├── authoritative live world and properties + ├── movement/physics/interaction/combat/magic/inventory/chat + ├── instance-scoped clocks, queues, commands, and events + └── no Silk.NET, GL, OpenAL, retained UI, or OS window dependency + +AcDream.Content [already exists] + ├── DAT interpretation + ├── deterministic bake pipeline + ├── mapped pak/content manifest + ├── immutable prepared render/collision metadata + └── shared read-only content store for multiple sessions + +AcDream.Core / AcDream.Core.Net [existing retail-shaped logic] +AcDream.Plugin.Abstractions [existing BCL-only contracts] +``` + +`AcDream.Runtime` is a target boundary, not the first implementation step. App +types move only after their presentation dependencies have been removed and +parity tests prove that the graphical host is still driving the same owner. + +### Dependency rules + +- `Core` never depends on Runtime, App, Content, or a backend. +- `Core.Net` depends on Core only. +- `Content` depends on Core, never App or a graphics backend. +- `Runtime` may depend on Core, Core.Net, Content, and Plugin.Abstractions. +- The graphical and headless hosts depend on Runtime; Runtime never depends on + either host. +- UI and rendering consume read-only runtime views plus explicit commands/events. +- Shared multi-session caches contain immutable content only. Credentials, + GUIDs, clocks, packets, plugins, object state, and automation remain + instance-scoped. + +--- + +## 4. Runtime ownership model + +| Concern | Canonical owner | Consumers | +|---|---|---| +| Server GUID/incarnation/timestamps/properties | `GameRuntime` evolved from `LiveEntityRuntime` | physics, UI, render projection, plugins, bots | +| Static prepared content | `ContentStore` | streaming, collision, render projection | +| Desired spatial window | `StreamingRegion` | streaming scheduler | +| Landblock generation and stage receipts | `StreamingWorkScheduler` | reveal, physics publication, render publication | +| Render entity/component data | `IRenderScene` / Arch implementation | visibility, snapshot builder | +| CPU/GPU asset residence | `ResidencyManager` | render scene, upload owner | +| GL names, fences, physical retirement | renderer resource owners | renderer only | +| Retail PView/visible-cell result | existing PView owner | render-scene query/snapshot builder | +| UI state | focused UI controllers/ViewModels | retained UI and optional dev UI | + +Identity types must remain explicit: + +```text +ServerGuid authoritative network identity +ObjectIncarnation ServerGuid + generation +RuntimeEntityId runtime-local logical identity +RenderEntityId presentation projection identity +AssetHandle immutable content identity + generation +GpuHandle render-thread physical resource identity +``` + +No layer may infer one identity from another through unchecked integer casts. + +--- + +## 5. Content and asset architecture + +### 5.1 Prepared assets + +The runtime world hot path should consume prepared records, not DBObj graphs: + +- Packed mesh vertices, indices, batch/material records, bounds, and part tables. +- Exact decoded texture pixels used by the current renderer. +- EnvCell geometry and portal records with content-deduplicated aliases. +- Terrain and deterministic scenery placement. +- Flattened collision/BSP arrays. +- Compact Setup presentation metadata: + default script, script table, animation, part availability, sound table, + collision references, and effect-relevant flags. +- Retail degrade tables. + +Dynamic character palette/appearance composition, low-volume UI assets, audio, +and motion/animation tables may remain runtime DAT consumers until separately +measured and migrated. + +### 5.2 Content-addressed pak manifest + +Finish the existing pak implementation rather than replacing it: + +```text +Source asset key -> manifest entry -> content hash/shared blob +``` + +- EnvCell file IDs may alias the same blob offset. +- Blob contents are deterministic and 64-byte aligned. +- DAT iteration stamps, serializer version, and bake-tool version invalidate + stale content. +- The writer builds a temporary artifact, validates it, then atomically replaces + the prior artifact. +- The reader memory-maps immutable data and exposes typed borrowed views. +- Corrupt/stale entries fail loudly and name the required rebake. +- Production streaming does not silently fall back to runtime decode after the + cutover. Developer equivalence tools retain both sources explicitly. + +### 5.3 Asset handles and residence + +Callers receive typed handles rather than retaining decoded arrays: + +```csharp +readonly record struct AssetHandle(uint Index, ushort Generation); +readonly record struct AssetLease(AssetHandle Handle, OwnerToken Owner); +``` + +The eventual `ResidencyManager` tracks: + +- CPU prepared bytes. +- Decoded/pinned/staging bytes. +- GPU buffer and texture bytes. +- Current owners and reference count. +- Last used generation/frame. +- Rebuild/reload cost. +- Priority: destination-critical, visible, near, far, speculative. +- State: absent, requested, prepared, upload-pending, resident, retiring. + +Eviction is generation-safe and owner-scoped. A stale completion cannot revive a +retired world generation or release the replacement generation's resource. + +--- + +## 6. Streaming and reveal architecture + +### 6.1 Staged pipeline + +```text +Request + -> I/O/map lookup + -> parse/borrow prepared records + -> build landblock publication + -> physics/collision publish + -> render asset request + -> GL upload + -> render-scene publish + -> reveal-ready +``` + +Every item carries: + +- World/session generation. +- Landblock/cell identity. +- Stage and priority. +- Estimated and actual byte cost. +- Entity-operation count. +- Cancellation token. +- Retryable stage receipt. + +Bounded queues provide back-pressure. Background stages may run concurrently; +authoritative world, render-scene publication, and GL upload remain ordered on +their owning thread. + +### 6.2 Cost budget + +Each frame receives independently configurable budgets: + +- Update-thread publication time. +- Entity create/retire operations. +- CPU bytes adopted. +- GPU bytes uploaded. +- GL resource retire operations. + +Immediate work: + +- Remove old landblocks from visibility, collision, picking, radar, and target + eligibility. +- Mark the old generation unavailable to new consumers. + +Budgeted work: + +- Walk old presentation owners. +- Release scripts/effects/lights/plugin snapshots. +- Retire CPU/GPU resources after fences permit. +- Publish far-ring content. + +This preserves correct visible lifetime while preventing a 600-landblock +retirement from becoming one frame transaction. + +### 6.3 Portal generation and reveal + +A destination generation progresses through: + +```text +Requested -> Prepared -> CollisionReady -> NearSceneReady -> Revealed + | + +-> FarSceneConverging +``` + +Reveal is an atomic edge. The world cannot become visible until the destination +near ring, collision root, camera identity, and required scene publication all +belong to the same generation. Far content continues under normal budgets after +reveal. + +This replaces “priority work bypasses the budget” with a prepared-behind-portal +contract and eliminates `viewport-before-ready`. + +--- + +## 7. Data-oriented render scene + +### 7.1 Scope + +Arch stores render projections only: + +- Static landblock objects and scenery. +- EnvCell objects. +- Live entities and equipped children. +- Lights and effect anchors where a packed projection is beneficial. + +It never owns gameplay properties, network sequencing, physics authority, +inventory, combat, or interaction state. + +### 7.2 Components + +Initial components are blittable or stable handles: + +```text +RenderTransform +PreviousRenderTransform +MeshAsset +MaterialVariant +SpatialResidency (landblock, cell) +WorldBounds +RenderFlags +DegradeState +SortKey +OwnerIncarnation +DirtyMask +``` + +Separate archetypes cover static, dynamic, equipped-child, translucent, and +light-bearing projections. Do not force every entity to carry every component. + +### 7.3 Incremental indices + +Creation, rebucketing, mutation, and removal maintain: + +- Outdoor-static set. +- Per-cell static sets. +- Dynamic set. +- Translucent set. +- Selectable/pickable spatial index. +- Light candidates. +- Dirty transform/material/mesh ranges. + +The frame loop no longer partitions every loaded entity. Existing retail PView +continues to determine visible cells; the render scene enumerates only those +cell buckets plus the dynamic set. + +### 7.4 Simulation-to-render seam + +`LiveEntityRuntime` and static publication emit ordered projection deltas: + +```text +Register +UpdateTransform +UpdateAppearance +UpdateFlags +Rebucket +Unregister +``` + +Deltas contain exact incarnation identity. The render world rejects stale +generation updates. It never calls back into simulation dictionaries while +drawing. + +--- + +## 8. Render-frame product and GPU submission + +The snapshot is a double-buffered, generation-stamped borrowed view: + +```text +RenderFrameView + Visible opaque instance ranges + Visible alpha instance ranges + Dynamic transforms + Light set + Effect draw records + Selection records + Existing PView/clip products +``` + +The update/render order initially remains on the accepted host thread. The +snapshot seam prevents mutation during draw and permits later thread separation +without requiring it now. + +GPU submission evolves in measured steps: + +1. Persistent global mesh/instance buffers remain. +2. Dirty ranges update only changed instance records. +3. MDI command buffers are reused and rewritten in place. +4. Static command templates are cached by scene generation/cell visibility. +5. Accurate timestamp queries bracket actual render passes. +6. GPU culling, command compaction, GPU particles, or GPU light selection are + introduced only when the corrected profile names them as the next bottleneck. + +The existing portal/PView pass graph, clipping, translucency ordering, and +bindless material behavior remain authoritative. + +--- + +## 9. Presentation-independent runtime and headless host + +### 9.1 Extraction rule + +Do not create a new parallel `GameRuntime` and synchronize it with +`LiveEntityRuntime`. Instead: + +1. Define narrow runtime read/command/event contracts around the current owner. +2. Remove presentation-specific fields from that owner into App projections. +3. Move the now presentation-independent owner and collaborators into + `AcDream.Runtime`. +4. Keep the graphical host using the same instance throughout the move. + +### 9.2 `GameRuntime` + +One instance owns: + +- Connection/authentication/character/session lifetime. +- Packet receive ordering and retail update phases. +- Authoritative objects, properties, containers, inventory, spell state, + enchantments, vitals, targets, and combat state. +- Movement, collision state required for gameplay, and interaction commands. +- Instance clock, random sources where applicable, queues, plugins/behaviors, + diagnostics identity, and teardown. + +It exposes: + +- Immutable or borrowed read views. +- Typed commands. +- Ordered events/deltas. +- Deterministic `Tick`. +- Retryable, complete shutdown. + +### 9.3 Hosts + +`GraphicalGameHost` provides input, camera, rendering, UI, audio, frame pacing, +and a single runtime instance. + +`HeadlessGameHost` provides a monotonic scheduler, navigation collision/content, +bot actions, diagnostics, and one or more runtime instances. It performs no +render-content bake lookup unless navigation/collision requires that content. + +For 30 clients: + +- Immutable content and flattened collision assets are shared. +- Every mutable session structure is instance-scoped. +- One process may host many sessions, but one-process-per-session remains a + supported diagnostic/isolation mode. +- Scheduling uses a deterministic round-robin/time-wheel rather than 30 busy + loops. +- Server-safe outbound rate limits are per session. + +--- + +## 10. Detailed implementation sequence + +Each slice is independently buildable, testable, bisectable, and behavior- +preserving. Each gets a focused implementation plan when it starts. + +### Slice A — Measurement and contract correction + +**Purpose:** Make every later gate trustworthy. + +- Correct `GpuFrameTimer` so queries cover actual GL submission/pass intervals, + not frame pacing. +- Extend canonical checkpoint JSON with CPU/GPU resident bytes, staging bytes, + per-stage queue depth, stage work time, exception counts, GC pause/heap + fields, and render-scene generation. +- Add a tracked analysis script that summarizes the existing nine-stop route. +- Record capped-RDP and uncapped-local results separately. +- Add a deterministic allocation/exception attribution recipe without enabling + developer UI or changing gameplay. + +**Gate:** repeated idle measurements have low observer effect; CPU stage sums +reconcile with active frame time; external GPU engine direction agrees with GL +timestamps. + +### Slice B — Finish MP1b EnvCell dedup and full bake + +**Purpose:** Make the existing pak physically usable. + +- Add shared-blob/alias support to `PakWriter` while retaining unique source + keys. +- Compute the existing runtime EnvCell geometry identity before extraction. +- Extract each unique geometry once; map all source file IDs to the shared blob. +- Make side-staged particle preloads and ordinary assets use the same + deterministic content table. +- Add alias, collision, determinism, stale-version, corruption, cancellation, + and atomic-replace tests. +- Run the complete bake and publish counts, unique ratios, size, time, failures, + and peak memory. + +**Gate:** full bake finishes in practical time/space; all fixture and random +sample equivalence tests pass; duplicate EnvCells share blob offsets; no runtime +code consumes the pak yet. + +### Slice C — Runtime prepared-asset source and MP1c cutover + +**Purpose:** Remove mesh/texture decode from portal frames. + +- Introduce `IPreparedAssetSource` in Content and inject it into App streaming. +- Teach `ObjectMeshManager`/landblock builders to adopt pak-backed prepared + payloads without reconstructing complete intermediate object graphs. +- Add compact Setup presentation metadata to the bake or a keyed prepared cache + so static activation does not parse arbitrary IDs as Setup. +- Replace `ResolveId().ToList().OrderBy...` with typed non-allocating lookup. +- Add explicit negative/type metadata; exceptions are not type tests. +- Retain live extraction only in bake/equivalence tooling and required dynamic + appearance paths. +- Remove production streaming fallback at the gate. + +**Gate:** byte/field equivalence; no invalid Setup exception storm; portal +single-frame allocation and p99 materially improve from the 2026-07-24 baseline; +connected nine-stop route and screenshots pass. + +### Slice D — Typed asset handles and unified residency + +**Purpose:** Bound and explain CPU/GPU memory. + +- Add `AssetHandle`, owner tokens, leases, generations, and accounting. +- Place object mesh, standalone texture, composite texture-array, staging, and + prepared-content residence behind one policy owner while retaining specialized + physical caches. +- Track logical, CPU, staging, GPU-requested, GPU-resident, and retiring bytes + separately. +- Add configurable budgets through `RuntimeOptions`/settings with current visual + behavior as the default. +- Implement generation-safe LRU/cost-aware eviction and fence-delayed physical + release. +- Expose exact budget/occupancy/fragmentation facts to diagnostics. + +**Gate:** same-location third-visit residence plateaus; no stale-generation +release; no missing textures; no resource growth after repeated portal loops. + +### Slice E — Cost-budgeted streaming and retirement + +**Purpose:** Remove update-thread portal transactions. + +- Introduce explicit stage queues and `StreamingWorkBudget`. +- Split immediate logical/spatial detach from budgeted owner/resource teardown. +- Make per-entity retirement cursor/time bounded. +- Make publication cursor/time/byte bounded. +- Reserve destination-critical work across portal frames rather than bypassing + all budgets on one frame. +- Preserve FIFO within priority/generation and exact retry receipts. +- Connect reveal to the single destination-generation readiness state. + +**Gate:** zero `viewport-before-ready`; traversal p99 at or below 16.67 ms target +and maximum below 33.3 ms target on the reference local run; no stranded old +generation, staged upload, collision, effect, or GPU owner. + +### Slice F — Incremental render scene foundation + +**Purpose:** Stop rebuilding/partitioning the world each frame. + +- Add acdream-owned `IRenderScene`, identifiers, components, and delta journal. +- Implement it with Arch in App only. +- Mirror static publication and live projection into a non-drawing shadow world. +- Add deterministic scene digest and compare it with current owners at + checkpoints. +- Maintain cell/outdoor/dynamic/translucent/light indices incrementally. +- Add stale-incarnation, duplicate-create, rebucket, delete/recreate, hidden, + parent-child, and session-reset tests. + +**Gate:** shadow scene matches canonical world/resource checkpoints through the +nine-stop route with bounded memory and zero authoritative ownership. + +### Slice G — Render snapshot and delta submission cutover + +**Purpose:** Make render cost proportional to visible/changed data. + +- Build the double-buffered `RenderFrameView`. +- Feed it the existing PView visible-cell/clip product. +- Replace full `InteriorEntityPartition` scans with render-scene bucket queries. +- Upload only dirty persistent instance ranges. +- Reuse MDI command/sort buffers. +- Build instance-set comparison and fixed-camera screenshot comparison. +- Run old and new submissions in compare mode without drawing twice. +- At the gate, delete the replaced production enumeration/submission path. + +**Gate:** instance sets and screenshots match; dense-town uncapped target is +300 FPS or the corrected profile identifies a new dominant stage; steady frame +allocation is near zero. + +### Slice H — Event-driven UI, diagnostics, lights, and frame cleanup + +**Purpose:** Remove remaining work that scales with uncapped FPS. + +- Skip diagnostics with no consumer and reuse renderer visibility facts. +- Dirty-layout retained UI: apply anchors only after geometry changes. +- Maintain an overlay-participant registry instead of a second full-tree walk. +- Track active cooldown items/effects/dialogs through events. +- Replace per-frame frame-input objects, LINQ arrays, iterator objects, and + liveness collections with reusable storage or borrowed views. +- Make light candidates spatial/cell-driven and select top-k without a complete + per-frame sort. +- Replace socket-timeout exceptions and datagram `ToArray` copies with normal + cancellable/poll/span-based I/O. + +**Gate:** no steady-state Gen-2 collections; steady allocation target +≤ 4 KiB/frame initially and zero for the core world/render loop; UI and network +behavior tests unchanged. + +### Slice I — Flat collision assets and residual zero-allocation work + +**Purpose:** Complete MP4 without changing retail math. + +- Bake flattened index-based BSP/collision records. +- Port traversal data access line-for-line while preserving ordering and + arithmetic. +- Remove parsed DBObj graphs from streaming collision publication. +- Address remaining measured allocation sites, including any physics transition + pooling only after identity/lifetime tests prove it safe. + +**Gate:** trajectory and retail conformance suites remain bit-equivalent; +navigation/collision fixtures match; portal and steady allocation targets pass. + +### Slice J — Presentation-independent `AcDream.Runtime` + +**Purpose:** Establish the graphical/headless shared client kernel. + +- Add the Runtime project and dependency guards. +- Define `IGameRuntimeView`, commands, ordered events, clock, and lifecycle. +- Adapt the current graphical host to those interfaces first. +- Remove App presentation dependencies from canonical gameplay owners. +- Move owners by coherent lifetime groups; never mirror state. +- Add host-parity and no-backend construction tests after each move. +- End with `GameWindow`/App composing one `GameRuntime`. + +**Gate:** the connected graphical route is unchanged; a no-window integration +test connects, enters world, moves, receives inventory/chat/world updates, +portals, logs out, reconnects, and tears down without loading App/Silk/OpenAL. + +### Slice K — Linux headless and multi-session host + +**Purpose:** Deliver efficient automated bots. + +- Add Linux CI for Core, Core.Net, Content, Runtime, and Headless. +- Implement portable path/config/credential handling. +- Add `AcDream.Headless` CLI and deterministic scheduler. +- Add bot command/event APIs for movement, selection, use, combat, spells, + looting, chat, and commands. +- Share immutable content/collision stores across sessions. +- Audit and eliminate mutable statics and process-wide session state. +- Add clean cancellation, reconnect, plugin isolation, and credential-safe logs. +- Stress 1, 5, 10, and 30 sessions. + +**Gate:** 30 local-server sessions in one Linux process, no GPU/display/audio +dependency, bounded incremental memory per session, no busy-loop CPU, clean +teardown/reconnect, and parity with graphical runtime command/event behavior. + +### Slice L — Linux graphical host and evidence-gated GPU work + +**Purpose:** Finish platform portability and only then pursue remaining GPU +opportunities. + +- Validate Linux OpenGL extension/driver matrix and package native dependencies. +- Port paths, frame pacing, input, audio, and packaging without renderer + fallback. +- Reprofile locally with accurate pass timing. +- Add GPU culling, command compaction, particle simulation, or light selection + only for stages proven dominant. + +**Gate:** graphical Linux connected route passes the same lifecycle, screenshot, +resource, and performance checks; every GPU migration has a CPU/GPU before/after +and visual equivalence result. + +--- + +## 11. Performance gates + +Reference measurements use a Release build, the same account/route, stable +camera scripts, and both capped and uncapped modes. RDP results are kept +separate from local-display results. + +| Metric | Target | +|---|---:| +| Traversal frame p99 | ≤ 16.67 ms on reference local hardware | +| Traversal maximum | ≤ 33.3 ms after warm process startup | +| Dense-town uncapped frame p50 | ≤ 3.33 ms or a newly attributed blocker | +| Steady update p95 | ≤ 2 ms | +| Core world/render allocation | 0 B/frame after warmup | +| Whole-client steady allocation | initial gate ≤ 4 KiB/frame | +| Portal single-frame allocation | ≤ 4 MiB, then tighten from evidence | +| Gen-2 collections during canonical route | 0 after startup/bake warmup | +| Same-location third-visit resource growth | ≤ 5% and explained | +| World-visible-before-ready events | 0 | +| Staged resources at stable checkpoints | 0 | +| Exception-as-control-flow | 0 known sites | +| Headless renderer/audio/window allocations | 0 | + +Memory budgets are measured by category rather than only process working set: + +- Live managed heap. +- GC committed/fragmented. +- Native prepared/staging buffers. +- Mapped content pages. +- Tracked GPU buffers/textures. +- Driver-reported dedicated/shared GPU memory. + +Absolute process/VRAM targets are fixed after Slice D produces trustworthy +category accounting; until then, the binding rule is plateau plus no regression +in pixels/range. + +--- + +## 12. Test and review matrix + +Every slice must pass the tests relevant to its boundary: + +1. **Pure unit tests:** handles, generations, budgets, queues, manifest, + deduplication, stale/corrupt data, snapshot buffers. +2. **DAT equivalence:** live extractor versus prepared asset for representative + outdoor, dungeon, town, portal, animated, translucent, and particle assets. +3. **Scene equivalence:** canonical entity/instance digest at fixed checkpoints. +4. **Render equivalence:** instance-set diff plus fixed-camera screenshots. +5. **Retail conformance:** named-retail citations for any AC-specific selection, + degradation, visibility, collision, or timing behavior touched. +6. **Lifecycle:** fresh login, same-location revisit, world edge, dungeon, + repeated recalls, rapid generation replacement, logout/reconnect, graceful + close, cancellation, and failure injection. +7. **Resource:** exact create/retire counts, heap diff, GPU accounting, no stale + owner or queued work at stable checkpoints. +8. **Performance:** capped and uncapped CPU/GPU/allocation/GC percentiles. +9. **Headless parity:** same runtime packet/order/command results with + presentation attached or absent. +10. **Platform:** Windows graphical/headless plus Linux headless; Linux graphical + begins in Slice L. + +For a replacement cutover: + +- Shadow/compare mode may exist only during development. +- Confirmed differences are fixed at their source. +- The gate commit removes the old production path. +- No test threshold is loosened to accept a regression. + +--- + +## 13. Risks and controls + +| Risk | Control | +|---|---| +| Pak repeats the 865 GB failure | content identity and aliasing land before another full bake; bake reports unique/shared ratios continuously | +| Prepared data changes pixels/geometry | byte/field equivalence against the current extractor; no compression in the equivalence phase | +| Streaming budget reveals an incomplete world | one destination-generation readiness barrier; reveal is an atomic state edge | +| Deferred retirement leaks or revives stale resources | incarnation/generation owner tokens and stable-checkpoint zero-outstanding assertions | +| Render ECS becomes a second game world | render-only component contract; no network properties or authoritative GUID map; digest checks against canonical owner | +| Snapshot adds a frame of latency | same-thread borrowed snapshot first; frame-number assertions; concurrency remains deferred | +| Arch dependency constrains the design | acdream-owned `IRenderScene`; Arch only in App; no Arch types cross the boundary | +| Headless extraction duplicates state | move the existing canonical owner in lifetime groups; adapters first, never synchronize two stores | +| Shared 30-client cache gains mutable state | immutable content only; tests create conflicting sessions/characters concurrently | +| Linux work forces renderer fallback | graphical host retains mandatory modern extensions and fails clearly on unsupported drivers | +| “Optimization” changes retail behavior | unchanged behavior is a hard gate; retail ports remain OO and traceable; visual/connected gates stay mandatory | + +--- + +## 14. Recommended starting point + +Begin with **Slice A**, then **Slice B**. + +Slice A is small and makes all later GPU and frame gates honest. Slice B completes +an implementation already present in the tree and directly attacks the +benchmark's largest allocation source. The first meaningful runtime cutover is +Slice C; do not begin the render-scene migration before the portal content storm +has been removed and measured. + +The intended order is therefore: + +```text +honest metrics + -> prepared content + -> typed residence + -> cost-budgeted streaming + -> incremental render scene + -> delta GPU submission + -> residual frame cleanup + -> presentation-independent runtime + -> Linux/headless/multi-session + -> evidence-gated GPU jobs +``` + +This order improves the current graphical client first while laying clean +boundaries for Linux and approximately 30 automated headless clients later.