Unify the unfinished Modern Pipeline work with Linux/headless goals around measured content, residency, streaming, render-scene, and runtime boundaries. Record twelve gated slices and preserve retail-shaped simulation plus the mandatory N.5 renderer. Co-authored-by: Codex <codex@openai.com>
797 lines
32 KiB
Markdown
797 lines
32 KiB
Markdown
# Modern Runtime Architecture — detailed execution plan
|
||
|
||
**Date:** 2026-07-24
|
||
|
||
**Status:** Planning complete; implementation not started
|
||
|
||
**Scope:** Reconcile and sequence the existing Modern Pipeline (`MP`) and
|
||
Linux/headless (`LH`) tracks using the 2026-07-24 connected performance audit.
|
||
|
||
**Primary objective:** Minimize CPU, managed/native memory, GPU memory,
|
||
allocation, and frame-time variance without reducing view distance, particles,
|
||
world detail, or retail behavior.
|
||
|
||
This plan refines rather than replaces
|
||
[`docs/superpowers/specs/2026-07-05-modern-pipeline-design.md`](../superpowers/specs/2026-07-05-modern-pipeline-design.md).
|
||
It preserves the shipped N.5 bindless/MDI renderer, the completed `GameWindow`
|
||
decomposition, canonical `LiveEntityRuntime` ownership, the retail PView/cell
|
||
visibility ports, and all behavior already accepted by the user.
|
||
|
||
---
|
||
|
||
## 1. Why the architecture should change
|
||
|
||
The 2026-07-24 six-minute, nine-stop connected route established:
|
||
|
||
| Observation | Measurement | Architectural implication |
|
||
|---|---:|---|
|
||
| Steady capped update work | approximately 0.4–1.3 ms median | The gameplay/update architecture is not the steady-state crisis |
|
||
| External GPU use | 7.6% median, 15.9% p95 | The tested route was not GPU-bound |
|
||
| Portal allocation | up to approximately 204 MiB in one frame | Runtime content preparation and publication must leave the frame transaction |
|
||
| Managed allocation rate | approximately 40.8 MB/s average, 2.22 GiB/s maximum | Portal work is creating short-lived graphs and pixel/mesh arrays faster than GC can absorb |
|
||
| GC | 38 Gen-2 collections; 256 ms longest pause | Frame spikes are dominated by allocation/publication, not upload time alone |
|
||
| Longest frame | 276 ms | Landblock-count budgets are not real cost budgets |
|
||
| Final live GC heap | approximately 194 MiB | The 1–2 GiB process high-water is not an unbounded live-object leak |
|
||
| Dedicated GPU memory | 633–818 MiB | Texture arrays/caches are bounded but expensive and insufficiently observable |
|
||
| Revisited locations | entity/emitter/particle counts returned to the same plateaus | Logical lifetime is broadly convergent; optimize rather than replace it |
|
||
| Exception traffic | 4,520 invalid DAT probes plus 1,168 receive timeouts | Exceptions are being used as normal control flow |
|
||
|
||
Allocation attribution names the runtime mesh/texture path
|
||
(`MeshExtractor.PrepareMeshData` → `PrepareGfxObjMeshData` →
|
||
`AddSurfaceToBatch`) as the largest portal source. The update-thread
|
||
publication/retirement graph then amplifies it with complete dictionaries,
|
||
sorts, effect profiles, and unmetered per-entity teardown.
|
||
|
||
The conclusion is:
|
||
|
||
> Keep retail-shaped gameplay and the modern GPU backend. Replace the runtime
|
||
> content, scene-projection, residency, and work-scheduling architecture that
|
||
> feeds the renderer.
|
||
|
||
---
|
||
|
||
## 2. Fixed architectural decisions
|
||
|
||
These decisions are binding unless a later measured gate disproves one.
|
||
|
||
1. **No whole-client rewrite.** Existing retail ports, network behavior,
|
||
physics, animation, UI, plugins, and the N.5 renderer remain in place.
|
||
2. **No simulation ECS.** Retail-shaped object-oriented code remains the oracle-
|
||
traceable form for physics, motion, sequencers, combat, inventory, and magic.
|
||
3. **A data-oriented render world only.** Arch remains the selected storage
|
||
implementation, hidden behind acdream-owned interfaces so it can be replaced
|
||
without changing simulation or rendering contracts.
|
||
4. **One authoritative live-object owner.** `LiveEntityRuntime` is evolved and
|
||
later extracted; no second GUID map, gameplay world, or mirrored entity state
|
||
may be introduced during migration.
|
||
5. **Headless is not a hidden window.** A headless session constructs no
|
||
Silk.NET window, GL context, render assets, particles, audio, or gameplay UI.
|
||
6. **One content interpretation.** Bake and runtime validation use the exact
|
||
existing `AcDream.Content` interpretation code. A baked artifact must be
|
||
byte/field equivalent before runtime uses it.
|
||
7. **No visual-quality optimization by default.** View horizon, retail PView,
|
||
particles, effects, and texture pixels stay unchanged. Retail distance LOD is
|
||
a faithful mechanism and remains separately configurable.
|
||
8. **Work is budgeted by cost, not object count.** Streaming stages use elapsed
|
||
microseconds, prepared bytes, uploaded bytes, and entity operations. “Four
|
||
landblocks” is not a meaningful budget.
|
||
9. **GL remains render-thread owned.** Workers may perform I/O, parsing,
|
||
decompression, mesh preparation, and collision preparation, but never mutate
|
||
GL or live scene state.
|
||
10. **Frame products are borrowed immutable views, not object graphs.** Snapshot
|
||
boundaries use double-buffered arrays/spans and generation stamps; they do
|
||
not allocate a deep immutable tree each frame.
|
||
11. **Parallelism follows data cleanup.** Do not add a job system to compensate
|
||
for repeated decode, sorting, allocation, or whole-world scans.
|
||
12. **Every cutover deletes the replaced production path.** Temporary shadow
|
||
execution is allowed for comparison, but no permanent dual renderer,
|
||
duplicate world, or silent decode fallback remains after its gate.
|
||
|
||
---
|
||
|
||
## 3. Target dependency architecture
|
||
|
||
```text
|
||
AcDream.GraphicalHost (current AcDream.App)
|
||
├── window/input/OpenGL/OpenAL/retail UI
|
||
├── render scene projection + renderer
|
||
└── owns one AcDream.Runtime.GameRuntime
|
||
|
||
AcDream.Headless
|
||
├── deterministic host clock
|
||
├── bot/plugin command surface
|
||
└── owns one or more AcDream.Runtime.GameRuntime instances
|
||
|
||
AcDream.Runtime [new, extracted incrementally]
|
||
├── session lifecycle and inbound ordering
|
||
├── authoritative live world and properties
|
||
├── movement/physics/interaction/combat/magic/inventory/chat
|
||
├── instance-scoped clocks, queues, commands, and events
|
||
└── no Silk.NET, GL, OpenAL, retained UI, or OS window dependency
|
||
|
||
AcDream.Content [already exists]
|
||
├── DAT interpretation
|
||
├── deterministic bake pipeline
|
||
├── mapped pak/content manifest
|
||
├── immutable prepared render/collision metadata
|
||
└── shared read-only content store for multiple sessions
|
||
|
||
AcDream.Core / AcDream.Core.Net [existing retail-shaped logic]
|
||
AcDream.Plugin.Abstractions [existing BCL-only contracts]
|
||
```
|
||
|
||
`AcDream.Runtime` is a target boundary, not the first implementation step. App
|
||
types move only after their presentation dependencies have been removed and
|
||
parity tests prove that the graphical host is still driving the same owner.
|
||
|
||
### Dependency rules
|
||
|
||
- `Core` never depends on Runtime, App, Content, or a backend.
|
||
- `Core.Net` depends on Core only.
|
||
- `Content` depends on Core, never App or a graphics backend.
|
||
- `Runtime` may depend on Core, Core.Net, Content, and Plugin.Abstractions.
|
||
- The graphical and headless hosts depend on Runtime; Runtime never depends on
|
||
either host.
|
||
- UI and rendering consume read-only runtime views plus explicit commands/events.
|
||
- Shared multi-session caches contain immutable content only. Credentials,
|
||
GUIDs, clocks, packets, plugins, object state, and automation remain
|
||
instance-scoped.
|
||
|
||
---
|
||
|
||
## 4. Runtime ownership model
|
||
|
||
| Concern | Canonical owner | Consumers |
|
||
|---|---|---|
|
||
| Server GUID/incarnation/timestamps/properties | `GameRuntime` evolved from `LiveEntityRuntime` | physics, UI, render projection, plugins, bots |
|
||
| Static prepared content | `ContentStore` | streaming, collision, render projection |
|
||
| Desired spatial window | `StreamingRegion` | streaming scheduler |
|
||
| Landblock generation and stage receipts | `StreamingWorkScheduler` | reveal, physics publication, render publication |
|
||
| Render entity/component data | `IRenderScene` / Arch implementation | visibility, snapshot builder |
|
||
| CPU/GPU asset residence | `ResidencyManager` | render scene, upload owner |
|
||
| GL names, fences, physical retirement | renderer resource owners | renderer only |
|
||
| Retail PView/visible-cell result | existing PView owner | render-scene query/snapshot builder |
|
||
| UI state | focused UI controllers/ViewModels | retained UI and optional dev UI |
|
||
|
||
Identity types must remain explicit:
|
||
|
||
```text
|
||
ServerGuid authoritative network identity
|
||
ObjectIncarnation ServerGuid + generation
|
||
RuntimeEntityId runtime-local logical identity
|
||
RenderEntityId presentation projection identity
|
||
AssetHandle<T> immutable content identity + generation
|
||
GpuHandle<T> render-thread physical resource identity
|
||
```
|
||
|
||
No layer may infer one identity from another through unchecked integer casts.
|
||
|
||
---
|
||
|
||
## 5. Content and asset architecture
|
||
|
||
### 5.1 Prepared assets
|
||
|
||
The runtime world hot path should consume prepared records, not DBObj graphs:
|
||
|
||
- Packed mesh vertices, indices, batch/material records, bounds, and part tables.
|
||
- Exact decoded texture pixels used by the current renderer.
|
||
- EnvCell geometry and portal records with content-deduplicated aliases.
|
||
- Terrain and deterministic scenery placement.
|
||
- Flattened collision/BSP arrays.
|
||
- Compact Setup presentation metadata:
|
||
default script, script table, animation, part availability, sound table,
|
||
collision references, and effect-relevant flags.
|
||
- Retail degrade tables.
|
||
|
||
Dynamic character palette/appearance composition, low-volume UI assets, audio,
|
||
and motion/animation tables may remain runtime DAT consumers until separately
|
||
measured and migrated.
|
||
|
||
### 5.2 Content-addressed pak manifest
|
||
|
||
Finish the existing pak implementation rather than replacing it:
|
||
|
||
```text
|
||
Source asset key -> manifest entry -> content hash/shared blob
|
||
```
|
||
|
||
- EnvCell file IDs may alias the same blob offset.
|
||
- Blob contents are deterministic and 64-byte aligned.
|
||
- DAT iteration stamps, serializer version, and bake-tool version invalidate
|
||
stale content.
|
||
- The writer builds a temporary artifact, validates it, then atomically replaces
|
||
the prior artifact.
|
||
- The reader memory-maps immutable data and exposes typed borrowed views.
|
||
- Corrupt/stale entries fail loudly and name the required rebake.
|
||
- Production streaming does not silently fall back to runtime decode after the
|
||
cutover. Developer equivalence tools retain both sources explicitly.
|
||
|
||
### 5.3 Asset handles and residence
|
||
|
||
Callers receive typed handles rather than retaining decoded arrays:
|
||
|
||
```csharp
|
||
readonly record struct AssetHandle<T>(uint Index, ushort Generation);
|
||
readonly record struct AssetLease<T>(AssetHandle<T> Handle, OwnerToken Owner);
|
||
```
|
||
|
||
The eventual `ResidencyManager` tracks:
|
||
|
||
- CPU prepared bytes.
|
||
- Decoded/pinned/staging bytes.
|
||
- GPU buffer and texture bytes.
|
||
- Current owners and reference count.
|
||
- Last used generation/frame.
|
||
- Rebuild/reload cost.
|
||
- Priority: destination-critical, visible, near, far, speculative.
|
||
- State: absent, requested, prepared, upload-pending, resident, retiring.
|
||
|
||
Eviction is generation-safe and owner-scoped. A stale completion cannot revive a
|
||
retired world generation or release the replacement generation's resource.
|
||
|
||
---
|
||
|
||
## 6. Streaming and reveal architecture
|
||
|
||
### 6.1 Staged pipeline
|
||
|
||
```text
|
||
Request
|
||
-> I/O/map lookup
|
||
-> parse/borrow prepared records
|
||
-> build landblock publication
|
||
-> physics/collision publish
|
||
-> render asset request
|
||
-> GL upload
|
||
-> render-scene publish
|
||
-> reveal-ready
|
||
```
|
||
|
||
Every item carries:
|
||
|
||
- World/session generation.
|
||
- Landblock/cell identity.
|
||
- Stage and priority.
|
||
- Estimated and actual byte cost.
|
||
- Entity-operation count.
|
||
- Cancellation token.
|
||
- Retryable stage receipt.
|
||
|
||
Bounded queues provide back-pressure. Background stages may run concurrently;
|
||
authoritative world, render-scene publication, and GL upload remain ordered on
|
||
their owning thread.
|
||
|
||
### 6.2 Cost budget
|
||
|
||
Each frame receives independently configurable budgets:
|
||
|
||
- Update-thread publication time.
|
||
- Entity create/retire operations.
|
||
- CPU bytes adopted.
|
||
- GPU bytes uploaded.
|
||
- GL resource retire operations.
|
||
|
||
Immediate work:
|
||
|
||
- Remove old landblocks from visibility, collision, picking, radar, and target
|
||
eligibility.
|
||
- Mark the old generation unavailable to new consumers.
|
||
|
||
Budgeted work:
|
||
|
||
- Walk old presentation owners.
|
||
- Release scripts/effects/lights/plugin snapshots.
|
||
- Retire CPU/GPU resources after fences permit.
|
||
- Publish far-ring content.
|
||
|
||
This preserves correct visible lifetime while preventing a 600-landblock
|
||
retirement from becoming one frame transaction.
|
||
|
||
### 6.3 Portal generation and reveal
|
||
|
||
A destination generation progresses through:
|
||
|
||
```text
|
||
Requested -> Prepared -> CollisionReady -> NearSceneReady -> Revealed
|
||
|
|
||
+-> FarSceneConverging
|
||
```
|
||
|
||
Reveal is an atomic edge. The world cannot become visible until the destination
|
||
near ring, collision root, camera identity, and required scene publication all
|
||
belong to the same generation. Far content continues under normal budgets after
|
||
reveal.
|
||
|
||
This replaces “priority work bypasses the budget” with a prepared-behind-portal
|
||
contract and eliminates `viewport-before-ready`.
|
||
|
||
---
|
||
|
||
## 7. Data-oriented render scene
|
||
|
||
### 7.1 Scope
|
||
|
||
Arch stores render projections only:
|
||
|
||
- Static landblock objects and scenery.
|
||
- EnvCell objects.
|
||
- Live entities and equipped children.
|
||
- Lights and effect anchors where a packed projection is beneficial.
|
||
|
||
It never owns gameplay properties, network sequencing, physics authority,
|
||
inventory, combat, or interaction state.
|
||
|
||
### 7.2 Components
|
||
|
||
Initial components are blittable or stable handles:
|
||
|
||
```text
|
||
RenderTransform
|
||
PreviousRenderTransform
|
||
MeshAsset
|
||
MaterialVariant
|
||
SpatialResidency (landblock, cell)
|
||
WorldBounds
|
||
RenderFlags
|
||
DegradeState
|
||
SortKey
|
||
OwnerIncarnation
|
||
DirtyMask
|
||
```
|
||
|
||
Separate archetypes cover static, dynamic, equipped-child, translucent, and
|
||
light-bearing projections. Do not force every entity to carry every component.
|
||
|
||
### 7.3 Incremental indices
|
||
|
||
Creation, rebucketing, mutation, and removal maintain:
|
||
|
||
- Outdoor-static set.
|
||
- Per-cell static sets.
|
||
- Dynamic set.
|
||
- Translucent set.
|
||
- Selectable/pickable spatial index.
|
||
- Light candidates.
|
||
- Dirty transform/material/mesh ranges.
|
||
|
||
The frame loop no longer partitions every loaded entity. Existing retail PView
|
||
continues to determine visible cells; the render scene enumerates only those
|
||
cell buckets plus the dynamic set.
|
||
|
||
### 7.4 Simulation-to-render seam
|
||
|
||
`LiveEntityRuntime` and static publication emit ordered projection deltas:
|
||
|
||
```text
|
||
Register
|
||
UpdateTransform
|
||
UpdateAppearance
|
||
UpdateFlags
|
||
Rebucket
|
||
Unregister
|
||
```
|
||
|
||
Deltas contain exact incarnation identity. The render world rejects stale
|
||
generation updates. It never calls back into simulation dictionaries while
|
||
drawing.
|
||
|
||
---
|
||
|
||
## 8. Render-frame product and GPU submission
|
||
|
||
The snapshot is a double-buffered, generation-stamped borrowed view:
|
||
|
||
```text
|
||
RenderFrameView
|
||
Visible opaque instance ranges
|
||
Visible alpha instance ranges
|
||
Dynamic transforms
|
||
Light set
|
||
Effect draw records
|
||
Selection records
|
||
Existing PView/clip products
|
||
```
|
||
|
||
The update/render order initially remains on the accepted host thread. The
|
||
snapshot seam prevents mutation during draw and permits later thread separation
|
||
without requiring it now.
|
||
|
||
GPU submission evolves in measured steps:
|
||
|
||
1. Persistent global mesh/instance buffers remain.
|
||
2. Dirty ranges update only changed instance records.
|
||
3. MDI command buffers are reused and rewritten in place.
|
||
4. Static command templates are cached by scene generation/cell visibility.
|
||
5. Accurate timestamp queries bracket actual render passes.
|
||
6. GPU culling, command compaction, GPU particles, or GPU light selection are
|
||
introduced only when the corrected profile names them as the next bottleneck.
|
||
|
||
The existing portal/PView pass graph, clipping, translucency ordering, and
|
||
bindless material behavior remain authoritative.
|
||
|
||
---
|
||
|
||
## 9. Presentation-independent runtime and headless host
|
||
|
||
### 9.1 Extraction rule
|
||
|
||
Do not create a new parallel `GameRuntime` and synchronize it with
|
||
`LiveEntityRuntime`. Instead:
|
||
|
||
1. Define narrow runtime read/command/event contracts around the current owner.
|
||
2. Remove presentation-specific fields from that owner into App projections.
|
||
3. Move the now presentation-independent owner and collaborators into
|
||
`AcDream.Runtime`.
|
||
4. Keep the graphical host using the same instance throughout the move.
|
||
|
||
### 9.2 `GameRuntime`
|
||
|
||
One instance owns:
|
||
|
||
- Connection/authentication/character/session lifetime.
|
||
- Packet receive ordering and retail update phases.
|
||
- Authoritative objects, properties, containers, inventory, spell state,
|
||
enchantments, vitals, targets, and combat state.
|
||
- Movement, collision state required for gameplay, and interaction commands.
|
||
- Instance clock, random sources where applicable, queues, plugins/behaviors,
|
||
diagnostics identity, and teardown.
|
||
|
||
It exposes:
|
||
|
||
- Immutable or borrowed read views.
|
||
- Typed commands.
|
||
- Ordered events/deltas.
|
||
- Deterministic `Tick`.
|
||
- Retryable, complete shutdown.
|
||
|
||
### 9.3 Hosts
|
||
|
||
`GraphicalGameHost` provides input, camera, rendering, UI, audio, frame pacing,
|
||
and a single runtime instance.
|
||
|
||
`HeadlessGameHost` provides a monotonic scheduler, navigation collision/content,
|
||
bot actions, diagnostics, and one or more runtime instances. It performs no
|
||
render-content bake lookup unless navigation/collision requires that content.
|
||
|
||
For 30 clients:
|
||
|
||
- Immutable content and flattened collision assets are shared.
|
||
- Every mutable session structure is instance-scoped.
|
||
- One process may host many sessions, but one-process-per-session remains a
|
||
supported diagnostic/isolation mode.
|
||
- Scheduling uses a deterministic round-robin/time-wheel rather than 30 busy
|
||
loops.
|
||
- Server-safe outbound rate limits are per session.
|
||
|
||
---
|
||
|
||
## 10. Detailed implementation sequence
|
||
|
||
Each slice is independently buildable, testable, bisectable, and behavior-
|
||
preserving. Each gets a focused implementation plan when it starts.
|
||
|
||
### Slice A — Measurement and contract correction
|
||
|
||
**Purpose:** Make every later gate trustworthy.
|
||
|
||
- Correct `GpuFrameTimer` so queries cover actual GL submission/pass intervals,
|
||
not frame pacing.
|
||
- Extend canonical checkpoint JSON with CPU/GPU resident bytes, staging bytes,
|
||
per-stage queue depth, stage work time, exception counts, GC pause/heap
|
||
fields, and render-scene generation.
|
||
- Add a tracked analysis script that summarizes the existing nine-stop route.
|
||
- Record capped-RDP and uncapped-local results separately.
|
||
- Add a deterministic allocation/exception attribution recipe without enabling
|
||
developer UI or changing gameplay.
|
||
|
||
**Gate:** repeated idle measurements have low observer effect; CPU stage sums
|
||
reconcile with active frame time; external GPU engine direction agrees with GL
|
||
timestamps.
|
||
|
||
### Slice B — Finish MP1b EnvCell dedup and full bake
|
||
|
||
**Purpose:** Make the existing pak physically usable.
|
||
|
||
- Add shared-blob/alias support to `PakWriter` while retaining unique source
|
||
keys.
|
||
- Compute the existing runtime EnvCell geometry identity before extraction.
|
||
- Extract each unique geometry once; map all source file IDs to the shared blob.
|
||
- Make side-staged particle preloads and ordinary assets use the same
|
||
deterministic content table.
|
||
- Add alias, collision, determinism, stale-version, corruption, cancellation,
|
||
and atomic-replace tests.
|
||
- Run the complete bake and publish counts, unique ratios, size, time, failures,
|
||
and peak memory.
|
||
|
||
**Gate:** full bake finishes in practical time/space; all fixture and random
|
||
sample equivalence tests pass; duplicate EnvCells share blob offsets; no runtime
|
||
code consumes the pak yet.
|
||
|
||
### Slice C — Runtime prepared-asset source and MP1c cutover
|
||
|
||
**Purpose:** Remove mesh/texture decode from portal frames.
|
||
|
||
- Introduce `IPreparedAssetSource` in Content and inject it into App streaming.
|
||
- Teach `ObjectMeshManager`/landblock builders to adopt pak-backed prepared
|
||
payloads without reconstructing complete intermediate object graphs.
|
||
- Add compact Setup presentation metadata to the bake or a keyed prepared cache
|
||
so static activation does not parse arbitrary IDs as Setup.
|
||
- Replace `ResolveId().ToList().OrderBy...` with typed non-allocating lookup.
|
||
- Add explicit negative/type metadata; exceptions are not type tests.
|
||
- Retain live extraction only in bake/equivalence tooling and required dynamic
|
||
appearance paths.
|
||
- Remove production streaming fallback at the gate.
|
||
|
||
**Gate:** byte/field equivalence; no invalid Setup exception storm; portal
|
||
single-frame allocation and p99 materially improve from the 2026-07-24 baseline;
|
||
connected nine-stop route and screenshots pass.
|
||
|
||
### Slice D — Typed asset handles and unified residency
|
||
|
||
**Purpose:** Bound and explain CPU/GPU memory.
|
||
|
||
- Add `AssetHandle<T>`, owner tokens, leases, generations, and accounting.
|
||
- Place object mesh, standalone texture, composite texture-array, staging, and
|
||
prepared-content residence behind one policy owner while retaining specialized
|
||
physical caches.
|
||
- Track logical, CPU, staging, GPU-requested, GPU-resident, and retiring bytes
|
||
separately.
|
||
- Add configurable budgets through `RuntimeOptions`/settings with current visual
|
||
behavior as the default.
|
||
- Implement generation-safe LRU/cost-aware eviction and fence-delayed physical
|
||
release.
|
||
- Expose exact budget/occupancy/fragmentation facts to diagnostics.
|
||
|
||
**Gate:** same-location third-visit residence plateaus; no stale-generation
|
||
release; no missing textures; no resource growth after repeated portal loops.
|
||
|
||
### Slice E — Cost-budgeted streaming and retirement
|
||
|
||
**Purpose:** Remove update-thread portal transactions.
|
||
|
||
- Introduce explicit stage queues and `StreamingWorkBudget`.
|
||
- Split immediate logical/spatial detach from budgeted owner/resource teardown.
|
||
- Make per-entity retirement cursor/time bounded.
|
||
- Make publication cursor/time/byte bounded.
|
||
- Reserve destination-critical work across portal frames rather than bypassing
|
||
all budgets on one frame.
|
||
- Preserve FIFO within priority/generation and exact retry receipts.
|
||
- Connect reveal to the single destination-generation readiness state.
|
||
|
||
**Gate:** zero `viewport-before-ready`; traversal p99 at or below 16.67 ms target
|
||
and maximum below 33.3 ms target on the reference local run; no stranded old
|
||
generation, staged upload, collision, effect, or GPU owner.
|
||
|
||
### Slice F — Incremental render scene foundation
|
||
|
||
**Purpose:** Stop rebuilding/partitioning the world each frame.
|
||
|
||
- Add acdream-owned `IRenderScene`, identifiers, components, and delta journal.
|
||
- Implement it with Arch in App only.
|
||
- Mirror static publication and live projection into a non-drawing shadow world.
|
||
- Add deterministic scene digest and compare it with current owners at
|
||
checkpoints.
|
||
- Maintain cell/outdoor/dynamic/translucent/light indices incrementally.
|
||
- Add stale-incarnation, duplicate-create, rebucket, delete/recreate, hidden,
|
||
parent-child, and session-reset tests.
|
||
|
||
**Gate:** shadow scene matches canonical world/resource checkpoints through the
|
||
nine-stop route with bounded memory and zero authoritative ownership.
|
||
|
||
### Slice G — Render snapshot and delta submission cutover
|
||
|
||
**Purpose:** Make render cost proportional to visible/changed data.
|
||
|
||
- Build the double-buffered `RenderFrameView`.
|
||
- Feed it the existing PView visible-cell/clip product.
|
||
- Replace full `InteriorEntityPartition` scans with render-scene bucket queries.
|
||
- Upload only dirty persistent instance ranges.
|
||
- Reuse MDI command/sort buffers.
|
||
- Build instance-set comparison and fixed-camera screenshot comparison.
|
||
- Run old and new submissions in compare mode without drawing twice.
|
||
- At the gate, delete the replaced production enumeration/submission path.
|
||
|
||
**Gate:** instance sets and screenshots match; dense-town uncapped target is
|
||
300 FPS or the corrected profile identifies a new dominant stage; steady frame
|
||
allocation is near zero.
|
||
|
||
### Slice H — Event-driven UI, diagnostics, lights, and frame cleanup
|
||
|
||
**Purpose:** Remove remaining work that scales with uncapped FPS.
|
||
|
||
- Skip diagnostics with no consumer and reuse renderer visibility facts.
|
||
- Dirty-layout retained UI: apply anchors only after geometry changes.
|
||
- Maintain an overlay-participant registry instead of a second full-tree walk.
|
||
- Track active cooldown items/effects/dialogs through events.
|
||
- Replace per-frame frame-input objects, LINQ arrays, iterator objects, and
|
||
liveness collections with reusable storage or borrowed views.
|
||
- Make light candidates spatial/cell-driven and select top-k without a complete
|
||
per-frame sort.
|
||
- Replace socket-timeout exceptions and datagram `ToArray` copies with normal
|
||
cancellable/poll/span-based I/O.
|
||
|
||
**Gate:** no steady-state Gen-2 collections; steady allocation target
|
||
≤ 4 KiB/frame initially and zero for the core world/render loop; UI and network
|
||
behavior tests unchanged.
|
||
|
||
### Slice I — Flat collision assets and residual zero-allocation work
|
||
|
||
**Purpose:** Complete MP4 without changing retail math.
|
||
|
||
- Bake flattened index-based BSP/collision records.
|
||
- Port traversal data access line-for-line while preserving ordering and
|
||
arithmetic.
|
||
- Remove parsed DBObj graphs from streaming collision publication.
|
||
- Address remaining measured allocation sites, including any physics transition
|
||
pooling only after identity/lifetime tests prove it safe.
|
||
|
||
**Gate:** trajectory and retail conformance suites remain bit-equivalent;
|
||
navigation/collision fixtures match; portal and steady allocation targets pass.
|
||
|
||
### Slice J — Presentation-independent `AcDream.Runtime`
|
||
|
||
**Purpose:** Establish the graphical/headless shared client kernel.
|
||
|
||
- Add the Runtime project and dependency guards.
|
||
- Define `IGameRuntimeView`, commands, ordered events, clock, and lifecycle.
|
||
- Adapt the current graphical host to those interfaces first.
|
||
- Remove App presentation dependencies from canonical gameplay owners.
|
||
- Move owners by coherent lifetime groups; never mirror state.
|
||
- Add host-parity and no-backend construction tests after each move.
|
||
- End with `GameWindow`/App composing one `GameRuntime`.
|
||
|
||
**Gate:** the connected graphical route is unchanged; a no-window integration
|
||
test connects, enters world, moves, receives inventory/chat/world updates,
|
||
portals, logs out, reconnects, and tears down without loading App/Silk/OpenAL.
|
||
|
||
### Slice K — Linux headless and multi-session host
|
||
|
||
**Purpose:** Deliver efficient automated bots.
|
||
|
||
- Add Linux CI for Core, Core.Net, Content, Runtime, and Headless.
|
||
- Implement portable path/config/credential handling.
|
||
- Add `AcDream.Headless` CLI and deterministic scheduler.
|
||
- Add bot command/event APIs for movement, selection, use, combat, spells,
|
||
looting, chat, and commands.
|
||
- Share immutable content/collision stores across sessions.
|
||
- Audit and eliminate mutable statics and process-wide session state.
|
||
- Add clean cancellation, reconnect, plugin isolation, and credential-safe logs.
|
||
- Stress 1, 5, 10, and 30 sessions.
|
||
|
||
**Gate:** 30 local-server sessions in one Linux process, no GPU/display/audio
|
||
dependency, bounded incremental memory per session, no busy-loop CPU, clean
|
||
teardown/reconnect, and parity with graphical runtime command/event behavior.
|
||
|
||
### Slice L — Linux graphical host and evidence-gated GPU work
|
||
|
||
**Purpose:** Finish platform portability and only then pursue remaining GPU
|
||
opportunities.
|
||
|
||
- Validate Linux OpenGL extension/driver matrix and package native dependencies.
|
||
- Port paths, frame pacing, input, audio, and packaging without renderer
|
||
fallback.
|
||
- Reprofile locally with accurate pass timing.
|
||
- Add GPU culling, command compaction, particle simulation, or light selection
|
||
only for stages proven dominant.
|
||
|
||
**Gate:** graphical Linux connected route passes the same lifecycle, screenshot,
|
||
resource, and performance checks; every GPU migration has a CPU/GPU before/after
|
||
and visual equivalence result.
|
||
|
||
---
|
||
|
||
## 11. Performance gates
|
||
|
||
Reference measurements use a Release build, the same account/route, stable
|
||
camera scripts, and both capped and uncapped modes. RDP results are kept
|
||
separate from local-display results.
|
||
|
||
| Metric | Target |
|
||
|---|---:|
|
||
| Traversal frame p99 | ≤ 16.67 ms on reference local hardware |
|
||
| Traversal maximum | ≤ 33.3 ms after warm process startup |
|
||
| Dense-town uncapped frame p50 | ≤ 3.33 ms or a newly attributed blocker |
|
||
| Steady update p95 | ≤ 2 ms |
|
||
| Core world/render allocation | 0 B/frame after warmup |
|
||
| Whole-client steady allocation | initial gate ≤ 4 KiB/frame |
|
||
| Portal single-frame allocation | ≤ 4 MiB, then tighten from evidence |
|
||
| Gen-2 collections during canonical route | 0 after startup/bake warmup |
|
||
| Same-location third-visit resource growth | ≤ 5% and explained |
|
||
| World-visible-before-ready events | 0 |
|
||
| Staged resources at stable checkpoints | 0 |
|
||
| Exception-as-control-flow | 0 known sites |
|
||
| Headless renderer/audio/window allocations | 0 |
|
||
|
||
Memory budgets are measured by category rather than only process working set:
|
||
|
||
- Live managed heap.
|
||
- GC committed/fragmented.
|
||
- Native prepared/staging buffers.
|
||
- Mapped content pages.
|
||
- Tracked GPU buffers/textures.
|
||
- Driver-reported dedicated/shared GPU memory.
|
||
|
||
Absolute process/VRAM targets are fixed after Slice D produces trustworthy
|
||
category accounting; until then, the binding rule is plateau plus no regression
|
||
in pixels/range.
|
||
|
||
---
|
||
|
||
## 12. Test and review matrix
|
||
|
||
Every slice must pass the tests relevant to its boundary:
|
||
|
||
1. **Pure unit tests:** handles, generations, budgets, queues, manifest,
|
||
deduplication, stale/corrupt data, snapshot buffers.
|
||
2. **DAT equivalence:** live extractor versus prepared asset for representative
|
||
outdoor, dungeon, town, portal, animated, translucent, and particle assets.
|
||
3. **Scene equivalence:** canonical entity/instance digest at fixed checkpoints.
|
||
4. **Render equivalence:** instance-set diff plus fixed-camera screenshots.
|
||
5. **Retail conformance:** named-retail citations for any AC-specific selection,
|
||
degradation, visibility, collision, or timing behavior touched.
|
||
6. **Lifecycle:** fresh login, same-location revisit, world edge, dungeon,
|
||
repeated recalls, rapid generation replacement, logout/reconnect, graceful
|
||
close, cancellation, and failure injection.
|
||
7. **Resource:** exact create/retire counts, heap diff, GPU accounting, no stale
|
||
owner or queued work at stable checkpoints.
|
||
8. **Performance:** capped and uncapped CPU/GPU/allocation/GC percentiles.
|
||
9. **Headless parity:** same runtime packet/order/command results with
|
||
presentation attached or absent.
|
||
10. **Platform:** Windows graphical/headless plus Linux headless; Linux graphical
|
||
begins in Slice L.
|
||
|
||
For a replacement cutover:
|
||
|
||
- Shadow/compare mode may exist only during development.
|
||
- Confirmed differences are fixed at their source.
|
||
- The gate commit removes the old production path.
|
||
- No test threshold is loosened to accept a regression.
|
||
|
||
---
|
||
|
||
## 13. Risks and controls
|
||
|
||
| Risk | Control |
|
||
|---|---|
|
||
| Pak repeats the 865 GB failure | content identity and aliasing land before another full bake; bake reports unique/shared ratios continuously |
|
||
| Prepared data changes pixels/geometry | byte/field equivalence against the current extractor; no compression in the equivalence phase |
|
||
| Streaming budget reveals an incomplete world | one destination-generation readiness barrier; reveal is an atomic state edge |
|
||
| Deferred retirement leaks or revives stale resources | incarnation/generation owner tokens and stable-checkpoint zero-outstanding assertions |
|
||
| Render ECS becomes a second game world | render-only component contract; no network properties or authoritative GUID map; digest checks against canonical owner |
|
||
| Snapshot adds a frame of latency | same-thread borrowed snapshot first; frame-number assertions; concurrency remains deferred |
|
||
| Arch dependency constrains the design | acdream-owned `IRenderScene`; Arch only in App; no Arch types cross the boundary |
|
||
| Headless extraction duplicates state | move the existing canonical owner in lifetime groups; adapters first, never synchronize two stores |
|
||
| Shared 30-client cache gains mutable state | immutable content only; tests create conflicting sessions/characters concurrently |
|
||
| Linux work forces renderer fallback | graphical host retains mandatory modern extensions and fails clearly on unsupported drivers |
|
||
| “Optimization” changes retail behavior | unchanged behavior is a hard gate; retail ports remain OO and traceable; visual/connected gates stay mandatory |
|
||
|
||
---
|
||
|
||
## 14. Recommended starting point
|
||
|
||
Begin with **Slice A**, then **Slice B**.
|
||
|
||
Slice A is small and makes all later GPU and frame gates honest. Slice B completes
|
||
an implementation already present in the tree and directly attacks the
|
||
benchmark's largest allocation source. The first meaningful runtime cutover is
|
||
Slice C; do not begin the render-scene migration before the portal content storm
|
||
has been removed and measured.
|
||
|
||
The intended order is therefore:
|
||
|
||
```text
|
||
honest metrics
|
||
-> prepared content
|
||
-> typed residence
|
||
-> cost-budgeted streaming
|
||
-> incremental render scene
|
||
-> delta GPU submission
|
||
-> residual frame cleanup
|
||
-> presentation-independent runtime
|
||
-> Linux/headless/multi-session
|
||
-> evidence-gated GPU jobs
|
||
```
|
||
|
||
This order improves the current graphical client first while laying clean
|
||
boundaries for Linux and approximately 30 automated headless clients later.
|