docs(architecture): consolidate modern runtime roadmap
Unify the unfinished Modern Pipeline work with Linux/headless goals around measured content, residency, streaming, render-scene, and runtime boundaries. Record twelve gated slices and preserve retail-shaped simulation plus the mandatory N.5 renderer. Co-authored-by: Codex <codex@openai.com>
This commit is contained in:
parent
2c3da8e153
commit
944ee55584
2 changed files with 808 additions and 0 deletions
|
|
@ -1568,6 +1568,17 @@ port in any phase — no separate listing here.
|
|||
> or throughput redesign. It is corrective ownership/reclamation work for a
|
||||
> reproduced crash-class regression in the mandatory renderer, using the current
|
||||
> architecture and preserving pixels/ranges.
|
||||
>
|
||||
> **2026-07-24 planning reconciliation:** a new connected nine-stop CPU/GPU/
|
||||
> memory audit reproduced a larger portal allocation/GC storm (approximately
|
||||
> 204 MiB in one frame, 276 ms maximum frame, 38 Gen-2 collections) while
|
||||
> confirming stable entity/effect/GPU plateaus and low steady GPU utilization.
|
||||
> The user requested a detailed modern architecture plan. The resulting
|
||||
> [`2026-07-24-modern-runtime-architecture.md`](2026-07-24-modern-runtime-architecture.md)
|
||||
> unifies the unfinished MP pak/render-world work with future Track LH, preserves
|
||||
> retail-shaped simulation and the N.5 renderer, and reorders execution around
|
||||
> prepared content and cost-budgeted streaming before render-ECS throughput.
|
||||
> This records the plan only; implementation remains not started.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-07-05-modern-pipeline-design.md` (the
|
||||
umbrella design — read it first). **Goal:** smoothness first (no frame over
|
||||
|
|
|
|||
797
docs/plans/2026-07-24-modern-runtime-architecture.md
Normal file
797
docs/plans/2026-07-24-modern-runtime-architecture.md
Normal file
|
|
@ -0,0 +1,797 @@
|
|||
# Modern Runtime Architecture — detailed execution plan
|
||||
|
||||
**Date:** 2026-07-24
|
||||
|
||||
**Status:** Planning complete; implementation not started
|
||||
|
||||
**Scope:** Reconcile and sequence the existing Modern Pipeline (`MP`) and
|
||||
Linux/headless (`LH`) tracks using the 2026-07-24 connected performance audit.
|
||||
|
||||
**Primary objective:** Minimize CPU, managed/native memory, GPU memory,
|
||||
allocation, and frame-time variance without reducing view distance, particles,
|
||||
world detail, or retail behavior.
|
||||
|
||||
This plan refines rather than replaces
|
||||
[`docs/superpowers/specs/2026-07-05-modern-pipeline-design.md`](../superpowers/specs/2026-07-05-modern-pipeline-design.md).
|
||||
It preserves the shipped N.5 bindless/MDI renderer, the completed `GameWindow`
|
||||
decomposition, canonical `LiveEntityRuntime` ownership, the retail PView/cell
|
||||
visibility ports, and all behavior already accepted by the user.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why the architecture should change
|
||||
|
||||
The 2026-07-24 six-minute, nine-stop connected route established:
|
||||
|
||||
| Observation | Measurement | Architectural implication |
|
||||
|---|---:|---|
|
||||
| Steady capped update work | approximately 0.4–1.3 ms median | The gameplay/update architecture is not the steady-state crisis |
|
||||
| External GPU use | 7.6% median, 15.9% p95 | The tested route was not GPU-bound |
|
||||
| Portal allocation | up to approximately 204 MiB in one frame | Runtime content preparation and publication must leave the frame transaction |
|
||||
| Managed allocation rate | approximately 40.8 MB/s average, 2.22 GiB/s maximum | Portal work is creating short-lived graphs and pixel/mesh arrays faster than GC can absorb |
|
||||
| GC | 38 Gen-2 collections; 256 ms longest pause | Frame spikes are dominated by allocation/publication, not upload time alone |
|
||||
| Longest frame | 276 ms | Landblock-count budgets are not real cost budgets |
|
||||
| Final live GC heap | approximately 194 MiB | The 1–2 GiB process high-water is not an unbounded live-object leak |
|
||||
| Dedicated GPU memory | 633–818 MiB | Texture arrays/caches are bounded but expensive and insufficiently observable |
|
||||
| Revisited locations | entity/emitter/particle counts returned to the same plateaus | Logical lifetime is broadly convergent; optimize rather than replace it |
|
||||
| Exception traffic | 4,520 invalid DAT probes plus 1,168 receive timeouts | Exceptions are being used as normal control flow |
|
||||
|
||||
Allocation attribution names the runtime mesh/texture path
|
||||
(`MeshExtractor.PrepareMeshData` → `PrepareGfxObjMeshData` →
|
||||
`AddSurfaceToBatch`) as the largest portal source. The update-thread
|
||||
publication/retirement graph then amplifies it with complete dictionaries,
|
||||
sorts, effect profiles, and unmetered per-entity teardown.
|
||||
|
||||
The conclusion is:
|
||||
|
||||
> Keep retail-shaped gameplay and the modern GPU backend. Replace the runtime
|
||||
> content, scene-projection, residency, and work-scheduling architecture that
|
||||
> feeds the renderer.
|
||||
|
||||
---
|
||||
|
||||
## 2. Fixed architectural decisions
|
||||
|
||||
These decisions are binding unless a later measured gate disproves one.
|
||||
|
||||
1. **No whole-client rewrite.** Existing retail ports, network behavior,
|
||||
physics, animation, UI, plugins, and the N.5 renderer remain in place.
|
||||
2. **No simulation ECS.** Retail-shaped object-oriented code remains the oracle-
|
||||
traceable form for physics, motion, sequencers, combat, inventory, and magic.
|
||||
3. **A data-oriented render world only.** Arch remains the selected storage
|
||||
implementation, hidden behind acdream-owned interfaces so it can be replaced
|
||||
without changing simulation or rendering contracts.
|
||||
4. **One authoritative live-object owner.** `LiveEntityRuntime` is evolved and
|
||||
later extracted; no second GUID map, gameplay world, or mirrored entity state
|
||||
may be introduced during migration.
|
||||
5. **Headless is not a hidden window.** A headless session constructs no
|
||||
Silk.NET window, GL context, render assets, particles, audio, or gameplay UI.
|
||||
6. **One content interpretation.** Bake and runtime validation use the exact
|
||||
existing `AcDream.Content` interpretation code. A baked artifact must be
|
||||
byte/field equivalent before runtime uses it.
|
||||
7. **No visual-quality optimization by default.** View horizon, retail PView,
|
||||
particles, effects, and texture pixels stay unchanged. Retail distance LOD is
|
||||
a faithful mechanism and remains separately configurable.
|
||||
8. **Work is budgeted by cost, not object count.** Streaming stages use elapsed
|
||||
microseconds, prepared bytes, uploaded bytes, and entity operations. “Four
|
||||
landblocks” is not a meaningful budget.
|
||||
9. **GL remains render-thread owned.** Workers may perform I/O, parsing,
|
||||
decompression, mesh preparation, and collision preparation, but never mutate
|
||||
GL or live scene state.
|
||||
10. **Frame products are borrowed immutable views, not object graphs.** Snapshot
|
||||
boundaries use double-buffered arrays/spans and generation stamps; they do
|
||||
not allocate a deep immutable tree each frame.
|
||||
11. **Parallelism follows data cleanup.** Do not add a job system to compensate
|
||||
for repeated decode, sorting, allocation, or whole-world scans.
|
||||
12. **Every cutover deletes the replaced production path.** Temporary shadow
|
||||
execution is allowed for comparison, but no permanent dual renderer,
|
||||
duplicate world, or silent decode fallback remains after its gate.
|
||||
|
||||
---
|
||||
|
||||
## 3. Target dependency architecture
|
||||
|
||||
```text
|
||||
AcDream.GraphicalHost (current AcDream.App)
|
||||
├── window/input/OpenGL/OpenAL/retail UI
|
||||
├── render scene projection + renderer
|
||||
└── owns one AcDream.Runtime.GameRuntime
|
||||
|
||||
AcDream.Headless
|
||||
├── deterministic host clock
|
||||
├── bot/plugin command surface
|
||||
└── owns one or more AcDream.Runtime.GameRuntime instances
|
||||
|
||||
AcDream.Runtime [new, extracted incrementally]
|
||||
├── session lifecycle and inbound ordering
|
||||
├── authoritative live world and properties
|
||||
├── movement/physics/interaction/combat/magic/inventory/chat
|
||||
├── instance-scoped clocks, queues, commands, and events
|
||||
└── no Silk.NET, GL, OpenAL, retained UI, or OS window dependency
|
||||
|
||||
AcDream.Content [already exists]
|
||||
├── DAT interpretation
|
||||
├── deterministic bake pipeline
|
||||
├── mapped pak/content manifest
|
||||
├── immutable prepared render/collision metadata
|
||||
└── shared read-only content store for multiple sessions
|
||||
|
||||
AcDream.Core / AcDream.Core.Net [existing retail-shaped logic]
|
||||
AcDream.Plugin.Abstractions [existing BCL-only contracts]
|
||||
```
|
||||
|
||||
`AcDream.Runtime` is a target boundary, not the first implementation step. App
|
||||
types move only after their presentation dependencies have been removed and
|
||||
parity tests prove that the graphical host is still driving the same owner.
|
||||
|
||||
### Dependency rules
|
||||
|
||||
- `Core` never depends on Runtime, App, Content, or a backend.
|
||||
- `Core.Net` depends on Core only.
|
||||
- `Content` depends on Core, never App or a graphics backend.
|
||||
- `Runtime` may depend on Core, Core.Net, Content, and Plugin.Abstractions.
|
||||
- The graphical and headless hosts depend on Runtime; Runtime never depends on
|
||||
either host.
|
||||
- UI and rendering consume read-only runtime views plus explicit commands/events.
|
||||
- Shared multi-session caches contain immutable content only. Credentials,
|
||||
GUIDs, clocks, packets, plugins, object state, and automation remain
|
||||
instance-scoped.
|
||||
|
||||
---
|
||||
|
||||
## 4. Runtime ownership model
|
||||
|
||||
| Concern | Canonical owner | Consumers |
|
||||
|---|---|---|
|
||||
| Server GUID/incarnation/timestamps/properties | `GameRuntime` evolved from `LiveEntityRuntime` | physics, UI, render projection, plugins, bots |
|
||||
| Static prepared content | `ContentStore` | streaming, collision, render projection |
|
||||
| Desired spatial window | `StreamingRegion` | streaming scheduler |
|
||||
| Landblock generation and stage receipts | `StreamingWorkScheduler` | reveal, physics publication, render publication |
|
||||
| Render entity/component data | `IRenderScene` / Arch implementation | visibility, snapshot builder |
|
||||
| CPU/GPU asset residence | `ResidencyManager` | render scene, upload owner |
|
||||
| GL names, fences, physical retirement | renderer resource owners | renderer only |
|
||||
| Retail PView/visible-cell result | existing PView owner | render-scene query/snapshot builder |
|
||||
| UI state | focused UI controllers/ViewModels | retained UI and optional dev UI |
|
||||
|
||||
Identity types must remain explicit:
|
||||
|
||||
```text
|
||||
ServerGuid authoritative network identity
|
||||
ObjectIncarnation ServerGuid + generation
|
||||
RuntimeEntityId runtime-local logical identity
|
||||
RenderEntityId presentation projection identity
|
||||
AssetHandle<T> immutable content identity + generation
|
||||
GpuHandle<T> render-thread physical resource identity
|
||||
```
|
||||
|
||||
No layer may infer one identity from another through unchecked integer casts.
|
||||
|
||||
---
|
||||
|
||||
## 5. Content and asset architecture
|
||||
|
||||
### 5.1 Prepared assets
|
||||
|
||||
The runtime world hot path should consume prepared records, not DBObj graphs:
|
||||
|
||||
- Packed mesh vertices, indices, batch/material records, bounds, and part tables.
|
||||
- Exact decoded texture pixels used by the current renderer.
|
||||
- EnvCell geometry and portal records with content-deduplicated aliases.
|
||||
- Terrain and deterministic scenery placement.
|
||||
- Flattened collision/BSP arrays.
|
||||
- Compact Setup presentation metadata:
|
||||
default script, script table, animation, part availability, sound table,
|
||||
collision references, and effect-relevant flags.
|
||||
- Retail degrade tables.
|
||||
|
||||
Dynamic character palette/appearance composition, low-volume UI assets, audio,
|
||||
and motion/animation tables may remain runtime DAT consumers until separately
|
||||
measured and migrated.
|
||||
|
||||
### 5.2 Content-addressed pak manifest
|
||||
|
||||
Finish the existing pak implementation rather than replacing it:
|
||||
|
||||
```text
|
||||
Source asset key -> manifest entry -> content hash/shared blob
|
||||
```
|
||||
|
||||
- EnvCell file IDs may alias the same blob offset.
|
||||
- Blob contents are deterministic and 64-byte aligned.
|
||||
- DAT iteration stamps, serializer version, and bake-tool version invalidate
|
||||
stale content.
|
||||
- The writer builds a temporary artifact, validates it, then atomically replaces
|
||||
the prior artifact.
|
||||
- The reader memory-maps immutable data and exposes typed borrowed views.
|
||||
- Corrupt/stale entries fail loudly and name the required rebake.
|
||||
- Production streaming does not silently fall back to runtime decode after the
|
||||
cutover. Developer equivalence tools retain both sources explicitly.
|
||||
|
||||
### 5.3 Asset handles and residence
|
||||
|
||||
Callers receive typed handles rather than retaining decoded arrays:
|
||||
|
||||
```csharp
|
||||
readonly record struct AssetHandle<T>(uint Index, ushort Generation);
|
||||
readonly record struct AssetLease<T>(AssetHandle<T> Handle, OwnerToken Owner);
|
||||
```
|
||||
|
||||
The eventual `ResidencyManager` tracks:
|
||||
|
||||
- CPU prepared bytes.
|
||||
- Decoded/pinned/staging bytes.
|
||||
- GPU buffer and texture bytes.
|
||||
- Current owners and reference count.
|
||||
- Last used generation/frame.
|
||||
- Rebuild/reload cost.
|
||||
- Priority: destination-critical, visible, near, far, speculative.
|
||||
- State: absent, requested, prepared, upload-pending, resident, retiring.
|
||||
|
||||
Eviction is generation-safe and owner-scoped. A stale completion cannot revive a
|
||||
retired world generation or release the replacement generation's resource.
|
||||
|
||||
---
|
||||
|
||||
## 6. Streaming and reveal architecture
|
||||
|
||||
### 6.1 Staged pipeline
|
||||
|
||||
```text
|
||||
Request
|
||||
-> I/O/map lookup
|
||||
-> parse/borrow prepared records
|
||||
-> build landblock publication
|
||||
-> physics/collision publish
|
||||
-> render asset request
|
||||
-> GL upload
|
||||
-> render-scene publish
|
||||
-> reveal-ready
|
||||
```
|
||||
|
||||
Every item carries:
|
||||
|
||||
- World/session generation.
|
||||
- Landblock/cell identity.
|
||||
- Stage and priority.
|
||||
- Estimated and actual byte cost.
|
||||
- Entity-operation count.
|
||||
- Cancellation token.
|
||||
- Retryable stage receipt.
|
||||
|
||||
Bounded queues provide back-pressure. Background stages may run concurrently;
|
||||
authoritative world, render-scene publication, and GL upload remain ordered on
|
||||
their owning thread.
|
||||
|
||||
### 6.2 Cost budget
|
||||
|
||||
Each frame receives independently configurable budgets:
|
||||
|
||||
- Update-thread publication time.
|
||||
- Entity create/retire operations.
|
||||
- CPU bytes adopted.
|
||||
- GPU bytes uploaded.
|
||||
- GL resource retire operations.
|
||||
|
||||
Immediate work:
|
||||
|
||||
- Remove old landblocks from visibility, collision, picking, radar, and target
|
||||
eligibility.
|
||||
- Mark the old generation unavailable to new consumers.
|
||||
|
||||
Budgeted work:
|
||||
|
||||
- Walk old presentation owners.
|
||||
- Release scripts/effects/lights/plugin snapshots.
|
||||
- Retire CPU/GPU resources after fences permit.
|
||||
- Publish far-ring content.
|
||||
|
||||
This preserves correct visible lifetime while preventing a 600-landblock
|
||||
retirement from becoming one frame transaction.
|
||||
|
||||
### 6.3 Portal generation and reveal
|
||||
|
||||
A destination generation progresses through:
|
||||
|
||||
```text
|
||||
Requested -> Prepared -> CollisionReady -> NearSceneReady -> Revealed
|
||||
|
|
||||
+-> FarSceneConverging
|
||||
```
|
||||
|
||||
Reveal is an atomic edge. The world cannot become visible until the destination
|
||||
near ring, collision root, camera identity, and required scene publication all
|
||||
belong to the same generation. Far content continues under normal budgets after
|
||||
reveal.
|
||||
|
||||
This replaces “priority work bypasses the budget” with a prepared-behind-portal
|
||||
contract and eliminates `viewport-before-ready`.
|
||||
|
||||
---
|
||||
|
||||
## 7. Data-oriented render scene
|
||||
|
||||
### 7.1 Scope
|
||||
|
||||
Arch stores render projections only:
|
||||
|
||||
- Static landblock objects and scenery.
|
||||
- EnvCell objects.
|
||||
- Live entities and equipped children.
|
||||
- Lights and effect anchors where a packed projection is beneficial.
|
||||
|
||||
It never owns gameplay properties, network sequencing, physics authority,
|
||||
inventory, combat, or interaction state.
|
||||
|
||||
### 7.2 Components
|
||||
|
||||
Initial components are blittable or stable handles:
|
||||
|
||||
```text
|
||||
RenderTransform
|
||||
PreviousRenderTransform
|
||||
MeshAsset
|
||||
MaterialVariant
|
||||
SpatialResidency (landblock, cell)
|
||||
WorldBounds
|
||||
RenderFlags
|
||||
DegradeState
|
||||
SortKey
|
||||
OwnerIncarnation
|
||||
DirtyMask
|
||||
```
|
||||
|
||||
Separate archetypes cover static, dynamic, equipped-child, translucent, and
|
||||
light-bearing projections. Do not force every entity to carry every component.
|
||||
|
||||
### 7.3 Incremental indices
|
||||
|
||||
Creation, rebucketing, mutation, and removal maintain:
|
||||
|
||||
- Outdoor-static set.
|
||||
- Per-cell static sets.
|
||||
- Dynamic set.
|
||||
- Translucent set.
|
||||
- Selectable/pickable spatial index.
|
||||
- Light candidates.
|
||||
- Dirty transform/material/mesh ranges.
|
||||
|
||||
The frame loop no longer partitions every loaded entity. Existing retail PView
|
||||
continues to determine visible cells; the render scene enumerates only those
|
||||
cell buckets plus the dynamic set.
|
||||
|
||||
### 7.4 Simulation-to-render seam
|
||||
|
||||
`LiveEntityRuntime` and static publication emit ordered projection deltas:
|
||||
|
||||
```text
|
||||
Register
|
||||
UpdateTransform
|
||||
UpdateAppearance
|
||||
UpdateFlags
|
||||
Rebucket
|
||||
Unregister
|
||||
```
|
||||
|
||||
Deltas contain exact incarnation identity. The render world rejects stale
|
||||
generation updates. It never calls back into simulation dictionaries while
|
||||
drawing.
|
||||
|
||||
---
|
||||
|
||||
## 8. Render-frame product and GPU submission
|
||||
|
||||
The snapshot is a double-buffered, generation-stamped borrowed view:
|
||||
|
||||
```text
|
||||
RenderFrameView
|
||||
Visible opaque instance ranges
|
||||
Visible alpha instance ranges
|
||||
Dynamic transforms
|
||||
Light set
|
||||
Effect draw records
|
||||
Selection records
|
||||
Existing PView/clip products
|
||||
```
|
||||
|
||||
The update/render order initially remains on the accepted host thread. The
|
||||
snapshot seam prevents mutation during draw and permits later thread separation
|
||||
without requiring it now.
|
||||
|
||||
GPU submission evolves in measured steps:
|
||||
|
||||
1. Persistent global mesh/instance buffers remain.
|
||||
2. Dirty ranges update only changed instance records.
|
||||
3. MDI command buffers are reused and rewritten in place.
|
||||
4. Static command templates are cached by scene generation/cell visibility.
|
||||
5. Accurate timestamp queries bracket actual render passes.
|
||||
6. GPU culling, command compaction, GPU particles, or GPU light selection are
|
||||
introduced only when the corrected profile names them as the next bottleneck.
|
||||
|
||||
The existing portal/PView pass graph, clipping, translucency ordering, and
|
||||
bindless material behavior remain authoritative.
|
||||
|
||||
---
|
||||
|
||||
## 9. Presentation-independent runtime and headless host
|
||||
|
||||
### 9.1 Extraction rule
|
||||
|
||||
Do not create a new parallel `GameRuntime` and synchronize it with
|
||||
`LiveEntityRuntime`. Instead:
|
||||
|
||||
1. Define narrow runtime read/command/event contracts around the current owner.
|
||||
2. Remove presentation-specific fields from that owner into App projections.
|
||||
3. Move the now presentation-independent owner and collaborators into
|
||||
`AcDream.Runtime`.
|
||||
4. Keep the graphical host using the same instance throughout the move.
|
||||
|
||||
### 9.2 `GameRuntime`
|
||||
|
||||
One instance owns:
|
||||
|
||||
- Connection/authentication/character/session lifetime.
|
||||
- Packet receive ordering and retail update phases.
|
||||
- Authoritative objects, properties, containers, inventory, spell state,
|
||||
enchantments, vitals, targets, and combat state.
|
||||
- Movement, collision state required for gameplay, and interaction commands.
|
||||
- Instance clock, random sources where applicable, queues, plugins/behaviors,
|
||||
diagnostics identity, and teardown.
|
||||
|
||||
It exposes:
|
||||
|
||||
- Immutable or borrowed read views.
|
||||
- Typed commands.
|
||||
- Ordered events/deltas.
|
||||
- Deterministic `Tick`.
|
||||
- Retryable, complete shutdown.
|
||||
|
||||
### 9.3 Hosts
|
||||
|
||||
`GraphicalGameHost` provides input, camera, rendering, UI, audio, frame pacing,
|
||||
and a single runtime instance.
|
||||
|
||||
`HeadlessGameHost` provides a monotonic scheduler, navigation collision/content,
|
||||
bot actions, diagnostics, and one or more runtime instances. It performs no
|
||||
render-content bake lookup unless navigation/collision requires that content.
|
||||
|
||||
For 30 clients:
|
||||
|
||||
- Immutable content and flattened collision assets are shared.
|
||||
- Every mutable session structure is instance-scoped.
|
||||
- One process may host many sessions, but one-process-per-session remains a
|
||||
supported diagnostic/isolation mode.
|
||||
- Scheduling uses a deterministic round-robin/time-wheel rather than 30 busy
|
||||
loops.
|
||||
- Server-safe outbound rate limits are per session.
|
||||
|
||||
---
|
||||
|
||||
## 10. Detailed implementation sequence
|
||||
|
||||
Each slice is independently buildable, testable, bisectable, and behavior-
|
||||
preserving. Each gets a focused implementation plan when it starts.
|
||||
|
||||
### Slice A — Measurement and contract correction
|
||||
|
||||
**Purpose:** Make every later gate trustworthy.
|
||||
|
||||
- Correct `GpuFrameTimer` so queries cover actual GL submission/pass intervals,
|
||||
not frame pacing.
|
||||
- Extend canonical checkpoint JSON with CPU/GPU resident bytes, staging bytes,
|
||||
per-stage queue depth, stage work time, exception counts, GC pause/heap
|
||||
fields, and render-scene generation.
|
||||
- Add a tracked analysis script that summarizes the existing nine-stop route.
|
||||
- Record capped-RDP and uncapped-local results separately.
|
||||
- Add a deterministic allocation/exception attribution recipe without enabling
|
||||
developer UI or changing gameplay.
|
||||
|
||||
**Gate:** repeated idle measurements have low observer effect; CPU stage sums
|
||||
reconcile with active frame time; external GPU engine direction agrees with GL
|
||||
timestamps.
|
||||
|
||||
### Slice B — Finish MP1b EnvCell dedup and full bake
|
||||
|
||||
**Purpose:** Make the existing pak physically usable.
|
||||
|
||||
- Add shared-blob/alias support to `PakWriter` while retaining unique source
|
||||
keys.
|
||||
- Compute the existing runtime EnvCell geometry identity before extraction.
|
||||
- Extract each unique geometry once; map all source file IDs to the shared blob.
|
||||
- Make side-staged particle preloads and ordinary assets use the same
|
||||
deterministic content table.
|
||||
- Add alias, collision, determinism, stale-version, corruption, cancellation,
|
||||
and atomic-replace tests.
|
||||
- Run the complete bake and publish counts, unique ratios, size, time, failures,
|
||||
and peak memory.
|
||||
|
||||
**Gate:** full bake finishes in practical time/space; all fixture and random
|
||||
sample equivalence tests pass; duplicate EnvCells share blob offsets; no runtime
|
||||
code consumes the pak yet.
|
||||
|
||||
### Slice C — Runtime prepared-asset source and MP1c cutover
|
||||
|
||||
**Purpose:** Remove mesh/texture decode from portal frames.
|
||||
|
||||
- Introduce `IPreparedAssetSource` in Content and inject it into App streaming.
|
||||
- Teach `ObjectMeshManager`/landblock builders to adopt pak-backed prepared
|
||||
payloads without reconstructing complete intermediate object graphs.
|
||||
- Add compact Setup presentation metadata to the bake or a keyed prepared cache
|
||||
so static activation does not parse arbitrary IDs as Setup.
|
||||
- Replace `ResolveId().ToList().OrderBy...` with typed non-allocating lookup.
|
||||
- Add explicit negative/type metadata; exceptions are not type tests.
|
||||
- Retain live extraction only in bake/equivalence tooling and required dynamic
|
||||
appearance paths.
|
||||
- Remove production streaming fallback at the gate.
|
||||
|
||||
**Gate:** byte/field equivalence; no invalid Setup exception storm; portal
|
||||
single-frame allocation and p99 materially improve from the 2026-07-24 baseline;
|
||||
connected nine-stop route and screenshots pass.
|
||||
|
||||
### Slice D — Typed asset handles and unified residency
|
||||
|
||||
**Purpose:** Bound and explain CPU/GPU memory.
|
||||
|
||||
- Add `AssetHandle<T>`, owner tokens, leases, generations, and accounting.
|
||||
- Place object mesh, standalone texture, composite texture-array, staging, and
|
||||
prepared-content residence behind one policy owner while retaining specialized
|
||||
physical caches.
|
||||
- Track logical, CPU, staging, GPU-requested, GPU-resident, and retiring bytes
|
||||
separately.
|
||||
- Add configurable budgets through `RuntimeOptions`/settings with current visual
|
||||
behavior as the default.
|
||||
- Implement generation-safe LRU/cost-aware eviction and fence-delayed physical
|
||||
release.
|
||||
- Expose exact budget/occupancy/fragmentation facts to diagnostics.
|
||||
|
||||
**Gate:** same-location third-visit residence plateaus; no stale-generation
|
||||
release; no missing textures; no resource growth after repeated portal loops.
|
||||
|
||||
### Slice E — Cost-budgeted streaming and retirement
|
||||
|
||||
**Purpose:** Remove update-thread portal transactions.
|
||||
|
||||
- Introduce explicit stage queues and `StreamingWorkBudget`.
|
||||
- Split immediate logical/spatial detach from budgeted owner/resource teardown.
|
||||
- Make per-entity retirement cursor/time bounded.
|
||||
- Make publication cursor/time/byte bounded.
|
||||
- Reserve destination-critical work across portal frames rather than bypassing
|
||||
all budgets on one frame.
|
||||
- Preserve FIFO within priority/generation and exact retry receipts.
|
||||
- Connect reveal to the single destination-generation readiness state.
|
||||
|
||||
**Gate:** zero `viewport-before-ready`; traversal p99 at or below 16.67 ms target
|
||||
and maximum below 33.3 ms target on the reference local run; no stranded old
|
||||
generation, staged upload, collision, effect, or GPU owner.
|
||||
|
||||
### Slice F — Incremental render scene foundation
|
||||
|
||||
**Purpose:** Stop rebuilding/partitioning the world each frame.
|
||||
|
||||
- Add acdream-owned `IRenderScene`, identifiers, components, and delta journal.
|
||||
- Implement it with Arch in App only.
|
||||
- Mirror static publication and live projection into a non-drawing shadow world.
|
||||
- Add deterministic scene digest and compare it with current owners at
|
||||
checkpoints.
|
||||
- Maintain cell/outdoor/dynamic/translucent/light indices incrementally.
|
||||
- Add stale-incarnation, duplicate-create, rebucket, delete/recreate, hidden,
|
||||
parent-child, and session-reset tests.
|
||||
|
||||
**Gate:** shadow scene matches canonical world/resource checkpoints through the
|
||||
nine-stop route with bounded memory and zero authoritative ownership.
|
||||
|
||||
### Slice G — Render snapshot and delta submission cutover
|
||||
|
||||
**Purpose:** Make render cost proportional to visible/changed data.
|
||||
|
||||
- Build the double-buffered `RenderFrameView`.
|
||||
- Feed it the existing PView visible-cell/clip product.
|
||||
- Replace full `InteriorEntityPartition` scans with render-scene bucket queries.
|
||||
- Upload only dirty persistent instance ranges.
|
||||
- Reuse MDI command/sort buffers.
|
||||
- Build instance-set comparison and fixed-camera screenshot comparison.
|
||||
- Run old and new submissions in compare mode without drawing twice.
|
||||
- At the gate, delete the replaced production enumeration/submission path.
|
||||
|
||||
**Gate:** instance sets and screenshots match; dense-town uncapped target is
|
||||
300 FPS or the corrected profile identifies a new dominant stage; steady frame
|
||||
allocation is near zero.
|
||||
|
||||
### Slice H — Event-driven UI, diagnostics, lights, and frame cleanup
|
||||
|
||||
**Purpose:** Remove remaining work that scales with uncapped FPS.
|
||||
|
||||
- Skip diagnostics with no consumer and reuse renderer visibility facts.
|
||||
- Dirty-layout retained UI: apply anchors only after geometry changes.
|
||||
- Maintain an overlay-participant registry instead of a second full-tree walk.
|
||||
- Track active cooldown items/effects/dialogs through events.
|
||||
- Replace per-frame frame-input objects, LINQ arrays, iterator objects, and
|
||||
liveness collections with reusable storage or borrowed views.
|
||||
- Make light candidates spatial/cell-driven and select top-k without a complete
|
||||
per-frame sort.
|
||||
- Replace socket-timeout exceptions and datagram `ToArray` copies with normal
|
||||
cancellable/poll/span-based I/O.
|
||||
|
||||
**Gate:** no steady-state Gen-2 collections; steady allocation target
|
||||
≤ 4 KiB/frame initially and zero for the core world/render loop; UI and network
|
||||
behavior tests unchanged.
|
||||
|
||||
### Slice I — Flat collision assets and residual zero-allocation work
|
||||
|
||||
**Purpose:** Complete MP4 without changing retail math.
|
||||
|
||||
- Bake flattened index-based BSP/collision records.
|
||||
- Port traversal data access line-for-line while preserving ordering and
|
||||
arithmetic.
|
||||
- Remove parsed DBObj graphs from streaming collision publication.
|
||||
- Address remaining measured allocation sites, including any physics transition
|
||||
pooling only after identity/lifetime tests prove it safe.
|
||||
|
||||
**Gate:** trajectory and retail conformance suites remain bit-equivalent;
|
||||
navigation/collision fixtures match; portal and steady allocation targets pass.
|
||||
|
||||
### Slice J — Presentation-independent `AcDream.Runtime`
|
||||
|
||||
**Purpose:** Establish the graphical/headless shared client kernel.
|
||||
|
||||
- Add the Runtime project and dependency guards.
|
||||
- Define `IGameRuntimeView`, commands, ordered events, clock, and lifecycle.
|
||||
- Adapt the current graphical host to those interfaces first.
|
||||
- Remove App presentation dependencies from canonical gameplay owners.
|
||||
- Move owners by coherent lifetime groups; never mirror state.
|
||||
- Add host-parity and no-backend construction tests after each move.
|
||||
- End with `GameWindow`/App composing one `GameRuntime`.
|
||||
|
||||
**Gate:** the connected graphical route is unchanged; a no-window integration
|
||||
test connects, enters world, moves, receives inventory/chat/world updates,
|
||||
portals, logs out, reconnects, and tears down without loading App/Silk/OpenAL.
|
||||
|
||||
### Slice K — Linux headless and multi-session host
|
||||
|
||||
**Purpose:** Deliver efficient automated bots.
|
||||
|
||||
- Add Linux CI for Core, Core.Net, Content, Runtime, and Headless.
|
||||
- Implement portable path/config/credential handling.
|
||||
- Add `AcDream.Headless` CLI and deterministic scheduler.
|
||||
- Add bot command/event APIs for movement, selection, use, combat, spells,
|
||||
looting, chat, and commands.
|
||||
- Share immutable content/collision stores across sessions.
|
||||
- Audit and eliminate mutable statics and process-wide session state.
|
||||
- Add clean cancellation, reconnect, plugin isolation, and credential-safe logs.
|
||||
- Stress 1, 5, 10, and 30 sessions.
|
||||
|
||||
**Gate:** 30 local-server sessions in one Linux process, no GPU/display/audio
|
||||
dependency, bounded incremental memory per session, no busy-loop CPU, clean
|
||||
teardown/reconnect, and parity with graphical runtime command/event behavior.
|
||||
|
||||
### Slice L — Linux graphical host and evidence-gated GPU work
|
||||
|
||||
**Purpose:** Finish platform portability and only then pursue remaining GPU
|
||||
opportunities.
|
||||
|
||||
- Validate Linux OpenGL extension/driver matrix and package native dependencies.
|
||||
- Port paths, frame pacing, input, audio, and packaging without renderer
|
||||
fallback.
|
||||
- Reprofile locally with accurate pass timing.
|
||||
- Add GPU culling, command compaction, particle simulation, or light selection
|
||||
only for stages proven dominant.
|
||||
|
||||
**Gate:** graphical Linux connected route passes the same lifecycle, screenshot,
|
||||
resource, and performance checks; every GPU migration has a CPU/GPU before/after
|
||||
and visual equivalence result.
|
||||
|
||||
---
|
||||
|
||||
## 11. Performance gates
|
||||
|
||||
Reference measurements use a Release build, the same account/route, stable
|
||||
camera scripts, and both capped and uncapped modes. RDP results are kept
|
||||
separate from local-display results.
|
||||
|
||||
| Metric | Target |
|
||||
|---|---:|
|
||||
| Traversal frame p99 | ≤ 16.67 ms on reference local hardware |
|
||||
| Traversal maximum | ≤ 33.3 ms after warm process startup |
|
||||
| Dense-town uncapped frame p50 | ≤ 3.33 ms or a newly attributed blocker |
|
||||
| Steady update p95 | ≤ 2 ms |
|
||||
| Core world/render allocation | 0 B/frame after warmup |
|
||||
| Whole-client steady allocation | initial gate ≤ 4 KiB/frame |
|
||||
| Portal single-frame allocation | ≤ 4 MiB, then tighten from evidence |
|
||||
| Gen-2 collections during canonical route | 0 after startup/bake warmup |
|
||||
| Same-location third-visit resource growth | ≤ 5% and explained |
|
||||
| World-visible-before-ready events | 0 |
|
||||
| Staged resources at stable checkpoints | 0 |
|
||||
| Exception-as-control-flow | 0 known sites |
|
||||
| Headless renderer/audio/window allocations | 0 |
|
||||
|
||||
Memory budgets are measured by category rather than only process working set:
|
||||
|
||||
- Live managed heap.
|
||||
- GC committed/fragmented.
|
||||
- Native prepared/staging buffers.
|
||||
- Mapped content pages.
|
||||
- Tracked GPU buffers/textures.
|
||||
- Driver-reported dedicated/shared GPU memory.
|
||||
|
||||
Absolute process/VRAM targets are fixed after Slice D produces trustworthy
|
||||
category accounting; until then, the binding rule is plateau plus no regression
|
||||
in pixels/range.
|
||||
|
||||
---
|
||||
|
||||
## 12. Test and review matrix
|
||||
|
||||
Every slice must pass the tests relevant to its boundary:
|
||||
|
||||
1. **Pure unit tests:** handles, generations, budgets, queues, manifest,
|
||||
deduplication, stale/corrupt data, snapshot buffers.
|
||||
2. **DAT equivalence:** live extractor versus prepared asset for representative
|
||||
outdoor, dungeon, town, portal, animated, translucent, and particle assets.
|
||||
3. **Scene equivalence:** canonical entity/instance digest at fixed checkpoints.
|
||||
4. **Render equivalence:** instance-set diff plus fixed-camera screenshots.
|
||||
5. **Retail conformance:** named-retail citations for any AC-specific selection,
|
||||
degradation, visibility, collision, or timing behavior touched.
|
||||
6. **Lifecycle:** fresh login, same-location revisit, world edge, dungeon,
|
||||
repeated recalls, rapid generation replacement, logout/reconnect, graceful
|
||||
close, cancellation, and failure injection.
|
||||
7. **Resource:** exact create/retire counts, heap diff, GPU accounting, no stale
|
||||
owner or queued work at stable checkpoints.
|
||||
8. **Performance:** capped and uncapped CPU/GPU/allocation/GC percentiles.
|
||||
9. **Headless parity:** same runtime packet/order/command results with
|
||||
presentation attached or absent.
|
||||
10. **Platform:** Windows graphical/headless plus Linux headless; Linux graphical
|
||||
begins in Slice L.
|
||||
|
||||
For a replacement cutover:
|
||||
|
||||
- Shadow/compare mode may exist only during development.
|
||||
- Confirmed differences are fixed at their source.
|
||||
- The gate commit removes the old production path.
|
||||
- No test threshold is loosened to accept a regression.
|
||||
|
||||
---
|
||||
|
||||
## 13. Risks and controls
|
||||
|
||||
| Risk | Control |
|
||||
|---|---|
|
||||
| Pak repeats the 865 GB failure | content identity and aliasing land before another full bake; bake reports unique/shared ratios continuously |
|
||||
| Prepared data changes pixels/geometry | byte/field equivalence against the current extractor; no compression in the equivalence phase |
|
||||
| Streaming budget reveals an incomplete world | one destination-generation readiness barrier; reveal is an atomic state edge |
|
||||
| Deferred retirement leaks or revives stale resources | incarnation/generation owner tokens and stable-checkpoint zero-outstanding assertions |
|
||||
| Render ECS becomes a second game world | render-only component contract; no network properties or authoritative GUID map; digest checks against canonical owner |
|
||||
| Snapshot adds a frame of latency | same-thread borrowed snapshot first; frame-number assertions; concurrency remains deferred |
|
||||
| Arch dependency constrains the design | acdream-owned `IRenderScene`; Arch only in App; no Arch types cross the boundary |
|
||||
| Headless extraction duplicates state | move the existing canonical owner in lifetime groups; adapters first, never synchronize two stores |
|
||||
| Shared 30-client cache gains mutable state | immutable content only; tests create conflicting sessions/characters concurrently |
|
||||
| Linux work forces renderer fallback | graphical host retains mandatory modern extensions and fails clearly on unsupported drivers |
|
||||
| “Optimization” changes retail behavior | unchanged behavior is a hard gate; retail ports remain OO and traceable; visual/connected gates stay mandatory |
|
||||
|
||||
---
|
||||
|
||||
## 14. Recommended starting point
|
||||
|
||||
Begin with **Slice A**, then **Slice B**.
|
||||
|
||||
Slice A is small and makes all later GPU and frame gates honest. Slice B completes
|
||||
an implementation already present in the tree and directly attacks the
|
||||
benchmark's largest allocation source. The first meaningful runtime cutover is
|
||||
Slice C; do not begin the render-scene migration before the portal content storm
|
||||
has been removed and measured.
|
||||
|
||||
The intended order is therefore:
|
||||
|
||||
```text
|
||||
honest metrics
|
||||
-> prepared content
|
||||
-> typed residence
|
||||
-> cost-budgeted streaming
|
||||
-> incremental render scene
|
||||
-> delta GPU submission
|
||||
-> residual frame cleanup
|
||||
-> presentation-independent runtime
|
||||
-> Linux/headless/multi-session
|
||||
-> evidence-gated GPU jobs
|
||||
```
|
||||
|
||||
This order improves the current graphical client first while laying clean
|
||||
boundaries for Linux and approximately 30 automated headless clients later.
|
||||
Loading…
Add table
Add a link
Reference in a new issue