diff --git a/docs/plans/2026-07-24-modern-runtime-slice-c-prepared-assets.md b/docs/plans/2026-07-24-modern-runtime-slice-c-prepared-assets.md new file mode 100644 index 00000000..18c69102 --- /dev/null +++ b/docs/plans/2026-07-24-modern-runtime-slice-c-prepared-assets.md @@ -0,0 +1,306 @@ +# Modern Runtime Slice C — prepared-asset cutover + +**Date:** 2026-07-24 + +**Status:** active + +**Parent:** [`2026-07-24-modern-runtime-architecture.md`](2026-07-24-modern-runtime-architecture.md), +Slice C / MP1c + +**Authoritative before-state:** +[`../research/2026-07-24-slice-a-physical-local-baselines.md`](../research/2026-07-24-slice-a-physical-local-baselines.md) +at source and binary commit +`49f3a48ee63b82d00731b0f91497930925279c86`. + +## 1. Outcome + +Production world-mesh streaming reads immutable `ObjectMeshData` from the +validated `acdream.pak`; it no longer reconstructs Setup, GfxObj, EnvCell, +Surface, texture, and particle dependency graphs from DAT on four background +workers during gameplay. + +DAT remains the retail source of truth for simulation, UI, animation, dynamic +appearance composition, physics, audio, and effects. The cutover changes when +static render payloads are prepared, not their bytes, pixels, ownership, or +gameplay meaning. + +The slice is complete only when: + +- the package is a required production input with exact DAT-iteration and bake + version validation; +- ordinary GfxObj and deduplicated EnvCell requests use typed pak keys; +- missing, corrupt, canceled, and successful reads remain distinct; +- production streaming has no implicit live-DAT extraction fallback; +- Setup activation uses an explicit typed presence check before parsing DAT; +- Portal DAT still wins when an ID exists in both Portal and HighRes; +- cancellation and unexpected exceptions are never swallowed; +- installed-DAT byte/field equivalence and the connected visual/lifetime gates + pass; +- portal allocation, exception count, p99, and repeated-Caul memory materially + improve against the committed physical baseline. + +## 2. Fixed architecture + +### 2.1 Content-layer contract + +Add a BCL/Content-only `IPreparedAssetSource` with a typed request and typed +result: + +```csharp +enum PreparedAssetReadStatus +{ + Loaded, + Missing, + Corrupt, +} + +readonly record struct PreparedAssetRequest( + PakAssetType Type, + uint SourceFileId, + ulong RuntimeObjectId, + PreparedEnvCellSchema? EnvCell); + +readonly record struct PreparedAssetReadResult( + PreparedAssetReadStatus Status, + ObjectMeshData? Data); + +interface IPreparedAssetSource : IDisposable +{ + PreparedAssetPresence Probe(PakAssetType type, uint sourceFileId); + PreparedAssetReadResult Read( + in PreparedAssetRequest request, + CancellationToken cancellationToken); + PreparedAssetSourceStats Stats { get; } +} +``` + +`CancellationToken` cancellation throws `OperationCanceledException`; it is +not converted into a missing asset. `Loaded` always carries data and +Missing/Corrupt never do. + +The existing `PakKey` already persists the asset-type byte in each TOC key. +Therefore Slice C does not invent a second type table or bump the binary +format. A TOC-only probe binary-searches the exact +`(PakAssetType, SourceFileId)` key without reading, CRC-checking, copying, or +deserializing the blob. This is the explicit positive/negative/type metadata. + +`PakPreparedAssetSource` owns one `PakReader`. It: + +- validates format, bake-tool version, and all four DAT iterations at open; +- exposes TOC-only presence; +- maps PakReader's missing/corrupt/loaded outcomes without ambiguity; +- validates the deserialized payload's logical type and runtime identity; +- never falls through to DAT extraction. + +An explicit `DatPreparedAssetSource` is retained only for bake/equivalence and +UI Studio tooling. It uses `MeshExtractor` and is never selected by +`GameWindow`. + +### 2.2 Request identity + +The renderer's internal runtime ID and the package lookup ID are not always +the same: + +| Request | Pak key | `ObjectMeshData.ObjectId` | +|---|---|---| +| ordinary world/live mesh | `GfxObjMesh + GfxObj DID` | GfxObj DID | +| Setup tooling request | `SetupMesh + Setup DID` | Setup DID | +| EnvCell shell | `EnvCellMesh + source Cell DID` | deduplicated geometry ID | + +`EnvCellShellPlacement.CellId` is therefore carried into +`EnvCellMeshPreparationScheduler` and retained beside the existing geometry +schema. Cancellation/re-arm reuses that exact descriptor; it never derives a +fake DAT ID from the synthetic geometry hash. + +Aliases in the pak make every source Cell DID resolve to the one shared +physical geometry blob. Runtime validates that the blob's `ObjectId` equals +the scheduler's deduplicated geometry ID before publishing it. + +### 2.3 Ownership and thread model + +- `ContentEffectsAudioComposition` opens DAT first, then the prepared source, + and publishes both through typed owners. +- `GameWindow` stores only the additional lifetime root and passes the source + through `ContentEffectsAudioResult`. +- `WorldRenderComposition` injects the source into `WbMeshAdapter`. +- `ObjectMeshManager`'s existing four workers read immutable prepared + payloads. They retain the current request deduplication, LIFO locality, + cancellation, CPU cache, staging high-water, ownership, and retry rules. +- Pak reads and deserialization remain worker-only. GL upload remains + render-thread-only. +- Shutdown disposes `WbMeshAdapter` and joins its workers before unmapping the + pak, then disposes DAT. Construction rollback uses the reverse order. +- No worker, source, or result stores a window reference or service locator. + +### 2.4 Setup activation and non-allocating DAT lookup + +`WorldEntity.SourceGfxObjOrSetupId` can be either a GfxObj or Setup. The current +`SequencerFactory` and `ResolveActivation` try to parse every value as Setup; +the physical baseline records 4,659 `ArgumentOutOfRangeException` events on one +uncapped route. + +Both paths first probe `SetupMesh + id` in the prepared source: + +- absent: return the existing no-Setup/no-script result without a DAT call; +- present: parse the valid Setup through the bounded DAT object cache; +- corrupt package entry: renderer load fails loudly; activation does not guess; +- cancellation: rethrow; +- known corrupt DAT data (`InvalidDataException`, `EndOfStreamException`, and + the exact verified DatReaderWriter malformed-record exception): diagnose and + return no activation; +- unexpected exceptions: diagnose and rethrow. + +The exception list is evidence-driven from installed fixtures and tests; a +blanket `catch` is forbidden. + +`IDatReaderWriter.TryResolvePreferred` is added as a non-allocating lookup. +The concrete adapter checks Portal first, then HighRes, Language, and Cell. +That encodes the existing load-bearing Portal-wins behavior directly. +`ResolveId()` remains for compatibility tooling, but production mesh +extraction and bounds code no longer use +`ResolveId().ToList().OrderByDescending(...)`. + +### 2.5 Texture identity + +Pak payloads contain default-palette texture bytes. Dynamic appearance +composition continues through the existing owner-scoped `TextureCache`. +Default and palette-overlaid pixels retain distinct cache keys; the prepared +source never inserts default bytes into the dynamic composite cache. + +## 3. Checkpoints + +### C1 — typed prepared-source boundary + +Implement: + +- typed request/presence/read-result/stat records; +- PakReader TOC-only probe and status-preserving read; +- `PakPreparedAssetSource` compatibility validation; +- explicit `DatPreparedAssetSource` for tooling; +- `RuntimeOptions.PreparedAssetPath`, defaulting to + `/acdream.pak`, with `ACDREAM_PAK_PATH` as the sole override; +- Content tests for absent/present/corrupt/canceled/type-mismatched reads, + alias identity, iteration mismatch, bake-version mismatch, and disposal. + +Gate: + +- Content and Bake tests green; +- current pak round-trip/equivalence tests remain green; +- no App/runtime behavior changed yet. + +Commit as one boundary unit. + +### C2 — production renderer injection + +Implement: + +- prepared-source acquisition/publication/teardown in composition; +- `WorldRenderComposition` and `WbMeshAdapter` injection; +- `ObjectMeshManager` worker reads through `IPreparedAssetSource`; +- source Cell DID propagation for EnvCell aliases; +- exact terminal-failure and cancellation semantics; +- current CPU cache, staging, upload, retry, ownership, and shutdown behavior + retained; +- UI Studio explicitly opts into the live-DAT tooling source; +- remove production worker calls to `MeshExtractor`. + +Focused tests: + +- GfxObj request key and identity; +- EnvCell alias key plus geometry identity; +- cached/re-armed/canceled generation; +- missing and corrupt entries become terminal preparation failures without + retry storms; +- source disposed only after workers and mesh adapter; +- production composition cannot construct without a valid prepared source; +- tooling source remains explicit. + +Gate: + +- App Release build/tests green; +- source audit finds no production `MeshExtractor.Prepare*` call from the + streaming workers; +- missing pak and incompatible pak fail at startup with actionable messages. + +Commit as one integration unit. + +### C3 — activation probes and lookup precedence + +Implement: + +- Setup-presence gate shared by `SequencerFactory` and `ResolveActivation`; +- specific corrupt-data exception handling and diagnostics; +- cancellation and unexpected-exception propagation; +- non-allocating Portal-first `TryResolvePreferred`; +- remove the old LINQ lookup in `MeshExtractor`, recursive part collection, + and remaining bounds helper; +- explicit default-palette versus palette-overlay cache-identity tests. + +Focused tests: + +- GfxObj entity never enters Setup parser; +- valid Setup parses once then hits bounded cache; +- corrupt Setup is diagnosed once; +- unexpected exception is visible; +- cancellation propagates; +- dual-DAT fixture chooses Portal; +- missing ID allocates no result collection; +- drudge, robed-player, and palette-dyed-armor seam fixtures preserve fields + and pixels. + +Gate: + +- no `ResolveActivation` bare catch; +- zero `ArgumentOutOfRangeException` Setup-probe storm on connected route; +- full Release suite green. + +Commit as one semantic-cleanup unit. + +### C4 — bake, connected equivalence, and performance closeout + +1. Re-bake installed DATs atomically to the production path with the current + tool. +2. Validate the full artifact, key counts, aliases, CRC contract, iterations, + and exact source/binary provenance. +3. Run focused byte/field equivalence and seam fixtures. +4. Run the capped and uncapped physical nine-stop routes plus pinned dense + Arwic with the same reference configuration. +5. Compare fixed-camera screenshots under the committed tolerance/mask rule. +6. Compare per-portal p99/allocation, process allocation/GC, exception count, + cache counters, and repeated-Caul ownership/memory to Slice A. +7. Run Release build and the complete Release suite. +8. Update the parent plan, roadmap, WorldBuilder inventory, divergence + register if and only if a real deviation changed, and durable memory. + +Acceptance: + +- every route reaches every destination and disconnects gracefully; +- no world, animation, particle, effect, texture, mesh, landblock, or GPU + ownership regression; +- fixed-camera screenshots pass; +- no invalid Setup exception storm; +- no production DAT mesh extraction; +- portal single-frame allocation and p99 materially improve; +- process allocation/GC materially improve; +- repeated-Caul memory passes the existing assertion without weakening it. + +If pixels or behavior differ, the slice is not complete even when performance +improves. If timing improves but heap reservation still fails, keep the failure +open and attribute it before considering Slice C closed. + +## 4. Review checklist per checkpoint + +Because this task is running without delegated reviewers, the primary engineer +performs three explicit read-only passes over each complete checkpoint diff: + +1. **Contract/conformance:** request identity, Portal precedence, package + compatibility, byte/field/pixel equivalence. +2. **Architecture/lifetime:** dependency direction, worker/render-thread + ownership, cancellation, construction rollback, shutdown order, no + production fallback. +3. **Adversarial:** corrupt/truncated package, missing key, stale/canceled + request, alias mismatch, callback failure, exception visibility, repeated + portal/GUID/landblock churn. + +Every confirmed finding is fixed and the full diff is re-read before the +checkpoint commit.