347 lines
14 KiB
Markdown
347 lines
14 KiB
Markdown
# Modern Runtime Slice C — prepared-asset cutover
|
|
|
|
**Date:** 2026-07-24
|
|
|
|
**Status:** complete — automated, connected, performance, and user visual
|
|
gates passed 2026-07-24
|
|
|
|
**Checkpoint ledger:**
|
|
|
|
- C1 typed prepared-source boundary — `c42f93b3`
|
|
- C2 production renderer injection — `f05afc07`
|
|
- C3 activation probes and lookup precedence — `230a7df4`; automated gate
|
|
green
|
|
- C4 prepared-source diagnostics — `b1ad4b7c`
|
|
- C4 connected equivalence and performance closeout —
|
|
[`../research/2026-07-24-slice-c-prepared-asset-cutover-report.md`](../research/2026-07-24-slice-c-prepared-asset-cutover-report.md)
|
|
|
|
**Parent:** [`2026-07-24-modern-runtime-architecture.md`](2026-07-24-modern-runtime-architecture.md),
|
|
Slice C / MP1c
|
|
|
|
**Authoritative before-state:**
|
|
[`../research/2026-07-24-slice-a-physical-local-baselines.md`](../research/2026-07-24-slice-a-physical-local-baselines.md)
|
|
at source and binary commit
|
|
`49f3a48ee63b82d00731b0f91497930925279c86`.
|
|
|
|
## 1. Outcome
|
|
|
|
Production world-mesh streaming reads immutable `ObjectMeshData` from the
|
|
validated `acdream.pak`; it no longer reconstructs Setup, GfxObj, EnvCell,
|
|
Surface, texture, and particle dependency graphs from DAT on four background
|
|
workers during gameplay.
|
|
|
|
DAT remains the retail source of truth for simulation, UI, animation, dynamic
|
|
appearance composition, physics, audio, and effects. The cutover changes when
|
|
static render payloads are prepared, not their bytes, pixels, ownership, or
|
|
gameplay meaning.
|
|
|
|
The slice is complete only when:
|
|
|
|
- the package is a required production input with exact DAT-iteration and bake
|
|
version validation;
|
|
- ordinary GfxObj and deduplicated EnvCell requests use typed pak keys;
|
|
- missing, corrupt, canceled, and successful reads remain distinct;
|
|
- production streaming has no implicit live-DAT extraction fallback;
|
|
- Setup activation uses an explicit typed presence check before parsing DAT;
|
|
- Portal DAT still wins when an ID exists in both Portal and HighRes;
|
|
- cancellation and unexpected exceptions are never swallowed;
|
|
- installed-DAT byte/field equivalence and the connected visual/lifetime gates
|
|
pass;
|
|
- portal allocation, exception count, p99, and repeated-Caul memory materially
|
|
improve against the committed physical baseline.
|
|
|
|
## 2. Fixed architecture
|
|
|
|
### 2.1 Content-layer contract
|
|
|
|
Add a BCL/Content-only `IPreparedAssetSource` with a typed request and typed
|
|
result:
|
|
|
|
```csharp
|
|
enum PreparedAssetReadStatus
|
|
{
|
|
Loaded,
|
|
Missing,
|
|
Corrupt,
|
|
}
|
|
|
|
readonly record struct PreparedAssetRequest(
|
|
PakAssetType Type,
|
|
uint SourceFileId,
|
|
ulong RuntimeObjectId,
|
|
PreparedEnvCellSchema? EnvCell);
|
|
|
|
readonly record struct PreparedAssetReadResult(
|
|
PreparedAssetReadStatus Status,
|
|
ObjectMeshData? Data);
|
|
|
|
interface IPreparedAssetSource : IDisposable
|
|
{
|
|
PreparedAssetPresence Probe(PakAssetType type, uint sourceFileId);
|
|
PreparedAssetReadResult Read(
|
|
in PreparedAssetRequest request,
|
|
CancellationToken cancellationToken);
|
|
PreparedAssetSourceStats Stats { get; }
|
|
}
|
|
```
|
|
|
|
`CancellationToken` cancellation throws `OperationCanceledException`; it is
|
|
not converted into a missing asset. `Loaded` always carries data and
|
|
Missing/Corrupt never do.
|
|
|
|
The existing `PakKey` already persists the asset-type byte in each TOC key.
|
|
Therefore Slice C does not invent a second type table or bump the binary
|
|
format. A TOC-only probe binary-searches the exact
|
|
`(PakAssetType, SourceFileId)` key without reading, CRC-checking, copying, or
|
|
deserializing the blob. This is the explicit positive/negative/type metadata.
|
|
|
|
`PakPreparedAssetSource` owns one `PakReader`. It:
|
|
|
|
- validates format, bake-tool version, and all four DAT iterations at open;
|
|
- exposes TOC-only presence;
|
|
- maps PakReader's missing/corrupt/loaded outcomes without ambiguity;
|
|
- validates the deserialized payload's logical type and runtime identity;
|
|
- never falls through to DAT extraction.
|
|
|
|
An explicit `DatPreparedAssetSource` is retained only for bake/equivalence and
|
|
UI Studio tooling. It uses `MeshExtractor` and is never selected by
|
|
`GameWindow`.
|
|
|
|
### 2.2 Request identity
|
|
|
|
The renderer's internal runtime ID and the package lookup ID are not always
|
|
the same:
|
|
|
|
| Request | Pak key | `ObjectMeshData.ObjectId` |
|
|
|---|---|---|
|
|
| ordinary world/live mesh | `GfxObjMesh + GfxObj DID` | GfxObj DID |
|
|
| Setup tooling request | `SetupMesh + Setup DID` | Setup DID |
|
|
| EnvCell shell | `EnvCellMesh + source Cell DID` | deduplicated geometry ID |
|
|
|
|
`EnvCellShellPlacement.CellId` is therefore carried into
|
|
`EnvCellMeshPreparationScheduler` and retained beside the existing geometry
|
|
schema. Cancellation/re-arm reuses that exact descriptor; it never derives a
|
|
fake DAT ID from the synthetic geometry hash.
|
|
|
|
Aliases in the pak make every source Cell DID resolve to the one shared
|
|
physical geometry blob. Runtime validates that the blob's `ObjectId` equals
|
|
the scheduler's deduplicated geometry ID before publishing it.
|
|
|
|
### 2.3 Ownership and thread model
|
|
|
|
- `ContentEffectsAudioComposition` opens DAT first, then the prepared source,
|
|
and publishes both through typed owners.
|
|
- `GameWindow` stores only the additional lifetime root and passes the source
|
|
through `ContentEffectsAudioResult`.
|
|
- `WorldRenderComposition` injects the source into `WbMeshAdapter`.
|
|
- `ObjectMeshManager`'s existing four workers read immutable prepared
|
|
payloads. They retain the current request deduplication, LIFO locality,
|
|
cancellation, CPU cache, staging high-water, ownership, and retry rules.
|
|
- Pak reads and deserialization remain worker-only. GL upload remains
|
|
render-thread-only.
|
|
- Shutdown disposes `WbMeshAdapter` and joins its workers before unmapping the
|
|
pak, then disposes DAT. Construction rollback uses the reverse order.
|
|
- No worker, source, or result stores a window reference or service locator.
|
|
|
|
### 2.4 Setup activation and non-allocating DAT lookup
|
|
|
|
`WorldEntity.SourceGfxObjOrSetupId` can be either a GfxObj or Setup. The current
|
|
`SequencerFactory` and `ResolveActivation` try to parse every value as Setup;
|
|
the physical baseline records 4,659 `ArgumentOutOfRangeException` events on one
|
|
uncapped route.
|
|
|
|
Both paths first probe `SetupMesh + id` in the prepared source:
|
|
|
|
- absent: return the existing no-Setup/no-script result without a DAT call;
|
|
- present: parse the valid Setup through the bounded DAT object cache;
|
|
- corrupt package entry: renderer load fails loudly; activation does not guess;
|
|
- cancellation: rethrow;
|
|
- known corrupt DAT data (`InvalidDataException`, `EndOfStreamException`, and
|
|
the exact verified DatReaderWriter malformed-record exception): diagnose and
|
|
return no activation;
|
|
- unexpected exceptions: diagnose and rethrow.
|
|
|
|
The exception list is evidence-driven from installed fixtures and tests; a
|
|
blanket `catch` is forbidden.
|
|
|
|
`IDatReaderWriter.TryResolvePreferred` is added as a non-allocating lookup.
|
|
The concrete adapter checks Portal first, then HighRes, Language, and Cell.
|
|
That encodes the existing load-bearing Portal-wins behavior directly.
|
|
`ResolveId()` remains for compatibility tooling, but production mesh
|
|
extraction and bounds code no longer use
|
|
`ResolveId().ToList().OrderByDescending(...)`.
|
|
|
|
### 2.5 Texture identity
|
|
|
|
Pak payloads contain default-palette texture bytes. Dynamic appearance
|
|
composition continues through the existing owner-scoped `TextureCache`.
|
|
Default and palette-overlaid pixels retain distinct cache keys; the prepared
|
|
source never inserts default bytes into the dynamic composite cache.
|
|
|
|
## 3. Checkpoints
|
|
|
|
### C1 — typed prepared-source boundary
|
|
|
|
Implement:
|
|
|
|
- typed request/presence/read-result/stat records;
|
|
- PakReader TOC-only probe and status-preserving read;
|
|
- `PakPreparedAssetSource` compatibility validation;
|
|
- explicit `DatPreparedAssetSource` for tooling;
|
|
- `RuntimeOptions.PreparedAssetPath`, defaulting to
|
|
`<DatDir>/acdream.pak`, with `ACDREAM_PAK_PATH` as the sole override;
|
|
- Content tests for absent/present/corrupt/canceled/type-mismatched reads,
|
|
alias identity, iteration mismatch, bake-version mismatch, and disposal.
|
|
|
|
Gate:
|
|
|
|
- Content and Bake tests green;
|
|
- current pak round-trip/equivalence tests remain green;
|
|
- no App/runtime behavior changed yet.
|
|
|
|
Commit as one boundary unit.
|
|
|
|
### C2 — production renderer injection
|
|
|
|
Implement:
|
|
|
|
- prepared-source acquisition/publication/teardown in composition;
|
|
- `WorldRenderComposition` and `WbMeshAdapter` injection;
|
|
- `ObjectMeshManager` worker reads through `IPreparedAssetSource`;
|
|
- source Cell DID propagation for EnvCell aliases;
|
|
- exact `TranslucencyKind` persisted per prepared texture batch, removing the
|
|
render-thread `GfxObjMesh.Build` metadata reconstruction (bake-tool identity
|
|
advances to version 3; the pak container remains format version 1);
|
|
- exact terminal-failure and cancellation semantics;
|
|
- current CPU cache, staging, upload, retry, ownership, and shutdown behavior
|
|
retained;
|
|
- UI Studio explicitly opts into the live-DAT tooling source;
|
|
- remove production worker calls to `MeshExtractor`.
|
|
|
|
Focused tests:
|
|
|
|
- GfxObj request key and identity;
|
|
- EnvCell alias key plus geometry identity;
|
|
- cached/re-armed/canceled generation;
|
|
- missing and corrupt entries become terminal preparation failures without
|
|
retry storms;
|
|
- source disposed only after workers and mesh adapter;
|
|
- production composition cannot construct without a valid prepared source;
|
|
- tooling source remains explicit.
|
|
|
|
Gate:
|
|
|
|
- App Release build/tests green;
|
|
- source audit finds no production `MeshExtractor.Prepare*` call from the
|
|
streaming workers;
|
|
- missing pak and incompatible pak fail at startup with actionable messages.
|
|
|
|
Commit as one integration unit.
|
|
|
|
### C3 — activation probes and lookup precedence
|
|
|
|
Implement:
|
|
|
|
- Setup-presence gate shared by `SequencerFactory` and `ResolveActivation`;
|
|
- specific corrupt-data exception handling and diagnostics;
|
|
- cancellation and unexpected-exception propagation;
|
|
- non-allocating Portal-first `TryResolvePreferred`;
|
|
- remove the old LINQ lookup in `MeshExtractor`, recursive part collection,
|
|
and remaining bounds helper;
|
|
- explicit default-palette versus palette-overlay cache-identity tests.
|
|
|
|
Focused tests:
|
|
|
|
- GfxObj entity never enters Setup parser;
|
|
- valid Setup parses once then hits bounded cache;
|
|
- corrupt Setup is diagnosed once;
|
|
- unexpected exception is visible;
|
|
- cancellation propagates;
|
|
- dual-DAT fixture chooses Portal;
|
|
- missing ID allocates no result collection;
|
|
- drudge, robed-player, and palette-dyed-armor seam fixtures preserve fields
|
|
and pixels.
|
|
|
|
Gate:
|
|
|
|
- no `ResolveActivation` bare catch;
|
|
- zero `ArgumentOutOfRangeException` Setup-probe storm on connected route;
|
|
- full Release suite green.
|
|
|
|
Commit as one semantic-cleanup unit.
|
|
|
|
### C4 — bake, connected equivalence, and performance closeout
|
|
|
|
1. Re-bake installed DATs atomically to the production path with the current
|
|
tool.
|
|
2. Validate the full artifact, key counts, aliases, CRC contract, iterations,
|
|
and exact source/binary provenance.
|
|
3. Run focused byte/field equivalence and seam fixtures.
|
|
4. Run the capped and uncapped physical nine-stop routes plus pinned dense
|
|
Arwic with the same reference configuration.
|
|
5. Compare fixed-camera screenshots under the committed tolerance/mask rule.
|
|
6. Compare per-portal p99/allocation, process allocation/GC, exception count,
|
|
cache counters, prepared-source probes/reads/loaded/missing/corrupt totals,
|
|
and repeated-Caul ownership/memory to Slice A.
|
|
7. Run Release build and the complete Release suite.
|
|
8. Update the parent plan, roadmap, WorldBuilder inventory, divergence
|
|
register if and only if a real deviation changed, and durable memory.
|
|
|
|
Acceptance:
|
|
|
|
- every route reaches every destination and disconnects gracefully;
|
|
- no world, animation, particle, effect, texture, mesh, landblock, or GPU
|
|
ownership regression;
|
|
- fixed-camera screenshots pass;
|
|
- no invalid Setup exception storm;
|
|
- no production DAT mesh extraction;
|
|
- portal single-frame allocation and p99 materially improve;
|
|
- process allocation/GC materially improve;
|
|
- repeated-Caul memory passes the existing assertion without weakening it.
|
|
|
|
If pixels or behavior differ, the slice is not complete even when performance
|
|
improves. If timing improves but heap reservation still fails, keep the failure
|
|
open and attribute it before considering Slice C closed.
|
|
|
|
### C4 result — completed 2026-07-24
|
|
|
|
The format-1/bake-tool-3 production package validated with 751,141 keys and
|
|
the exact installed DAT iterations. Installed-DAT seam/equivalence tests, all
|
|
three connected physical-display routes, canonical lifetime assertions, and
|
|
the complete Release suite passed. The user accepted the prepared path's
|
|
geometry, materials, textures, and world presentation.
|
|
|
|
Against the committed Slice A physical baseline:
|
|
|
|
- capped CPU p95/p99 improved 15.8%/16.9%, GPU p95/p99 17.4%/20.0%;
|
|
- uncapped CPU p95/p99 improved 13.8%/17.0%, GPU p95/p99 22.5%/20.6%;
|
|
- dense-Arwic CPU/GPU p99 improved 12.0%/16.8%;
|
|
- process allocation fell 54.9% capped, 41.8% uncapped, and 61.9% dense;
|
|
- GC pause time fell 52.8%, 48.4%, and 54.1%;
|
|
- the invalid typed-Setup exception count fell to zero in every route;
|
|
- the largest frame-thread allocation fell from roughly 203 MiB to 46.8 MiB
|
|
capped and 39.9 MiB uncapped;
|
|
- both warm repeated-Caul memory comparisons passed the unchanged assertion.
|
|
|
|
Windows reports clean memory-mapped package pages in working set, so total
|
|
working set is higher even though private memory and ownership remain bounded.
|
|
That is file cache, not retained world state. Exact measurements and the
|
|
artifact policy are recorded in the
|
|
[Slice C report](../research/2026-07-24-slice-c-prepared-asset-cutover-report.md).
|
|
|
|
## 4. Review checklist per checkpoint
|
|
|
|
Because this task is running without delegated reviewers, the primary engineer
|
|
performs three explicit read-only passes over each complete checkpoint diff:
|
|
|
|
1. **Contract/conformance:** request identity, Portal precedence, package
|
|
compatibility, byte/field/pixel equivalence.
|
|
2. **Architecture/lifetime:** dependency direction, worker/render-thread
|
|
ownership, cancellation, construction rollback, shutdown order, no
|
|
production fallback.
|
|
3. **Adversarial:** corrupt/truncated package, missing key, stale/canceled
|
|
request, alias mismatch, callback failure, exception visibility, repeated
|
|
portal/GUID/landblock churn.
|
|
|
|
Every confirmed finding is fixed and the full diff is re-read before the
|
|
checkpoint commit.
|