docs(perf): plan Slice B EnvCell dedup gate

This commit is contained in:
Erik 2026-07-24 13:16:33 +02:00
parent 4285f1dbb1
commit 1560c0513f

View file

@ -0,0 +1,136 @@
# Modern Runtime Slice B — EnvCell geometry deduplication and full bake
**Date:** 2026-07-24
**Status:** Planned; implementation follows the Slice A diagnostic capture
**Parent:** [`2026-07-24-modern-runtime-architecture.md`](2026-07-24-modern-runtime-architecture.md), Slice B
**Prior implementation:** [`../superpowers/plans/2026-07-05-mp1b-pak-and-bake.md`](../superpowers/plans/2026-07-05-mp1b-pak-and-bake.md)
## Outcome
Complete the existing MP1b bake without changing runtime behavior. Every
EnvCell file ID remains an independently addressable pak key, while cells with
the same WorldBuilder geometry identity share one serialized blob and one
physical byte range. The client does not consume the pak in this slice.
The full gate must replace the failed 865 GB / 52 minute per-cell bake with a
bounded artifact whose size and time scale with unique geometry, not with all
729,888 cell instances.
## Fixed contracts
1. `DatCollection` remains the only DAT reader.
2. `MeshExtractor` remains the only mesh/texture interpreter used by both the
live path and the bake.
3. Geometry identity is the already-shipped WorldBuilder calculation:
`environmentId`, then `cellStructure`, then the ordered surface IDs through
the `hash = hash * 31 + value` chain, with bit 33 set.
4. The pure identity helper moves to `AcDream.Core.Rendering.Wb`; App and Bake
call the same implementation. No second or adapted hash is allowed.
5. Every EnvCell TOC key is still
`PakKey.Compose(PakAssetType.EnvCellMesh, fileId)`.
6. Aliases share `offset`, `length`, and `crc32`. The pak format and reader do
not gain an indirection table or a second lookup.
7. The shared payload carries the runtime geometry ID in
`ObjectMeshData.ObjectId`, matching the current live
`PrepareCellStructMeshData(geometryId, ...)` path.
8. A geometry-ID collision whose full source tuple differs fails loudly. It
may never alias silently.
9. Blob and TOC order remain deterministic across worker counts.
10. The destination pak is replaced only after a complete temporary artifact
opens and validates. Cancellation or failure preserves the prior file and
deletes the temporary file.
11. The pak itself and raw bake logs are machine artifacts and are not
committed. A compact, path-free bake report is committed.
## Implementation checkpoints
### B1 — One geometry identity seam
- Extract the pure WorldBuilder identity calculation from the App transaction
builder into `AcDream.Core.Rendering.Wb.EnvCellGeometryIdentity`.
- Retain the existing App entry points as delegates so production behavior and
callers do not change.
- Add Core conformance tests for empty surfaces, ordered surfaces, the bit-33
namespace, and App/Bake shared values.
**Gate:** existing App geometry-ID tests and new Core tests pass with the exact
same values.
### B2 — Pak blob aliases
- Make `PakWriter` retain each written key's immutable TOC receipt.
- Add `AddAlias(aliasKey, existingKey)`, rejecting a duplicate alias key,
missing source key, alias after finish, and accidental self/duplicate use.
- Emit a normal TOC row with the alias key and the source receipt's physical
range and CRC.
- Keep the reader unchanged except for test-only receipt inspection.
**Gate:** primary and alias deserialize field-for-field equal, have identical
offset/length/CRC, use one physical blob, preserve sorted TOC order, and isolate
corruption consistently.
### B3 — Deduplicated bake catalog
- Enumerate EnvCell file IDs exactly as today.
- Read each EnvCell header once and produce:
- a sorted `fileId -> geometryId` alias stream;
- one full source descriptor per unique geometry ID.
- Validate equal geometry IDs against the complete tuple
`(environmentId, cellStructure, ordered surfaces)`.
- Extract only the unique descriptors through
`MeshExtractor.PrepareCellStructMeshData(geometryId, ...)`.
- Write the first sorted cell key as the physical blob and every remaining
cell key as an alias.
- Keep GfxObj, Setup, and side-staged particle-preload assets in the same
deterministic writer transaction.
- Report enumerated cells, valid aliases, unique geometries, dedup ratio,
physical blobs, failures, wall time, output bytes, and peak working set.
**Gate:** a filtered real-DAT fixture containing duplicate cells produces more
EnvCell keys than physical EnvCell blobs, and every key reads the same payload
as a live unique-geometry extraction.
### B4 — Transactional output and compatibility
- Bake beside the destination under a unique temporary filename.
- Pass cancellation through enumeration and parallel extraction.
- Open the completed temporary pak and validate its header, TOC count, source
DAT iterations, bake-tool version, and every TOC range before replacement.
- Atomically replace the destination on the same volume.
- Add stale-version, corruption, cancellation, partial-write, existing-target
preservation, and temporary-file cleanup tests.
**Gate:** no failure path damages or replaces a previously valid pak.
### B5 — Full-scale gate
- Run Release build and the focused Content/Bake suites.
- Run two filtered bakes with different thread counts and require byte-identical
output.
- Run the complete unfiltered bake against the installed retail DAT files.
- Open the result, validate every TOC entry, sample aliases across multiple
landblocks, and run the existing live-vs-pak field equivalence suite.
- Record counts, dedup ratio, failures, elapsed time, peak working set, and
artifact size in `docs/research/`.
- Delete the generated pak after validation unless it is intentionally retained
outside the repository for the later Slice C cutover.
**Exit:** the full bake completes in practical time and disk space, all
equivalence and determinism gates are green, duplicate EnvCells demonstrably
share physical offsets, and no runtime code consumes the pak.
## Verification
Run:
```powershell
dotnet test tests/AcDream.Content.Tests/AcDream.Content.Tests.csproj -c Release
dotnet test tests/AcDream.Bake.Tests/AcDream.Bake.Tests.csproj -c Release
dotnet build AcDream.slnx -c Release
dotnet test AcDream.slnx -c Release
```
Then run the full bake from the Release binary, capture its compact report, and
validate the output with `PakReader`. No visual gate is required because this
slice changes no runtime path.