acdream/docs/plans/2026-07-24-modern-runtime-slice-b-envcell-dedup.md

# Modern Runtime Slice B — EnvCell geometry deduplication and full bake

**Date:** 2026-07-24  
**Status:** Complete 2026-07-24; full installed-DAT gate passed
**Parent:** [`2026-07-24-modern-runtime-architecture.md`](2026-07-24-modern-runtime-architecture.md), Slice B  
**Prior implementation:** [`../superpowers/plans/2026-07-05-mp1b-pak-and-bake.md`](../superpowers/plans/2026-07-05-mp1b-pak-and-bake.md)

## Outcome

Complete the existing MP1b bake without changing runtime behavior. Every
EnvCell file ID remains an independently addressable pak key, while cells with
the same collision-resistant geometry identity share one serialized blob and one
physical byte range. The client does not consume the pak in this slice.

The full gate must replace the failed 865 GB / 52 minute per-cell bake with a
bounded artifact whose size and time scale with unique geometry, not with all
729,888 cell instances.

## Fixed contracts

1. `DatCollection` remains the only DAT reader.
2. `MeshExtractor` remains the only mesh/texture interpreter used by both the
   live path and the bake.
3. Geometry identity is one deterministic hash of the complete typed tuple:
   `environmentId`, `cellStructure`, surface count, then the ordered surface
   IDs. App and Bake call the same Core implementation.
4. The pure identity helper lives in
   `AcDream.Core.Rendering.Wb.EnvCellGeometryIdentity`. It uses FNV-1a in a
   dedicated high-bit namespace and retains bit 33 for compatibility with
   existing synthetic-geometry diagnostics. WorldBuilder's original
   `hash = hash * 31 + value` calculation remains as a conformance helper only.
5. Every EnvCell TOC key is still
   `PakKey.Compose(PakAssetType.EnvCellMesh, fileId)`.
6. Aliases share `offset`, `length`, and `crc32`. The pak format and reader do
   not gain an indirection table or a second lookup.
7. The shared payload carries the runtime geometry ID in
   `ObjectMeshData.ObjectId`, matching the current live
   `PrepareCellStructMeshData(geometryId, ...)` path.
8. A geometry-ID collision whose full source tuple differs fails loudly. It
   may never alias silently.
9. Blob and TOC order remain deterministic across worker counts.
10. The destination pak is replaced only after a complete temporary artifact
    opens and validates. Cancellation or failure preserves the prior file and
    deletes the temporary file.
11. The pak itself and raw bake logs are machine artifacts and are not
    committed. A compact, path-free bake report is committed.

**Full-DAT correction (2026-07-24):** the first guarded full-catalog run proved
that the original fixed contract was impossible. Installed retail DAT cells
`0x00030175` (`environment=0x277`, structure `0`, surfaces `0x013B,0x0034`)
and `0x01BC0105` (`environment=0x276`, structure `0`, surfaces
`0x04FC,0x0034`) both produce WorldBuilder ID `0x00000002020E8C13` despite
having visibly different geometry. The difference is algebraic:
one environment step contributes `31³`, exactly canceled by the first-surface
difference `0x04FC - 0x013B = 31²`. The collision guard stopped the bake before
writing an artifact. B1 was therefore corrected at the root: the old polynomial
is retained and tested as historical evidence, while runtime and bake now share
the stronger namespaced identity. The full-catalog collision gate remains
mandatory.

## Implementation checkpoints

### B1 — One geometry identity seam

- Extract one pure identity calculation from the App transaction builder into
  `AcDream.Core.Rendering.Wb.EnvCellGeometryIdentity`.
- Retain the existing App entry points as delegates so production behavior and
  callers do not change.
- Add Core conformance tests for empty surfaces, ordered surfaces, the
  dedicated namespace, App/Bake shared values, and the installed-DAT collision
  in the legacy WorldBuilder polynomial.

**Gate:** existing App geometry-ID tests and new Core tests pass with the exact
same values.

### B2 — Pak blob aliases

- Make `PakWriter` retain each written key's immutable TOC receipt.
- Add `AddAlias(aliasKey, existingKey)`, rejecting a duplicate alias key,
  missing source key, alias after finish, and accidental self/duplicate use.
- Emit a normal TOC row with the alias key and the source receipt's physical
  range and CRC.
- Keep the reader unchanged except for test-only receipt inspection.

**Gate:** primary and alias deserialize field-for-field equal, have identical
offset/length/CRC, use one physical blob, preserve sorted TOC order, and isolate
corruption consistently.

### B3 — Deduplicated bake catalog

- Enumerate EnvCell file IDs exactly as today.
- Read each EnvCell header once and produce:
  - a sorted `fileId -> geometryId` alias stream;
  - one full source descriptor per unique geometry ID.
- Validate equal geometry IDs against the complete tuple
  `(environmentId, cellStructure, ordered surfaces)`.
- Extract only the unique descriptors through
  `MeshExtractor.PrepareCellStructMeshData(geometryId, ...)`.
- Write the first sorted cell key as the physical blob and every remaining
  cell key as an alias.
- Keep GfxObj, Setup, and side-staged particle-preload assets in the same
  deterministic writer transaction.
- Report enumerated cells, valid aliases, unique geometries, dedup ratio,
  physical blobs, failures, wall time, output bytes, and peak working set.

**Gate:** a filtered real-DAT fixture containing duplicate cells produces more
EnvCell keys than physical EnvCell blobs, and every key reads the same payload
as a live unique-geometry extraction.

### B4 — Transactional output and compatibility

- Bake beside the destination under a unique temporary filename.
- Pass cancellation through enumeration and parallel extraction.
- Open the completed temporary pak and validate its header, TOC count, source
  DAT iterations, bake-tool version, and every TOC range before replacement.
- Atomically replace the destination on the same volume.
- Add stale-version, corruption, cancellation, partial-write, existing-target
  preservation, and temporary-file cleanup tests.

**Gate:** no failure path damages or replaces a previously valid pak.

### B5 — Full-scale gate

- Run Release build and the focused Content/Bake suites.
- Run two filtered bakes with different thread counts and require byte-identical
  output.
- Run the complete unfiltered bake against the installed retail DAT files.
- Open the result, validate every TOC entry, sample aliases across multiple
  landblocks, and run the existing live-vs-pak field equivalence suite.
- Record counts, dedup ratio, failures, elapsed time, peak working set, and
  artifact size in `docs/research/`.
- Delete the generated pak after validation unless it is intentionally retained
  outside the repository for the later Slice C cutover.

**Exit:** the full bake completes in practical time and disk space, all
equivalence and determinism gates are green, duplicate EnvCells demonstrably
share physical offsets, and no runtime code consumes the pak.

## Completion record

All checkpoints B1-B5 are complete. The full installed-DAT bake produced
751,141 addressable keys backed by 38,370 physical blobs with zero failures.
The 729,888 EnvCell keys resolve to 17,117 unique geometries and 712,771 aliases
(42.6×). The validated package is 28,192.4 MiB and publishes atomically in
81.4 seconds on the reference machine. A cross-landblock eight-key alias fixture
is byte-identical with one versus eight workers. Full evidence:
[`../research/2026-07-24-slice-b-full-bake-report.md`](../research/2026-07-24-slice-b-full-bake-report.md).

No runtime code reads the pak in this slice. Slice C remains blocked by the
physical-local Slice A baseline gate. Release build and the complete solution
suite pass (8,059 passed, five skipped, zero failed); generated pak artifacts
were deleted after validation.

## Verification

Run:

```powershell
dotnet test tests/AcDream.Content.Tests/AcDream.Content.Tests.csproj -c Release
dotnet test tests/AcDream.Bake.Tests/AcDream.Bake.Tests.csproj -c Release
dotnet build AcDream.slnx -c Release
dotnet test AcDream.slnx -c Release
```

Then run the full bake from the Release binary, capture its compact report, and
validate the output with `PakReader`. No visual gate is required because this
slice changes no runtime path.