docs(perf): close Slice B with full-bake evidence

This commit is contained in:
Erik 2026-07-24 14:10:50 +02:00
parent 90bf6bbf45
commit 49f3a48ee6
5 changed files with 196 additions and 20 deletions

View file

@ -1554,16 +1554,18 @@ port in any phase — no separate listing here.
### Track MP — Modern Pipeline (performance side track; user-commissioned 2026-07-05) ### Track MP — Modern Pipeline (performance side track; user-commissioned 2026-07-05)
> **⏸ PARKED 2026-07-05 (user decision) — WINS BANKED, REMAINDER DEFERRED TO THE FUTURE.** > **PREPARED-CONTENT WORK RESUMED 2026-07-24; ECS REMAINS PARKED.**
> The track's actual trigger — the FPS/ms counter *fluctuating wildly* (read by the > The track's actual trigger — the FPS/ms counter *fluctuating wildly* (read by the
> user as inefficiency) — is **RESOLVED**: MP0 (frame profiler), MP1a (Content > user as inefficiency) — is **RESOLVED**: MP0 (frame profiler), MP1a (Content
> extraction), and **MP-Alloc safe batch** shipped and merged to local main > extraction), and **MP-Alloc safe batch** shipped and merged to local main
> (`093cdb6d`, suite 4188 green, user-confirmed steady FPS). **ECS (MP3) + the > (`093cdb6d`, suite 4188 green, user-confirmed steady FPS). **ECS (MP3)
> rest are DEFERRED**: ECS is the *throughput* lever (raise avg FPS ~230→300), a > remains DEFERRED**: ECS is the *throughput* lever (raise avg FPS ~230→300), a
> DIFFERENT goal from steadiness — revisit ONLY if the user wants a higher raw FPS > DIFFERENT goal from steadiness — revisit ONLY if the user wants a higher raw FPS
> number and accepts the rewrite risk. Rust rejected. Parked (resume from the table > number and accepts the rewrite risk. Rust rejected. Parked (resume from the table
> below): MP1b pak/bake (needs the 865 GB EnvCell-dedup slice; loading-speed lever), > below): MP1b pak/bake is now complete; MP1c remains gated on corrected
> MP-Alloc hard sites, MP1c, MP2. Full rationale: `project_mp_track_findings.md`. > physical-local baselines. The remaining parked items are
> MP-Alloc hard sites and MP1c. MP2 has shipped. Full rationale:
> `project_mp_track_findings.md`.
> The 2026-07-18 portal/resource-lifetime repair does not resume MP's ECS, pak, > The 2026-07-18 portal/resource-lifetime repair does not resume MP's ECS, pak,
> or throughput redesign. It is corrective ownership/reclamation work for a > or throughput redesign. It is corrective ownership/reclamation work for a
> reproduced crash-class regression in the mandatory renderer, using the current > reproduced crash-class regression in the mandatory renderer, using the current
@ -1584,8 +1586,9 @@ port in any phase — no separate listing here.
> RDP captures have landed. The committed evidence includes capped/uncapped > RDP captures have landed. The committed evidence includes capped/uncapped
> nine-stop routes, dense Arwic, portal windows, observer-effect measurement, > nine-stop routes, dense Arwic, portal windows, observer-effect measurement,
> binary provenance, and contention stacks. The physical-local AMD-display gate > binary provenance, and contention stacks. The physical-local AMD-display gate
> remains open and blocks Slice C. Slice B EnvCell dedup is active under > remains open and blocks Slice C. Slice B EnvCell dedup is complete; its
> [`2026-07-24-modern-runtime-slice-b-envcell-dedup.md`](2026-07-24-modern-runtime-slice-b-envcell-dedup.md). > [full-bake report](../research/2026-07-24-slice-b-full-bake-report.md)
> records the 42.6× result and corrected geometry-identity collision.
**Spec:** `docs/superpowers/specs/2026-07-05-modern-pipeline-design.md` (the **Spec:** `docs/superpowers/specs/2026-07-05-modern-pipeline-design.md` (the
umbrella design — read it first). **Goal:** smoothness first (no frame over umbrella design — read it first). **Goal:** smoothness first (no frame over
@ -1607,10 +1610,10 @@ hitch and follows.
|---|---|---| |---|---|---|
| MP0 | Honest frame profiler (`[frame-prof]`) + baseline capture | ✅ SHIPPED 2026-07-05 — gate PASSED (`docs/research/2026-07-05-mp0-baseline.md`); headline: town spikes = GC churn (1.53 MB/frame), teleport hitch 211 ms | | MP0 | Honest frame profiler (`[frame-prof]`) + baseline capture | ✅ SHIPPED 2026-07-05 — gate PASSED (`docs/research/2026-07-05-mp0-baseline.md`); headline: town spikes = GC churn (1.53 MB/frame), teleport hitch 211 ms |
| MP1a | `AcDream.Content` extraction (GL-free MeshExtractor + boundary records out of App) | ✅ SHIPPED 2026-07-05 — user-gated (renders identical, zero tripwires, perf-neutral); 8 commits `651d041e`..`b0758d77` | | MP1a | `AcDream.Content` extraction (GL-free MeshExtractor + boundary records out of App) | ✅ SHIPPED 2026-07-05 — user-gated (renders identical, zero tripwires, perf-neutral); 8 commits `651d041e`..`b0758d77` |
| MP1b | Pak format + `acdream-bake` CLI + mmap zero-copy `PakReader` + equivalence test | 🔵 **ACTIVE 2026-07-24** — implementing content-identity EnvCell dedup, shared TOC aliases, atomic output, and a new full-bake gate; detailed plan: [`2026-07-24-modern-runtime-slice-b-envcell-dedup.md`](2026-07-24-modern-runtime-slice-b-envcell-dedup.md) | | MP1b | Pak format + `acdream-bake` CLI + mmap `PakReader` + equivalence/full-scale gate | ✅ **SHIPPED 2026-07-24** — 729,888 EnvCell keys → 17,117 unique geometries + 712,771 aliases (42.6×), 751,141 total keys / 38,370 physical blobs, zero failures, 28,192.4 MiB, 81.4 s validated atomic publish; full-DAT gate also corrected a real collision in WB's legacy geometry hash. [Report](../research/2026-07-24-slice-b-full-bake-report.md) |
| **MP-Alloc (safe batch)** | Reuse per-frame buffers: anim pose, particle draw-list, interior partition, animatedIds/drawableCells sets | ✅ SHIPPED + USER-GATED 2026-07-05 — dense-town frame-time spikes 2087ms → 610ms, alloc spikes (3075MB single-frame) eliminated, gen2 GC 511/window → ~0; user confirms FPS steady. 4 commits `b8c05e2b`..`91afea24` | | **MP-Alloc (safe batch)** | Reuse per-frame buffers: anim pose, particle draw-list, interior partition, animatedIds/drawableCells sets | ✅ SHIPPED + USER-GATED 2026-07-05 — dense-town frame-time spikes 2087ms → 610ms, alloc spikes (3075MB single-frame) eliminated, gen2 GC 511/window → ~0; user confirms FPS steady. 4 commits `b8c05e2b`..`91afea24` |
| MP-Alloc (hard sites) | EnvCell settled-camera rebuild gate + physics `Transition` pooling — the residual steady ~1.6MB/frame | ⚪ OPTIONAL follow-up — would lower the median too, but the wild-swing complaint is already resolved; each needs its own careful gate (batch correctness / physics faithfulness) | | MP-Alloc (hard sites) | EnvCell settled-camera rebuild gate + physics `Transition` pooling — the residual steady ~1.6MB/frame | ⚪ OPTIONAL follow-up — would lower the median too, but the wild-swing complaint is already resolved; each needs its own careful gate (batch correctness / physics faithfulness) |
| MP1c | Streaming cutover to pak + hitch gate (vs the 211 ms baseline) | ⚪ after MP-Alloc | | MP1c | Streaming cutover to pak + hitch gate | ⏸ blocked on the physical-local corrected Slice A baseline; Slice B package foundation is complete |
| MP2 | Retail particle distance/cell-view degradation: DAT-authored range, exact infinite/finite off-view update branches, emitter-first render scan, user-requested 2× default range | ✅ SHIPPED 2026-07-17 — Aerlinthe hotspot `0x32000223` resolves to retail 64m and defaults to 128m; landscape/entity streaming distance unchanged | | MP2 | Retail particle distance/cell-view degradation: DAT-authored range, exact infinite/finite off-view update branches, emitter-first render scan, user-requested 2× default range | ✅ SHIPPED 2026-07-17 — Aerlinthe hotspot `0x32000223` resolves to retail 64m and defaults to 128m; landscape/entity streaming distance unchanged |
| MP3 | Arch ECS render world + delta submission (the 300-FPS lever) | ⚪ — note: does NOT fix the steady-state GC churn (that's MP-Alloc); MP3 is the draw-submission throughput lever | | MP3 | Arch ECS render world + delta submission (the 300-FPS lever) | ⚪ — note: does NOT fix the steady-state GC churn (that's MP-Alloc); MP3 is the draw-submission throughput lever |
| MP4 | Zero-alloc frame loop + flat physics data (residual, post-MP-Alloc) | ⚪ hard-queued behind M1.5 #137 | | MP4 | Zero-alloc frame loop + flat physics data (residual, post-MP-Alloc) | ⚪ hard-queued behind M1.5 #137 |

View file

@ -2,8 +2,8 @@
**Date:** 2026-07-24 **Date:** 2026-07-24
**Status:** Slice A tooling and diagnostic RDP baselines landed; Slice B active. **Status:** Slice A tooling and diagnostic RDP baselines landed; Slice B
The physical-local Slice A gate still blocks Slice C. complete. The physical-local Slice A gate still blocks Slice C.
**Scope:** Reconcile and sequence the existing Modern Pipeline (`MP`) and **Scope:** Reconcile and sequence the existing Modern Pipeline (`MP`) and
Linux/headless (`LH`) tracks using the 2026-07-24 connected performance audit. Linux/headless (`LH`) tracks using the 2026-07-24 connected performance audit.
@ -578,7 +578,7 @@ effect, portal windows, and contention stacks are committed in
[`../research/2026-07-24-slice-a-rdp-baselines.md`](../research/2026-07-24-slice-a-rdp-baselines.md) [`../research/2026-07-24-slice-a-rdp-baselines.md`](../research/2026-07-24-slice-a-rdp-baselines.md)
and its machine-readable companion. They prove the tooling is usable but are and its machine-readable companion. They prove the tooling is usable but are
diagnostic-only. The physical-local AMD-display repetitions remain required, diagnostic-only. The physical-local AMD-display repetitions remain required,
so Slice C is still blocked. Slice B is baseline-independent and is now active. so Slice C is still blocked. Slice B was baseline-independent and is complete.
### Slice B — Finish MP1b EnvCell dedup and full bake ### Slice B — Finish MP1b EnvCell dedup and full bake
@ -599,6 +599,13 @@ so Slice C is still blocked. Slice B is baseline-independent and is now active.
sample equivalence tests pass; duplicate EnvCells share blob offsets; no runtime sample equivalence tests pass; duplicate EnvCells share blob offsets; no runtime
code consumes the pak yet. code consumes the pak yet.
**Completed 2026-07-24:** 729,888 EnvCell keys collapsed to 17,117 unique
geometries plus 712,771 aliases (42.6×), with 751,141 total keys, 38,370
physical blobs, zero failures, a 28,192.4 MiB artifact, and an 81.4-second
validated atomic publication. The gate also exposed and corrected a real
collision in WorldBuilder's legacy 31× geometry hash. Evidence:
[`../research/2026-07-24-slice-b-full-bake-report.md`](../research/2026-07-24-slice-b-full-bake-report.md).
### Slice C — Runtime prepared-asset source and MP1c cutover ### Slice C — Runtime prepared-asset source and MP1c cutover
**Purpose:** Remove mesh/texture decode from portal frames. **Purpose:** Remove mesh/texture decode from portal frames.
@ -1116,12 +1123,11 @@ an improvement" carries the same evidence burden as a regression claim.
## 14. Current execution point ## 14. Current execution point
Slice A's tooling and diagnostic RDP evidence are complete. Its physical-local Slice A's tooling and diagnostic RDP evidence are complete, and Slice B's
exit gate remains open. Execute **Slice B** now: it is baseline-independent, prepared-content bake is complete. The next implementation is Slice C, but its
completes an implementation already present in the tree, and directly attacks physical-local prerequisite remains open. Do not start Slice C until the
the benchmark's largest process-wide allocation source. Do not start Slice C capped/uncapped nine-stop and pinned dense-Arwic routes have been repeated on
until the capped/uncapped nine-stop and pinned dense-Arwic routes have been the physical local AMD display path and committed.
repeated on the physical local AMD display path and committed.
The intended order is therefore: The intended order is therefore:

View file

@ -1,7 +1,7 @@
# Modern Runtime Slice B — EnvCell geometry deduplication and full bake # Modern Runtime Slice B — EnvCell geometry deduplication and full bake
**Date:** 2026-07-24 **Date:** 2026-07-24
**Status:** Planned; implementation follows the Slice A diagnostic capture **Status:** Complete 2026-07-24; full installed-DAT gate passed
**Parent:** [`2026-07-24-modern-runtime-architecture.md`](2026-07-24-modern-runtime-architecture.md), Slice B **Parent:** [`2026-07-24-modern-runtime-architecture.md`](2026-07-24-modern-runtime-architecture.md), Slice B
**Prior implementation:** [`../superpowers/plans/2026-07-05-mp1b-pak-and-bake.md`](../superpowers/plans/2026-07-05-mp1b-pak-and-bake.md) **Prior implementation:** [`../superpowers/plans/2026-07-05-mp1b-pak-and-bake.md`](../superpowers/plans/2026-07-05-mp1b-pak-and-bake.md)
@ -9,7 +9,7 @@
Complete the existing MP1b bake without changing runtime behavior. Every Complete the existing MP1b bake without changing runtime behavior. Every
EnvCell file ID remains an independently addressable pak key, while cells with EnvCell file ID remains an independently addressable pak key, while cells with
the same WorldBuilder geometry identity share one serialized blob and one the same collision-resistant geometry identity share one serialized blob and one
physical byte range. The client does not consume the pak in this slice. physical byte range. The client does not consume the pak in this slice.
The full gate must replace the failed 865 GB / 52 minute per-cell bake with a The full gate must replace the failed 865 GB / 52 minute per-cell bake with a
@ -136,6 +136,21 @@ as a live unique-geometry extraction.
equivalence and determinism gates are green, duplicate EnvCells demonstrably equivalence and determinism gates are green, duplicate EnvCells demonstrably
share physical offsets, and no runtime code consumes the pak. share physical offsets, and no runtime code consumes the pak.
## Completion record
All checkpoints B1-B5 are complete. The full installed-DAT bake produced
751,141 addressable keys backed by 38,370 physical blobs with zero failures.
The 729,888 EnvCell keys resolve to 17,117 unique geometries and 712,771 aliases
(42.6×). The validated package is 28,192.4 MiB and publishes atomically in
81.4 seconds on the reference machine. A cross-landblock eight-key alias fixture
is byte-identical with one versus eight workers. Full evidence:
[`../research/2026-07-24-slice-b-full-bake-report.md`](../research/2026-07-24-slice-b-full-bake-report.md).
No runtime code reads the pak in this slice. Slice C remains blocked by the
physical-local Slice A baseline gate. Release build and the complete solution
suite pass (8,059 passed, five skipped, zero failed); generated pak artifacts
were deleted after validation.
## Verification ## Verification
Run: Run:

View file

@ -0,0 +1,76 @@
{
"schemaVersion": 1,
"capturedUtcDate": "2026-07-24",
"binaryCommit": "90bf6bbf455654dd1cfe1749b19bfd36b769939c",
"pakFormatVersion": 1,
"bakeToolVersion": 2,
"datIterations": {
"portal": 2072,
"cell": 982,
"highRes": 497,
"language": 994
},
"threads": 8,
"enumerated": {
"gfxObj": 15318,
"setup": 5935,
"envCell": 729888
},
"output": {
"gfxObjKeys": 15318,
"setupKeys": 5935,
"envCellKeys": 729888,
"uniqueEnvCellGeometries": 17117,
"envCellAliases": 712771,
"envCellDedupRatio": 42.64,
"sideStagedNewKeys": 0,
"sideStagedDuplicateKeys": 13284,
"physicalBlobs": 38370,
"totalKeys": 751141,
"failures": 0,
"bytes": 29561864760,
"mibibytes": 28192.4
},
"timing": {
"extractAndWriteSeconds": 81.2,
"validatedAndPublishedSeconds": 81.4
},
"memory": {
"peakWorkingSetMibibytes": 5073.7,
"peakPrivateMibibytes": 4507.3,
"note": "Offline bake-process peak while constructing and serializing 28.2 GiB of immutable payloads; this is not client runtime residency."
},
"validation": {
"headerAndDatIterations": "pass",
"tocCountOrderAndEveryRange": "pass",
"atomicOverwrite": "pass",
"temporaryFilesRemaining": 0,
"crossLandblockAliasSample": {
"sharedOffset": 16170873792,
"aliasCountAtReceipt": 25,
"sampleFileIds": [
"0x00A90230",
"0x00B901A8",
"0x00E50602",
"0x00E50610",
"0x00E5061F",
"0x00F90784",
"0x00F9078E",
"0x00FA073E"
]
},
"filteredThreadCountDeterminism": {
"threads": [1, 8],
"envCellKeys": 8,
"physicalBlobs": 1,
"sha256": "C8187761ED30CE9412C9081F52067159AECA5B02EB316E73DA29DC88D4339642"
},
"releaseBuild": "pass-with-17-pre-existing-test-warnings",
"completeReleaseSuite": {
"passed": 8059,
"skipped": 5,
"failed": 0
}
},
"artifactPolicy": "The generated pak is an untracked machine artifact and is deleted after final verification. This compact path-free report is the committed evidence."
}

View file

@ -0,0 +1,76 @@
# Slice B full-bake report
**Date:** 2026-07-24
**Measured binary:** `90bf6bbf455654dd1cfe1749b19bfd36b769939c`
**Result:** pass, zero extraction failures
Slice B replaces the failed 865 GB per-cell bake with a deterministic,
content-identity package. The client does not consume this package yet; this is
the offline prepared-content foundation for Slice C.
## Full installed-DAT result
| Measure | Result |
|---|---:|
| GfxObj keys | 15,318 |
| Setup keys | 5,935 |
| EnvCell keys | 729,888 |
| Unique EnvCell geometries | 17,117 |
| EnvCell aliases | 712,771 |
| EnvCell deduplication | 42.6× |
| Physical blobs | 38,370 |
| Total TOC keys | 751,141 |
| Failures | 0 |
| Extract + write | 81.2 seconds |
| Validate + atomic publish total | 81.4 seconds |
| Artifact size | 29,561,864,760 bytes (28,192.4 MiB) |
| Peak working set | 5,073.7 MiB |
| Peak private bytes | 4,507.3 MiB |
The memory figures belong to the short-lived offline bake process while it
constructs and serializes 28.2 GiB of immutable payloads. They are not expected
client residency: the later runtime source memory-maps the package and
deserializes only demanded assets. Sixteen-result extraction batches and
immediate rejection of already-written side-staged meshes keep live bake graphs
bounded; 13,284 redundant particle-preload results were discarded without
retention.
The header carries format version 1, bake-tool version 2, and source DAT
iterations portal 2072, cell 982, high-res 497, language 994. Validation opened
the completed temporary package, checked those values, strict TOC ordering,
all 751,141 physical ranges, and then published with a same-volume atomic
overwrite move. No temporary artifact remained.
## Identity correction discovered by the gate
The first guarded full catalog stopped on a real collision in WorldBuilders
31× polynomial:
- Cell `0x00030175`: environment `0x277`, structure 0, surfaces
`0x013B, 0x0034`.
- Cell `0x01BC0105`: environment `0x276`, structure 0, surfaces
`0x04FC, 0x0034`.
- Both legacy identities: `0x00000002020E8C13`.
Those cells have different vertices and polygons. Silently aliasing them would
render the wrong room. Runtime and bake now share a namespaced FNV-1a identity
over the complete typed tuple, while the catalog still compares full tuples and
fails loudly on any future collision. The legacy calculation remains as a
conformance test proving this exact installed-DAT case.
## Alias and determinism gates
A sampled physical receipt at offset `16170873792` is shared by 25 EnvCell keys
across multiple landblocks. Eight of those keys were baked independently with
one and eight workers. Each output contained eight keys backed by one physical
blob, and both files were byte-identical:
`C8187761ED30CE9412C9081F52067159AECA5B02EB316E73DA29DC88D4339642`
The generated full pak remains an untracked machine artifact and is deleted
after final verification. The machine-readable, path-free evidence is
[`2026-07-24-slice-b-full-bake-report.json`](2026-07-24-slice-b-full-bake-report.json).
Release build passed with 17 pre-existing test-source warnings. The complete
Release suite passed 8,059 tests with five skips and zero failures. The generated
full and filtered pak artifacts were then deleted as required.