docs(perf): close Slice B with full-bake evidence
This commit is contained in:
parent
90bf6bbf45
commit
49f3a48ee6
5 changed files with 196 additions and 20 deletions
|
|
@ -1554,16 +1554,18 @@ port in any phase — no separate listing here.
|
|||
|
||||
### Track MP — Modern Pipeline (performance side track; user-commissioned 2026-07-05)
|
||||
|
||||
> **⏸ PARKED 2026-07-05 (user decision) — WINS BANKED, REMAINDER DEFERRED TO THE FUTURE.**
|
||||
> **PREPARED-CONTENT WORK RESUMED 2026-07-24; ECS REMAINS PARKED.**
|
||||
> The track's actual trigger — the FPS/ms counter *fluctuating wildly* (read by the
|
||||
> user as inefficiency) — is **RESOLVED**: MP0 (frame profiler), MP1a (Content
|
||||
> extraction), and **MP-Alloc safe batch** shipped and merged to local main
|
||||
> (`093cdb6d`, suite 4188 green, user-confirmed steady FPS). **ECS (MP3) + the
|
||||
> rest are DEFERRED**: ECS is the *throughput* lever (raise avg FPS ~230→300), a
|
||||
> (`093cdb6d`, suite 4188 green, user-confirmed steady FPS). **ECS (MP3)
|
||||
> remains DEFERRED**: ECS is the *throughput* lever (raise avg FPS ~230→300), a
|
||||
> DIFFERENT goal from steadiness — revisit ONLY if the user wants a higher raw FPS
|
||||
> number and accepts the rewrite risk. Rust rejected. Parked (resume from the table
|
||||
> below): MP1b pak/bake (needs the 865 GB EnvCell-dedup slice; loading-speed lever),
|
||||
> MP-Alloc hard sites, MP1c, MP2. Full rationale: `project_mp_track_findings.md`.
|
||||
> below): MP1b pak/bake is now complete; MP1c remains gated on corrected
|
||||
> physical-local baselines. The remaining parked items are
|
||||
> MP-Alloc hard sites and MP1c. MP2 has shipped. Full rationale:
|
||||
> `project_mp_track_findings.md`.
|
||||
> The 2026-07-18 portal/resource-lifetime repair does not resume MP's ECS, pak,
|
||||
> or throughput redesign. It is corrective ownership/reclamation work for a
|
||||
> reproduced crash-class regression in the mandatory renderer, using the current
|
||||
|
|
@ -1584,8 +1586,9 @@ port in any phase — no separate listing here.
|
|||
> RDP captures have landed. The committed evidence includes capped/uncapped
|
||||
> nine-stop routes, dense Arwic, portal windows, observer-effect measurement,
|
||||
> binary provenance, and contention stacks. The physical-local AMD-display gate
|
||||
> remains open and blocks Slice C. Slice B EnvCell dedup is active under
|
||||
> [`2026-07-24-modern-runtime-slice-b-envcell-dedup.md`](2026-07-24-modern-runtime-slice-b-envcell-dedup.md).
|
||||
> remains open and blocks Slice C. Slice B EnvCell dedup is complete; its
|
||||
> [full-bake report](../research/2026-07-24-slice-b-full-bake-report.md)
|
||||
> records the 42.6× result and corrected geometry-identity collision.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-07-05-modern-pipeline-design.md` (the
|
||||
umbrella design — read it first). **Goal:** smoothness first (no frame over
|
||||
|
|
@ -1607,10 +1610,10 @@ hitch and follows.
|
|||
|---|---|---|
|
||||
| MP0 | Honest frame profiler (`[frame-prof]`) + baseline capture | ✅ SHIPPED 2026-07-05 — gate PASSED (`docs/research/2026-07-05-mp0-baseline.md`); headline: town spikes = GC churn (1.5–3 MB/frame), teleport hitch 211 ms |
|
||||
| MP1a | `AcDream.Content` extraction (GL-free MeshExtractor + boundary records out of App) | ✅ SHIPPED 2026-07-05 — user-gated (renders identical, zero tripwires, perf-neutral); 8 commits `651d041e`..`b0758d77` |
|
||||
| MP1b | Pak format + `acdream-bake` CLI + mmap zero-copy `PakReader` + equivalence test | 🔵 **ACTIVE 2026-07-24** — implementing content-identity EnvCell dedup, shared TOC aliases, atomic output, and a new full-bake gate; detailed plan: [`2026-07-24-modern-runtime-slice-b-envcell-dedup.md`](2026-07-24-modern-runtime-slice-b-envcell-dedup.md) |
|
||||
| MP1b | Pak format + `acdream-bake` CLI + mmap `PakReader` + equivalence/full-scale gate | ✅ **SHIPPED 2026-07-24** — 729,888 EnvCell keys → 17,117 unique geometries + 712,771 aliases (42.6×), 751,141 total keys / 38,370 physical blobs, zero failures, 28,192.4 MiB, 81.4 s validated atomic publish; full-DAT gate also corrected a real collision in WB's legacy geometry hash. [Report](../research/2026-07-24-slice-b-full-bake-report.md) |
|
||||
| **MP-Alloc (safe batch)** | Reuse per-frame buffers: anim pose, particle draw-list, interior partition, animatedIds/drawableCells sets | ✅ SHIPPED + USER-GATED 2026-07-05 — dense-town frame-time spikes 20–87ms → 6–10ms, alloc spikes (30–75MB single-frame) eliminated, gen2 GC 5–11/window → ~0; user confirms FPS steady. 4 commits `b8c05e2b`..`91afea24` |
|
||||
| MP-Alloc (hard sites) | EnvCell settled-camera rebuild gate + physics `Transition` pooling — the residual steady ~1.6MB/frame | ⚪ OPTIONAL follow-up — would lower the median too, but the wild-swing complaint is already resolved; each needs its own careful gate (batch correctness / physics faithfulness) |
|
||||
| MP1c | Streaming cutover to pak + hitch gate (vs the 211 ms baseline) | ⚪ after MP-Alloc |
|
||||
| MP1c | Streaming cutover to pak + hitch gate | ⏸ blocked on the physical-local corrected Slice A baseline; Slice B package foundation is complete |
|
||||
| MP2 | Retail particle distance/cell-view degradation: DAT-authored range, exact infinite/finite off-view update branches, emitter-first render scan, user-requested 2× default range | ✅ SHIPPED 2026-07-17 — Aerlinthe hotspot `0x32000223` resolves to retail 64m and defaults to 128m; landscape/entity streaming distance unchanged |
|
||||
| MP3 | Arch ECS render world + delta submission (the 300-FPS lever) | ⚪ — note: does NOT fix the steady-state GC churn (that's MP-Alloc); MP3 is the draw-submission throughput lever |
|
||||
| MP4 | Zero-alloc frame loop + flat physics data (residual, post-MP-Alloc) | ⚪ hard-queued behind M1.5 #137 |
|
||||
|
|
|
|||
|
|
@ -2,8 +2,8 @@
|
|||
|
||||
**Date:** 2026-07-24
|
||||
|
||||
**Status:** Slice A tooling and diagnostic RDP baselines landed; Slice B active.
|
||||
The physical-local Slice A gate still blocks Slice C.
|
||||
**Status:** Slice A tooling and diagnostic RDP baselines landed; Slice B
|
||||
complete. The physical-local Slice A gate still blocks Slice C.
|
||||
|
||||
**Scope:** Reconcile and sequence the existing Modern Pipeline (`MP`) and
|
||||
Linux/headless (`LH`) tracks using the 2026-07-24 connected performance audit.
|
||||
|
|
@ -578,7 +578,7 @@ effect, portal windows, and contention stacks are committed in
|
|||
[`../research/2026-07-24-slice-a-rdp-baselines.md`](../research/2026-07-24-slice-a-rdp-baselines.md)
|
||||
and its machine-readable companion. They prove the tooling is usable but are
|
||||
diagnostic-only. The physical-local AMD-display repetitions remain required,
|
||||
so Slice C is still blocked. Slice B is baseline-independent and is now active.
|
||||
so Slice C is still blocked. Slice B was baseline-independent and is complete.
|
||||
|
||||
### Slice B — Finish MP1b EnvCell dedup and full bake
|
||||
|
||||
|
|
@ -599,6 +599,13 @@ so Slice C is still blocked. Slice B is baseline-independent and is now active.
|
|||
sample equivalence tests pass; duplicate EnvCells share blob offsets; no runtime
|
||||
code consumes the pak yet.
|
||||
|
||||
**Completed 2026-07-24:** 729,888 EnvCell keys collapsed to 17,117 unique
|
||||
geometries plus 712,771 aliases (42.6×), with 751,141 total keys, 38,370
|
||||
physical blobs, zero failures, a 28,192.4 MiB artifact, and an 81.4-second
|
||||
validated atomic publication. The gate also exposed and corrected a real
|
||||
collision in WorldBuilder's legacy 31× geometry hash. Evidence:
|
||||
[`../research/2026-07-24-slice-b-full-bake-report.md`](../research/2026-07-24-slice-b-full-bake-report.md).
|
||||
|
||||
### Slice C — Runtime prepared-asset source and MP1c cutover
|
||||
|
||||
**Purpose:** Remove mesh/texture decode from portal frames.
|
||||
|
|
@ -1116,12 +1123,11 @@ an improvement" carries the same evidence burden as a regression claim.
|
|||
|
||||
## 14. Current execution point
|
||||
|
||||
Slice A's tooling and diagnostic RDP evidence are complete. Its physical-local
|
||||
exit gate remains open. Execute **Slice B** now: it is baseline-independent,
|
||||
completes an implementation already present in the tree, and directly attacks
|
||||
the benchmark's largest process-wide allocation source. Do not start Slice C
|
||||
until the capped/uncapped nine-stop and pinned dense-Arwic routes have been
|
||||
repeated on the physical local AMD display path and committed.
|
||||
Slice A's tooling and diagnostic RDP evidence are complete, and Slice B's
|
||||
prepared-content bake is complete. The next implementation is Slice C, but its
|
||||
physical-local prerequisite remains open. Do not start Slice C until the
|
||||
capped/uncapped nine-stop and pinned dense-Arwic routes have been repeated on
|
||||
the physical local AMD display path and committed.
|
||||
|
||||
The intended order is therefore:
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
# Modern Runtime Slice B — EnvCell geometry deduplication and full bake
|
||||
|
||||
**Date:** 2026-07-24
|
||||
**Status:** Planned; implementation follows the Slice A diagnostic capture
|
||||
**Status:** Complete 2026-07-24; full installed-DAT gate passed
|
||||
**Parent:** [`2026-07-24-modern-runtime-architecture.md`](2026-07-24-modern-runtime-architecture.md), Slice B
|
||||
**Prior implementation:** [`../superpowers/plans/2026-07-05-mp1b-pak-and-bake.md`](../superpowers/plans/2026-07-05-mp1b-pak-and-bake.md)
|
||||
|
||||
|
|
@ -9,7 +9,7 @@
|
|||
|
||||
Complete the existing MP1b bake without changing runtime behavior. Every
|
||||
EnvCell file ID remains an independently addressable pak key, while cells with
|
||||
the same WorldBuilder geometry identity share one serialized blob and one
|
||||
the same collision-resistant geometry identity share one serialized blob and one
|
||||
physical byte range. The client does not consume the pak in this slice.
|
||||
|
||||
The full gate must replace the failed 865 GB / 52 minute per-cell bake with a
|
||||
|
|
@ -136,6 +136,21 @@ as a live unique-geometry extraction.
|
|||
equivalence and determinism gates are green, duplicate EnvCells demonstrably
|
||||
share physical offsets, and no runtime code consumes the pak.
|
||||
|
||||
## Completion record
|
||||
|
||||
All checkpoints B1-B5 are complete. The full installed-DAT bake produced
|
||||
751,141 addressable keys backed by 38,370 physical blobs with zero failures.
|
||||
The 729,888 EnvCell keys resolve to 17,117 unique geometries and 712,771 aliases
|
||||
(42.6×). The validated package is 28,192.4 MiB and publishes atomically in
|
||||
81.4 seconds on the reference machine. A cross-landblock eight-key alias fixture
|
||||
is byte-identical with one versus eight workers. Full evidence:
|
||||
[`../research/2026-07-24-slice-b-full-bake-report.md`](../research/2026-07-24-slice-b-full-bake-report.md).
|
||||
|
||||
No runtime code reads the pak in this slice. Slice C remains blocked by the
|
||||
physical-local Slice A baseline gate. Release build and the complete solution
|
||||
suite pass (8,059 passed, five skipped, zero failed); generated pak artifacts
|
||||
were deleted after validation.
|
||||
|
||||
## Verification
|
||||
|
||||
Run:
|
||||
|
|
|
|||
76
docs/research/2026-07-24-slice-b-full-bake-report.json
Normal file
76
docs/research/2026-07-24-slice-b-full-bake-report.json
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
{
|
||||
"schemaVersion": 1,
|
||||
"capturedUtcDate": "2026-07-24",
|
||||
"binaryCommit": "90bf6bbf455654dd1cfe1749b19bfd36b769939c",
|
||||
"pakFormatVersion": 1,
|
||||
"bakeToolVersion": 2,
|
||||
"datIterations": {
|
||||
"portal": 2072,
|
||||
"cell": 982,
|
||||
"highRes": 497,
|
||||
"language": 994
|
||||
},
|
||||
"threads": 8,
|
||||
"enumerated": {
|
||||
"gfxObj": 15318,
|
||||
"setup": 5935,
|
||||
"envCell": 729888
|
||||
},
|
||||
"output": {
|
||||
"gfxObjKeys": 15318,
|
||||
"setupKeys": 5935,
|
||||
"envCellKeys": 729888,
|
||||
"uniqueEnvCellGeometries": 17117,
|
||||
"envCellAliases": 712771,
|
||||
"envCellDedupRatio": 42.64,
|
||||
"sideStagedNewKeys": 0,
|
||||
"sideStagedDuplicateKeys": 13284,
|
||||
"physicalBlobs": 38370,
|
||||
"totalKeys": 751141,
|
||||
"failures": 0,
|
||||
"bytes": 29561864760,
|
||||
"mibibytes": 28192.4
|
||||
},
|
||||
"timing": {
|
||||
"extractAndWriteSeconds": 81.2,
|
||||
"validatedAndPublishedSeconds": 81.4
|
||||
},
|
||||
"memory": {
|
||||
"peakWorkingSetMibibytes": 5073.7,
|
||||
"peakPrivateMibibytes": 4507.3,
|
||||
"note": "Offline bake-process peak while constructing and serializing 28.2 GiB of immutable payloads; this is not client runtime residency."
|
||||
},
|
||||
"validation": {
|
||||
"headerAndDatIterations": "pass",
|
||||
"tocCountOrderAndEveryRange": "pass",
|
||||
"atomicOverwrite": "pass",
|
||||
"temporaryFilesRemaining": 0,
|
||||
"crossLandblockAliasSample": {
|
||||
"sharedOffset": 16170873792,
|
||||
"aliasCountAtReceipt": 25,
|
||||
"sampleFileIds": [
|
||||
"0x00A90230",
|
||||
"0x00B901A8",
|
||||
"0x00E50602",
|
||||
"0x00E50610",
|
||||
"0x00E5061F",
|
||||
"0x00F90784",
|
||||
"0x00F9078E",
|
||||
"0x00FA073E"
|
||||
]
|
||||
},
|
||||
"filteredThreadCountDeterminism": {
|
||||
"threads": [1, 8],
|
||||
"envCellKeys": 8,
|
||||
"physicalBlobs": 1,
|
||||
"sha256": "C8187761ED30CE9412C9081F52067159AECA5B02EB316E73DA29DC88D4339642"
|
||||
},
|
||||
"releaseBuild": "pass-with-17-pre-existing-test-warnings",
|
||||
"completeReleaseSuite": {
|
||||
"passed": 8059,
|
||||
"skipped": 5,
|
||||
"failed": 0
|
||||
}
|
||||
},
|
||||
"artifactPolicy": "The generated pak is an untracked machine artifact and is deleted after final verification. This compact path-free report is the committed evidence."
|
||||
}
|
||||
76
docs/research/2026-07-24-slice-b-full-bake-report.md
Normal file
76
docs/research/2026-07-24-slice-b-full-bake-report.md
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
# Slice B full-bake report
|
||||
|
||||
**Date:** 2026-07-24
|
||||
**Measured binary:** `90bf6bbf455654dd1cfe1749b19bfd36b769939c`
|
||||
**Result:** pass, zero extraction failures
|
||||
|
||||
Slice B replaces the failed 865 GB per-cell bake with a deterministic,
|
||||
content-identity package. The client does not consume this package yet; this is
|
||||
the offline prepared-content foundation for Slice C.
|
||||
|
||||
## Full installed-DAT result
|
||||
|
||||
| Measure | Result |
|
||||
|---|---:|
|
||||
| GfxObj keys | 15,318 |
|
||||
| Setup keys | 5,935 |
|
||||
| EnvCell keys | 729,888 |
|
||||
| Unique EnvCell geometries | 17,117 |
|
||||
| EnvCell aliases | 712,771 |
|
||||
| EnvCell deduplication | 42.6× |
|
||||
| Physical blobs | 38,370 |
|
||||
| Total TOC keys | 751,141 |
|
||||
| Failures | 0 |
|
||||
| Extract + write | 81.2 seconds |
|
||||
| Validate + atomic publish total | 81.4 seconds |
|
||||
| Artifact size | 29,561,864,760 bytes (28,192.4 MiB) |
|
||||
| Peak working set | 5,073.7 MiB |
|
||||
| Peak private bytes | 4,507.3 MiB |
|
||||
|
||||
The memory figures belong to the short-lived offline bake process while it
|
||||
constructs and serializes 28.2 GiB of immutable payloads. They are not expected
|
||||
client residency: the later runtime source memory-maps the package and
|
||||
deserializes only demanded assets. Sixteen-result extraction batches and
|
||||
immediate rejection of already-written side-staged meshes keep live bake graphs
|
||||
bounded; 13,284 redundant particle-preload results were discarded without
|
||||
retention.
|
||||
|
||||
The header carries format version 1, bake-tool version 2, and source DAT
|
||||
iterations portal 2072, cell 982, high-res 497, language 994. Validation opened
|
||||
the completed temporary package, checked those values, strict TOC ordering,
|
||||
all 751,141 physical ranges, and then published with a same-volume atomic
|
||||
overwrite move. No temporary artifact remained.
|
||||
|
||||
## Identity correction discovered by the gate
|
||||
|
||||
The first guarded full catalog stopped on a real collision in WorldBuilder’s
|
||||
31× polynomial:
|
||||
|
||||
- Cell `0x00030175`: environment `0x277`, structure 0, surfaces
|
||||
`0x013B, 0x0034`.
|
||||
- Cell `0x01BC0105`: environment `0x276`, structure 0, surfaces
|
||||
`0x04FC, 0x0034`.
|
||||
- Both legacy identities: `0x00000002020E8C13`.
|
||||
|
||||
Those cells have different vertices and polygons. Silently aliasing them would
|
||||
render the wrong room. Runtime and bake now share a namespaced FNV-1a identity
|
||||
over the complete typed tuple, while the catalog still compares full tuples and
|
||||
fails loudly on any future collision. The legacy calculation remains as a
|
||||
conformance test proving this exact installed-DAT case.
|
||||
|
||||
## Alias and determinism gates
|
||||
|
||||
A sampled physical receipt at offset `16170873792` is shared by 25 EnvCell keys
|
||||
across multiple landblocks. Eight of those keys were baked independently with
|
||||
one and eight workers. Each output contained eight keys backed by one physical
|
||||
blob, and both files were byte-identical:
|
||||
|
||||
`C8187761ED30CE9412C9081F52067159AECA5B02EB316E73DA29DC88D4339642`
|
||||
|
||||
The generated full pak remains an untracked machine artifact and is deleted
|
||||
after final verification. The machine-readable, path-free evidence is
|
||||
[`2026-07-24-slice-b-full-bake-report.json`](2026-07-24-slice-b-full-bake-report.json).
|
||||
|
||||
Release build passed with 17 pre-existing test-source warnings. The complete
|
||||
Release suite passed 8,059 tests with five skips and zero failures. The generated
|
||||
full and filtered pak artifacts were then deleted as required.
|
||||
Loading…
Add table
Add a link
Reference in a new issue