diff --git a/docs/plans/2026-04-11-roadmap.md b/docs/plans/2026-04-11-roadmap.md index 0cdfe39c..25e48014 100644 --- a/docs/plans/2026-04-11-roadmap.md +++ b/docs/plans/2026-04-11-roadmap.md @@ -1580,7 +1580,12 @@ port in any phase — no separate listing here. > unifies the unfinished MP pak/render-world work with future Track LH, preserves > retail-shaped simulation and the N.5 renderer, and reorders execution around > prepared content and cost-budgeted streaming before render-ECS throughput. -> This records the plan only; implementation remains not started. +> **Execution update 2026-07-24:** Slice A's corrected tooling and diagnostic +> RDP captures have landed. The committed evidence includes capped/uncapped +> nine-stop routes, dense Arwic, portal windows, observer-effect measurement, +> binary provenance, and contention stacks. The physical-local AMD-display gate +> remains open and blocks Slice C. Slice B EnvCell dedup is active under +> [`2026-07-24-modern-runtime-slice-b-envcell-dedup.md`](2026-07-24-modern-runtime-slice-b-envcell-dedup.md). **Spec:** `docs/superpowers/specs/2026-07-05-modern-pipeline-design.md` (the umbrella design — read it first). **Goal:** smoothness first (no frame over @@ -1602,7 +1607,7 @@ hitch and follows. |---|---|---| | MP0 | Honest frame profiler (`[frame-prof]`) + baseline capture | ✅ SHIPPED 2026-07-05 — gate PASSED (`docs/research/2026-07-05-mp0-baseline.md`); headline: town spikes = GC churn (1.5–3 MB/frame), teleport hitch 211 ms | | MP1a | `AcDream.Content` extraction (GL-free MeshExtractor + boundary records out of App) | ✅ SHIPPED 2026-07-05 — user-gated (renders identical, zero tripwires, perf-neutral); 8 commits `651d041e`..`b0758d77` | -| MP1b | Pak format + `acdream-bake` CLI + mmap zero-copy `PakReader` + equivalence test | 🔵 code + review-fixes done; **full-bake gate found 865 GB → EnvCell dedup slice REQUIRED before shipped** (spec §6.6) | +| MP1b | Pak format + `acdream-bake` CLI + mmap zero-copy `PakReader` + equivalence test | 🔵 **ACTIVE 2026-07-24** — implementing content-identity EnvCell dedup, shared TOC aliases, atomic output, and a new full-bake gate; detailed plan: [`2026-07-24-modern-runtime-slice-b-envcell-dedup.md`](2026-07-24-modern-runtime-slice-b-envcell-dedup.md) | | **MP-Alloc (safe batch)** | Reuse per-frame buffers: anim pose, particle draw-list, interior partition, animatedIds/drawableCells sets | ✅ SHIPPED + USER-GATED 2026-07-05 — dense-town frame-time spikes 20–87ms → 6–10ms, alloc spikes (30–75MB single-frame) eliminated, gen2 GC 5–11/window → ~0; user confirms FPS steady. 4 commits `b8c05e2b`..`91afea24` | | MP-Alloc (hard sites) | EnvCell settled-camera rebuild gate + physics `Transition` pooling — the residual steady ~1.6MB/frame | ⚪ OPTIONAL follow-up — would lower the median too, but the wild-swing complaint is already resolved; each needs its own careful gate (batch correctness / physics faithfulness) | | MP1c | Streaming cutover to pak + hitch gate (vs the 211 ms baseline) | ⚪ after MP-Alloc | diff --git a/docs/plans/2026-07-24-modern-runtime-architecture.md b/docs/plans/2026-07-24-modern-runtime-architecture.md index 56140d93..acddb516 100644 --- a/docs/plans/2026-07-24-modern-runtime-architecture.md +++ b/docs/plans/2026-07-24-modern-runtime-architecture.md @@ -2,7 +2,8 @@ **Date:** 2026-07-24 -**Status:** Planning complete; implementation not started +**Status:** Slice A tooling and diagnostic RDP baselines landed; Slice B active. +The physical-local Slice A gate still blocks Slice C. **Scope:** Reconcile and sequence the existing Modern Pipeline (`MP`) and Linux/headless (`LH`) tracks using the 2026-07-24 connected performance audit. @@ -569,6 +570,16 @@ capped-local nine-stop baselines, a pinned dense-town uncapped baseline (name the town, pin the camera), and per-portal allocation/frame timelines around each teleport materialization. +**Progress (2026-07-24):** the corrected GL timer, per-frame history, process +counter/trace capture, screenshot comparison, reference configuration, binary +provenance, and route-bound stationary sampling have landed. Corrected capped +and uncapped nine-stop RDP routes, a pinned dense-Arwic RDP sample, observer +effect, portal windows, and contention stacks are committed in +[`../research/2026-07-24-slice-a-rdp-baselines.md`](../research/2026-07-24-slice-a-rdp-baselines.md) +and its machine-readable companion. They prove the tooling is usable but are +diagnostic-only. The physical-local AMD-display repetitions remain required, +so Slice C is still blocked. Slice B is baseline-independent and is now active. + ### Slice B — Finish MP1b EnvCell dedup and full bake **Purpose:** Make the existing pak physically usable. @@ -1103,15 +1114,14 @@ an improvement" carries the same evidence burden as a regression claim. --- -## 14. Recommended starting point +## 14. Current execution point -Begin with **Slice A**, then **Slice B**. - -Slice A is small and makes all later GPU and frame gates honest. Slice B completes -an implementation already present in the tree and directly attacks the -benchmark's largest allocation source. The first meaningful runtime cutover is -Slice C; do not begin the render-scene migration before the portal content storm -has been removed and measured. +Slice A's tooling and diagnostic RDP evidence are complete. Its physical-local +exit gate remains open. Execute **Slice B** now: it is baseline-independent, +completes an implementation already present in the tree, and directly attacks +the benchmark's largest process-wide allocation source. Do not start Slice C +until the capped/uncapped nine-stop and pinned dense-Arwic routes have been +repeated on the physical local AMD display path and committed. The intended order is therefore: diff --git a/docs/research/2026-07-24-slice-a-rdp-baselines.json b/docs/research/2026-07-24-slice-a-rdp-baselines.json new file mode 100644 index 00000000..a46c7572 --- /dev/null +++ b/docs/research/2026-07-24-slice-a-rdp-baselines.json @@ -0,0 +1,158 @@ +{ + "schemaVersion": 1, + "capturedUtcDate": "2026-07-24", + "authority": "diagnostic-rdp-only", + "purpose": "Corrected Slice A tooling validation and directional baseline. This does not satisfy the physical-local exit criterion.", + "referenceConfig": "docs/research/2026-07-24-performance-reference-config.md", + "rawArtifactPolicy": "Raw CSV, EventPipe, screenshot, and process logs remain local and uncommitted. This file contains the sanitized reproducible summary.", + "captures": [ + { + "name": "capped-nine-stop", + "binaryCommit": "2c874b09771cfd0062b4e23083ee13142761fe80", + "sourceCommit": "2c874b09771cfd0062b4e23083ee13142761fe80", + "binaryMatchesSource": true, + "route": "Caul, Sawato, Rynthid, Aerlinthe, Sawato, Holtburg, Caul, Sawato, Caul", + "result": "pass-graceful", + "elapsedSeconds": 560.170, + "measuredSeconds": 549.427, + "frames": 17400, + "frameCpuUs": { "p50": 31240, "p95": 31868, "p99": 33032, "p999": 136090, "max": 396953 }, + "gpuUs": { "p50": 3626, "p95": 5826, "p99": 6807, "p999": 7654, "max": 389392 }, + "frameThreadAllocationBytes": { "p50": 31552, "p95": 159944, "p99": 2835704, "p999": 13780848, "max": 212677752 }, + "updateUs": { "p50": 1071, "p95": 1962, "p99": 4079, "max": 247943 }, + "process": { + "allocationBytesPerSecondMean": 36609676, + "allocationBytesPerSecondMax": 2754634720, + "gcPauseSecondsTotal": 2.623917, + "gcPauseSecondsMax": 0.270281, + "workingSetBytesMax": 1642217472, + "gen0Collections": 138, + "gen1Collections": 132, + "gen2Collections": 70, + "contentions": 5994 + }, + "portalWindows": [ + { "destination": "Caul", "cpuP99Us": 76758, "cpuMaxUs": 110974, "gpuP99Us": 7365, "allocationMaxBytes": 56550848, "updateP99Us": 47519, "uploadP99Us": 3637 }, + { "destination": "Sawato", "cpuP99Us": 99077, "cpuMaxUs": 396953, "gpuP99Us": 6957, "allocationMaxBytes": 212677752, "updateP99Us": 19542, "uploadP99Us": 4402 }, + { "destination": "Rynthid", "cpuP99Us": 55614, "cpuMaxUs": 308664, "gpuP99Us": 3118, "allocationMaxBytes": 15018512, "updateP99Us": 22441, "uploadP99Us": 2502 }, + { "destination": "Aerlinthe", "cpuP99Us": 47918, "cpuMaxUs": 259394, "gpuP99Us": 6205, "allocationMaxBytes": 13780848, "updateP99Us": 13808, "uploadP99Us": 553 }, + { "destination": "Sawato-revisit", "cpuP99Us": 64704, "cpuMaxUs": 325502, "gpuP99Us": 5356, "allocationMaxBytes": 145024240, "updateP99Us": 34717, "uploadP99Us": 907 }, + { "destination": "Holtburg", "cpuP99Us": 48965, "cpuMaxUs": 83478, "gpuP99Us": 5525, "allocationMaxBytes": 16950536, "updateP99Us": 19088, "uploadP99Us": 5231 }, + { "destination": "Caul-return", "cpuP99Us": 58393, "cpuMaxUs": 90350, "gpuP99Us": 6642, "allocationMaxBytes": 35985776, "updateP99Us": 29162, "uploadP99Us": 1333 }, + { "destination": "Sawato-plateau", "cpuP99Us": 76006, "cpuMaxUs": 312187, "gpuP99Us": 6454, "allocationMaxBytes": 144965224, "updateP99Us": 18399, "uploadP99Us": 2060 }, + { "destination": "Caul-plateau", "cpuP99Us": 48700, "cpuMaxUs": 87818, "gpuP99Us": 6788, "allocationMaxBytes": 35966704, "updateP99Us": 23387, "uploadP99Us": 329 } + ] + }, + { + "name": "uncapped-nine-stop", + "binaryCommit": "4285f1db", + "sourceCommit": "4285f1db", + "binaryMatchesSource": true, + "route": "Caul, Sawato, Rynthid, Aerlinthe, Sawato, Holtburg, Caul, Sawato, Caul", + "result": "pass-graceful", + "elapsedSeconds": 512.792, + "measuredSeconds": 501.819, + "frames": 229412, + "frameCpuUs": { "p50": 1537, "p95": 6077, "p99": 8938, "p999": 16157, "max": 439275 }, + "gpuUs": { "p50": 935, "p95": 3288, "p99": 3470, "max": 357565 }, + "frameThreadAllocationBytes": { "p50": 28968, "p95": 71136, "p99": 97296, "max": 216373536 }, + "updateUs": { "p50": 177, "p95": 777, "p99": 1277, "max": 437906 }, + "process": { + "allocationBytesPerSecondMean": 48457217, + "allocationBytesPerSecondMax": 2167972952, + "gcPauseSecondsTotal": 2.656187, + "gcPauseSecondsMax": 0.259869, + "workingSetBytesMax": 921333760, + "gen0Collections": 279, + "gen1Collections": 136, + "gen2Collections": 80, + "contentions": 5493 + }, + "portalWindows": [ + { "destination": "Caul", "cpuP99Us": 15768, "cpuMaxUs": 40775, "gpuP99Us": 3521, "allocationMaxBytes": 516296, "updateP99Us": 2112, "uploadP99Us": 61 }, + { "destination": "Sawato", "cpuP99Us": 9596, "cpuMaxUs": 361556, "gpuP99Us": 3339, "allocationMaxBytes": 216373536, "updateP99Us": 1360, "uploadP99Us": 71 }, + { "destination": "Rynthid", "cpuP99Us": 2753, "cpuMaxUs": 325975, "gpuP99Us": 1259, "allocationMaxBytes": 13296880, "updateP99Us": 735, "uploadP99Us": 56 }, + { "destination": "Aerlinthe", "cpuP99Us": 2351, "cpuMaxUs": 255745, "gpuP99Us": 1279, "allocationMaxBytes": 13790552, "updateP99Us": 431, "uploadP99Us": 17 }, + { "destination": "Sawato-revisit", "cpuP99Us": 3466, "cpuMaxUs": 322834, "gpuP99Us": 2177, "allocationMaxBytes": 146032504, "updateP99Us": 487, "uploadP99Us": 22 }, + { "destination": "Holtburg", "cpuP99Us": 2448, "cpuMaxUs": 291110, "gpuP99Us": 1104, "allocationMaxBytes": 15978352, "updateP99Us": 618, "uploadP99Us": 104 }, + { "destination": "Caul-return", "cpuP99Us": 6763, "cpuMaxUs": 292269, "gpuP99Us": 2344, "allocationMaxBytes": 36035840, "updateP99Us": 1072, "uploadP99Us": 70 }, + { "destination": "Sawato-plateau", "cpuP99Us": 9682, "cpuMaxUs": 439275, "gpuP99Us": 3371, "allocationMaxBytes": 213092152, "updateP99Us": 1407, "uploadP99Us": 67 }, + { "destination": "Caul-plateau", "cpuP99Us": 2185, "cpuMaxUs": 319036, "gpuP99Us": 1138, "allocationMaxBytes": 36032184, "updateP99Us": 908, "uploadP99Us": 47 } + ] + }, + { + "name": "uncapped-dense-arwic", + "binaryCommit": "24f898c54d7c6547555a4402543b6a3cd7ced7f6", + "sourceCommit": "24f898c54d7c6547555a4402543b6a3cd7ced7f6", + "binaryMatchesSource": true, + "result": "pass-graceful", + "elapsedSeconds": 66.861, + "measuredSeconds": 56.752, + "frames": 16350, + "frameCpuUs": { "p50": 3487, "p95": 7351, "p99": 12690, "p999": 75737, "max": 346276 }, + "gpuUs": { "p50": 2299, "p95": 3236, "p99": 3653, "max": 342913 }, + "frameThreadAllocationBytes": { "p50": 52000, "p95": 82416, "max": 33240128 }, + "updateUs": { "p95": 1463 }, + "stationaryFiveSeconds": { + "frames": 925, + "frameCpuUs": { "p50": 4802, "p95": 7422, "p99": 8281 }, + "gpuUs": { "p50": 2711, "p99": 3242 }, + "frameThreadAllocationBytes": { "p50": 37688, "max": 13654512 }, + "updateUs": { "p95": 1328 } + }, + "process": { + "allocationBytesPerSecondMean": 60704238, + "allocationBytesPerSecondMax": 2059370560, + "gcPauseSecondsTotal": 0.328081, + "gcPauseSecondsMax": 0.120149, + "workingSetBytesMax": 834510848, + "gen0Collections": 30, + "gen1Collections": 19, + "gen2Collections": 12, + "contentions": 2145 + } + }, + { + "name": "uncapped-dense-arwic-no-process-counters", + "binaryCommit": "24f898c54d7c6547555a4402543b6a3cd7ced7f6", + "sourceCommit": "24f898c54d7c6547555a4402543b6a3cd7ced7f6", + "binaryMatchesSource": true, + "result": "pass-graceful", + "stationaryFiveSeconds": { + "frames": 918, + "frameCpuUs": { "p50": 4870, "p95": 7212, "p99": 8081 }, + "gpuUs": { "p50": 2726, "p99": 2861 }, + "frameThreadAllocationBytes": { "p50": 38632, "max": 13661072 }, + "updateUs": { "p95": 1245 } + } + } + ], + "observerEffect": { + "comparison": "uncapped-dense-arwic process counters enabled versus disabled", + "cpuP50Percent": -1.396, + "cpuP95Percent": 2.912, + "cpuP99Percent": 2.475, + "gpuP50Percent": -0.550, + "frameThreadAllocationP50Percent": -2.440, + "interpretation": "The process-counter observer changes the stable sample by less than normal route variance." + }, + "contentionStacks": { + "capture": "capped-nine-stop", + "totalAttributed": 5994, + "sites": [ + { "site": "AcDream.Content.DatDatabaseWrapper.TryGet", "count": 4516 }, + { "site": "ObjectMeshManager.ProcessQueue", "count": 976 }, + { "site": "DecodedTextureCache.GetOrCreate", "count": 254 }, + { "site": "LandblockBuildFactory.Build", "count": 90 }, + { "site": "LiveEntityHydrationController.OnCreate", "count": 74 }, + { "site": "DatDatabaseWrapper.TryGetFileBytes", "count": 27 }, + { "site": "other", "count": 57 } + ] + }, + "remainingGate": [ + "Repeat the capped nine-stop route on the physical local AMD display path.", + "Repeat the uncapped nine-stop route on the physical local AMD display path.", + "Repeat the pinned uncapped dense-Arwic sample on the physical local AMD display path.", + "Commit the physical-local summary before starting behavior-changing Slice C." + ] +} diff --git a/docs/research/2026-07-24-slice-a-rdp-baselines.md b/docs/research/2026-07-24-slice-a-rdp-baselines.md new file mode 100644 index 00000000..70098a71 --- /dev/null +++ b/docs/research/2026-07-24-slice-a-rdp-baselines.md @@ -0,0 +1,72 @@ +# Slice A corrected RDP baselines + +**Date:** 2026-07-24 +**Authority:** diagnostic only; these captures do not satisfy the physical-local +Slice A exit gate. + +The corrected tooling was exercised over capped and uncapped nine-stop routes, +plus a pinned uncapped dense-Arwic sample. All runs reached the world, completed +their route, and disconnected gracefully. The final dense run was made after +binary provenance became part of every soak report: source and measured binary +were both commit `24f898c54d7c6547555a4402543b6a3cd7ced7f6`, the tracked source +tree was clean, and the report recorded an exact match. + +The machine/display contract is defined in +[`2026-07-24-performance-reference-config.md`](2026-07-24-performance-reference-config.md). +The full sanitized figures and portal windows are in +[`2026-07-24-slice-a-rdp-baselines.json`](2026-07-24-slice-a-rdp-baselines.json). +Raw CSV, EventPipe, screenshot, and process logs are intentionally local, +untracked diagnostic artifacts. + +## Headline measurements + +| Capture | Frames | CPU p50 / p95 / p99 | GPU p50 / p95 / p99 | Frame-thread allocation p50 / max | +|---|---:|---:|---:|---:| +| Capped nine-stop | 17,400 | 31.240 / 31.868 / 33.032 ms | 3.626 / 5.826 / 6.807 ms | 31.6 KiB / 202.8 MiB | +| Uncapped nine-stop | 229,412 | 1.537 / 6.077 / 8.938 ms | 0.935 / 3.288 / 3.470 ms | 28.3 KiB / 206.3 MiB | +| Uncapped dense Arwic | 16,350 | 3.487 / 7.351 / 12.690 ms | 2.299 / 3.236 / 3.653 ms | 50.8 KiB / 31.7 MiB | +| Dense Arwic, stable 5 s | 925 | 4.802 / 7.422 / 8.281 ms | 2.711 / — / 3.242 ms | 36.8 KiB / 13.0 MiB | + +The capped CPU distribution is the expected 32 ms RDP/pacing envelope, while +the GL timestamps show only 3.6–6.8 ms of ordinary GPU work. The corrected GPU +timer therefore no longer measures frame pacing as GPU submission time. + +Portal materialization remains the dominant transient cost. The uncapped +Sawato materializations reached 206.3 MiB and 203.2 MiB of frame-thread +allocation in a single frame. Process-wide allocation averaged 48.5 MB/s over +the uncapped route, with 80 Gen-2 collections and a longest GC pause of +260 ms. This evidence keeps prepared-content work as the next architectural +priority. + +## Observer effect + +The dense-Arwic stationary sample was repeated without the process counter +observer. Enabling it changed CPU p50 by -1.4%, p95 by +2.9%, p99 by +2.5%, +GPU p50 by -0.6%, and frame-thread allocation p50 by -2.4%. Those differences +are within natural route variance, so the observer is suitable for directional +process-wide allocation and GC attribution. + +## Contention attribution + +The capped route recorded 5,994 contentions with stacks: + +| Site | Count | Share | +|---|---:|---:| +| `DatDatabaseWrapper.TryGet` | 4,516 | 75.3% | +| `ObjectMeshManager.ProcessQueue` | 976 | 16.3% | +| `DecodedTextureCache.GetOrCreate` | 254 | 4.2% | +| `LandblockBuildFactory.Build` | 90 | 1.5% | +| `LiveEntityHydrationController.OnCreate` | 74 | 1.2% | +| `DatDatabaseWrapper.TryGetFileBytes` | 27 | 0.5% | +| Other | 57 | 1.0% | + +This replaces speculation with a concrete ranking. It does not authorize +lock-shape changes yet; the prepared-content slices should first remove demand +from these paths and then the same capture must be repeated. + +## Gate state + +Slice A’s instrumentation and RDP validation are complete. The remaining exit +gate is a capped nine-stop, uncapped nine-stop, and pinned dense-Arwic capture +on the physical local AMD display path. Slice B is baseline-independent and may +proceed. Slice C remains blocked until the physical-local evidence is committed.