feat(streaming): shadow-publish flat collision assets

Carry one immutable prepared collision closure with each accepted near-tier generation and install graph plus flat views through the same retained publication receipt. Apply the same strict package-only rule to live entities, add exact sampled graph-authoritative comparison artifacts and lifecycle counters, and prove cancellation, demotion, rehydrate, revisit, teardown, reconnect, and the nine-stop route with 14,064 zero-mismatch samples.

Co-authored-by: OpenAI Codex <codex@openai.com>
This commit is contained in:
Erik 2026-07-25 16:38:54 +02:00
parent f7ff9f4eea
commit d9446030e6
32 changed files with 2777 additions and 92 deletions

View file

@ -144,6 +144,17 @@ EnvCell topology remains independently keyed. Production traversal is still
the parsed graph oracle until Slice I6. Full-catalog evidence:
`docs/research/2026-07-25-slice-i3-prepared-collision-package.md`.
**Slice I5 dual-publication seam (2026-07-25).** Near-tier
`LandblockBuild` payloads now carry one immutable prepared-collision closure
outside the DAT lock. `LandblockPhysicsPublisher` installs the parsed oracle
and flat view under the same retained receipt; live objects use the strict
`LiveCollisionAssetPublisher`. Cell/topology ownership is landblock-scoped and
withdrawn on replacement, demotion, removal, and reset. The connected
graph-authoritative referee completed 14,064 exact samples with no mismatch or
fault. This is still one `DatCollection` and one prepared-package mmap, not a
second reader or a WorldBuilder runtime dependency. Evidence:
`docs/research/2026-07-25-slice-i5-dual-collision-shadow.md`.
**Retail VFX hook compatibility seam (2026-07-14).** Chorizite.DatReaderWriter
2.1.7 models `CreateBlockingParticleHook` as the common hook header only, while
retail inherits the complete `CreateParticleHook` payload. The narrow readers in

View file

@ -1661,13 +1661,18 @@ port in any phase — no separate listing here.
> installed-DAT comparisons, 1,200 complete resolver frames, and 10,000 warmed
> iterations at zero managed bytes with no mismatch. Production remains
> graph-authoritative while I5 dual publication and the connected shadow gate
> are active. Release build and 8,384 solution tests / 5 skips pass.
> execute. I5 is now complete: immutable near-build collision closures and the
> strict live-object publisher install both representations under canonical
> ownership. Lifecycle/reconnect plus the nine-stop route sampled 14,064
> queries with exact state/float-bit equality, zero mismatch/fault/artifact,
> equal graph/flat cell residency, and graceful teardown. Release build and
> 8,396 solution tests / 5 skips pass. I6 production cutover is active.
> The historical exact cutover
> rollback remains
> `git revert ef1d263337997bb030eadb7b8e71d73dc659907a`. Slice H is complete:
> retained UI/frame work, exact bounded light selection, and
> pooled/borrowed/direct network I/O pass 8,292 Release tests / 5 skips and the
> connected lifecycle/reconnect gate. Slice I is active at I5 from
> connected lifecycle/reconnect gate. Slice I is active at I6 from
> [its detailed plan](2026-07-25-modern-runtime-slice-i.md).
**Spec:** `docs/superpowers/specs/2026-07-05-modern-pipeline-design.md` (the
@ -1692,11 +1697,11 @@ hitch and follows.
| MP1a | `AcDream.Content` extraction (GL-free MeshExtractor + boundary records out of App) | ✅ SHIPPED 2026-07-05 — user-gated (renders identical, zero tripwires, perf-neutral); 8 commits `651d041e`..`b0758d77` |
| MP1b | Pak format + `acdream-bake` CLI + mmap `PakReader` + equivalence/full-scale gate | ✅ **SHIPPED 2026-07-24** — 729,888 EnvCell keys → 17,117 unique geometries + 712,771 aliases (42.6×), 751,141 total keys / 38,370 physical blobs, zero failures, 28,192.4 MiB, 81.4 s validated atomic publish; full-DAT gate also corrected a real collision in WB's legacy geometry hash. [Report](../research/2026-07-24-slice-b-full-bake-report.md) |
| **MP-Alloc (safe batch)** | Reuse per-frame buffers: anim pose, particle draw-list, interior partition, animatedIds/drawableCells sets | ✅ SHIPPED + USER-GATED 2026-07-05 — dense-town frame-time spikes 2087ms → 610ms, alloc spikes (3075MB single-frame) eliminated, gen2 GC 511/window → ~0; user confirms FPS steady. 4 commits `b8c05e2b`..`91afea24` |
| MP-Alloc (hard sites) | EnvCell settled-camera rebuild gate + physics `Transition` reuse | 🟡 Physics half SHIPPED 2026-07-25 — retail-shaped reset-complete transition/query scratch is bit-identical and 0 B/resolve across five profiles; immutable flat collision storage, its deterministic prepared package, and the zero-mismatch/zero-allocation flat traversal referee are shipped; connected dual publication continues in Slice I5. |
| MP-Alloc (hard sites) | EnvCell settled-camera rebuild gate + physics `Transition` reuse | 🟡 Physics half SHIPPED 2026-07-25 — retail-shaped reset-complete transition/query scratch is bit-identical and 0 B/resolve across five profiles; immutable flat collision storage, deterministic package, zero-allocation traversal, strict dual publication, and the 14,064-sample connected shadow gate are shipped; production flat cutover continues in Slice I6. |
| MP1c | Streaming cutover to pak + hitch gate | ✅ **SHIPPED 2026-07-24** — typed package-only production source, explicit Setup probes, non-allocating Portal-first lookup, exact translucency metadata, and ordered teardown; capped/uncapped/dense physical routes + user visual gate pass. Uncapped CPU p99 7.825→6.496 ms, GPU p99 3.406→2.706 ms, portal frame max 202.9→39.9 MiB, process allocation 41.8%, zero invalid Setup probes. [Report](../research/2026-07-24-slice-c-prepared-asset-cutover-report.md) |
| MP2 | Retail particle distance/cell-view degradation: DAT-authored range, exact infinite/finite off-view update branches, emitter-first render scan, user-requested 2× default range | ✅ SHIPPED 2026-07-17 — Aerlinthe hotspot `0x32000223` resolves to retail 64m and defaults to 128m; landscape/entity streaming distance unchanged |
| MP3 | Arch ECS render world + delta submission (the 300-FPS lever) | ⚪ — note: does NOT fix the steady-state GC churn (that's MP-Alloc); MP3 is the draw-submission throughput lever |
| MP4 | Zero-alloc frame loop + flat physics data (residual, post-MP-Alloc) | 🟡 ACTIVE — Slice I1 zero-allocation transition scratch, I2 immutable schema, I3 deterministic collision package, and I4 exact flat traversal referee shipped; I5 dual publication and connected shadow verification are active. |
| MP4 | Zero-alloc frame loop + flat physics data (residual, post-MP-Alloc) | 🟡 ACTIVE — Slice I1 zero-allocation transition scratch, I2 immutable schema, I3 deterministic collision package, I4 exact flat traversal, and I5 strict dual publication plus connected shadow verification shipped; I6 production cutover is active. |
| MP5 | Job-system parallelism | ⚪ stretch, evidence-gated |
---

View file

@ -134,8 +134,14 @@ randomized static/swept comparisons, 5,000 walkable comparisons, 7,500
installed-DAT comparisons, and 1,200 complete resolver frames have exact
state/float-bit equality; 10,000 warmed flat iterations allocate zero managed
bytes. Release build and 8,384 solution tests / 5 skips pass.
Slice H is complete: retained UI/frame work, exact bounded light selection,
and pooled/borrowed/direct network I/O pass. Slice I is active at I5 under
Slice I5 is complete: immutable near-build collision closures and strict live
collision publication install graph and prepared-flat views together. The
lifecycle/reconnect and nine-stop connected routes sampled 14,064 exact
queries with zero mismatch, fault, or artifact; graph/flat cell residency
matched at every stable checkpoint and teardown converged. Release build and
8,396 solution tests / 5 skips pass. Slice H is complete: retained UI/frame
work, exact bounded light selection, and pooled/borrowed/direct network I/O
pass. Slice I is active at I6 under
[`2026-07-25-modern-runtime-slice-i.md`](2026-07-25-modern-runtime-slice-i.md).
The historical exact G4 visual rollback remains
`git revert ef1d263337997bb030eadb7b8e71d73dc659907a`.

View file

@ -1,8 +1,9 @@
# Modern runtime Slice I — flat collision assets and zero-allocation physics
**Status:** I0-I4 COMPLETE - retail oracle, zero-allocation transition
**Status:** I0-I5 COMPLETE - retail oracle, zero-allocation transition
scratch, deterministic immutable collision records, complete prepared
package, and exact flat-traversal differential gate fixed; I5 active
package, exact flat traversal, dual publication, and connected shadow gate;
I6 active
**Parent:** `2026-07-24-modern-runtime-architecture.md`, Slice I
**Purpose:** remove parsed DAT object graphs and steady collision allocations
without changing one retail collision decision, float, comparison, or traversal
@ -241,6 +242,17 @@ until I5/I6. Release build and 8,384 solution tests / 5 skips pass. Evidence:
Gate: zero shadow mismatch and no retained collision owner after landblock or
session teardown.
**Complete 2026-07-25.** Near-tier builds carry one immutable prepared
collision closure; streamed and live objects publish graph and flat assets
strictly, with no graph-only fallback. Cancellation, publication, demotion,
rehydration, revisit, adjacent-landblock isolation, removal, and session
teardown gates pass. The complete Release solution passes 8,396 tests / 5
skips. The connected lifecycle/reconnect and nine-stop routes sampled 14,064
queries with exact state/float-bit equality, zero mismatch, zero flat fault,
zero mismatch artifact, equal graph/flat cell residency at every stable
checkpoint, and graceful exits. Evidence:
[`../research/2026-07-25-slice-i5-dual-collision-shadow.md`](../research/2026-07-25-slice-i5-dual-collision-shadow.md).
### I6 — production cutover and graph removal
1. Flip canonical `PhysicsDataCache` records and every `BSPQuery` production

View file

@ -0,0 +1,91 @@
# Slice I5 — dual collision publication and connected shadow gate
## Outcome
Slice I5 is complete. Production gameplay remains graph-authoritative for this
checkpoint, but every streamed near-tier generation and every live DAT object
now publishes the matching prepared flat collision representation. The sampled
flat path runs from a complete cloned transition, compares exact state and
float bits, and cannot alter the graph result used by gameplay.
No visual, collision, movement, or retail algorithm changed in this slice.
## Ownership
- `LandblockBuild` carries one immutable `LandblockCollisionBuild` containing
sorted source IDs, flat GfxObj/Setup collision, flat CellStruct geometry, and
per-EnvCell topology.
- `LandblockBuildFactory` reads that closure from the existing
`IPreparedCollisionSource` after the serialized DAT transaction. A missing or
corrupt prepared record rejects the complete near-tier generation; a
graph-only partial publication is forbidden.
- `LandblockPhysicsPublisher` commits graph and flat views through one retained
publication receipt. Cell and topology ownership is withdrawn on exact
replacement, near-to-far demotion, full removal, and session teardown.
- `LiveCollisionAssetPublisher` applies the same strict dual-publication rule
to CreateObject, appearance, and local-player Setup paths.
- `PhysicsDataCache` exposes typed graph/flat counts in canonical lifecycle
snapshots. Landblock-scoped cell and topology catalogs retire together.
## Shadow referee
`CollisionShadowVerifier` samples the production entry points while leaving the
graph result authoritative. The flat query receives one retained, reset-complete
clone of `ObjectInfo`, `SpherePath`, and `CollisionInfo`. Comparisons cover:
- containment availability, point containment, sphere containment;
- root bounding spheres and physics availability;
- cell and GfxObj moving-collision results;
- complete transition state, collision/contact state, sphere path, candidate
ordering, walkable vertices, GUID lists, and exact IEEE-754 float bits.
A mismatch or flat fault writes one deterministic JSON artifact. The disabled
production path allocates zero managed bytes over 10,000 warmed queries.
## Automated evidence
- Focused Core and App collision/publication suites pass.
- Cancellation publishes neither representation.
- Pending receipt to accepted publication, demotion, rehydration,
same-location replacement, adjacent-landblock isolation, full removal, and
session reset preserve exact ownership without duplication.
- Release build: zero warnings and zero errors.
- Complete solution: 8,396 passed / 5 skipped.
## Connected evidence
Lifecycle/reconnect gate:
- artifact:
`logs/connected-world-gate-20260725-162020/report.json`
- capped login, Aerlinthe, Rynthid, Facility Hub, Holtburg, Aerlinthe revisit,
then a fresh uncapped reconnect process;
- 12,730 sampled queries, 12,730 exact matches;
- zero mismatches, zero flat faults, zero mismatch artifacts;
- graph/flat cell counts equal at every stable checkpoint;
- all publication, retirement, upload, and warmup backlogs converged to zero;
- both processes disconnected gracefully with exit code 0.
Nine-stop R6 route:
- artifact:
`logs/connected-r6-soak-20260725-162614.report.json`
- Caul, Sawato, Rynthid, Aerlinthe, Sawato revisit, Holtburg, Caul revisit,
Sawato plateau, and Caul plateau;
- turn, forward/release, jump, combat-toggle, and movement-truth exercises;
- 42,719 eligible queries, 1,334 sampled queries, 1,334 exact matches;
- zero mismatches, zero flat faults, zero mismatch artifacts;
- graph/flat cell counts equal at all nine stable checkpoints;
- graceful client shutdown and zero gate failures.
The run occurred under RDP/lock-screen conditions. It is accepted for
deterministic correctness and lifetime evidence only; its absolute frame/GPU
measurements are not a physical-display performance gate.
## Next
Slice I6 flips canonical production traversal to the prepared flat
representation in one bisectable commit. The graph remains a temporary
automated referee for the initial cutover gate. The exact cutover commit and
`git revert <sha>` command must be recorded before correctness/visual
acceptance, after which production graph retention can be removed.