erik/acdream
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

docs(spatial): master plan — VERBATIM port of the retail spatial pipeline (no hybrids)

Browse source 
The doorway saga (void -> transparent walls -> flaps) proved patching the hybrid is hopeless:
retail does membership + collision + camera + render as ONE coupled pipeline; acdream
reimplemented pieces with mismatched criteria at the seams. Master plan to port ALL of it
verbatim: A membership (find_cell_list/find_transit_cells/find_building_transit_cells intrinsic,
no bridge), B uniform collision (no indoor/outdoor fork) + door collision, C camera
(update_viewer + find_visible_child_cell), D the full PView render (ConstructView/InitCell/
ClipPortals/GetClip/DrawCells/DrawPortal + the update_count watermark). KEEP/REPLACE/DELETE
lists, decomp anchors per function, P0-P6 sequence (apparatus-first, foundation-up, visual gate
each), and the kickoff prompt. Supersedes the render-only redesign's scope.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Erik 2026-06-03 13:57:25 +02:00
parent 0cc561c4d0
commit a859116d5f
1 changed files with 184 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

184
docs/superpowers/specs/2026-06-03-verbatim-spatial-pipeline-port-master-plan.md Normal file
View file

@ -0,0 +1,184 @@
# Master Plan — VERBATIM port of the retail spatial pipeline (membership + collision + camera + render)
> **Mandate (user, 2026-06-03):** *"No hybrids, no bandaids. I want FULLY ported behavior. Don't be
> afraid to break stuff. Port everything now."* The doorway saga (void → transparent walls → flaps)
> proved that patching the hybrid produces a new break for every fix. **Stop patching. Port retail's
> integrated spatial pipeline verbatim** from `docs/research/named-retail/acclient_2013_pseudo_c.txt`
> + `acclient.h`. The code is modern C#; the behavior is retail, line-for-line. Breaking intermediate
> states is acceptable — correctness-by-construction is the goal, not incremental safety.
>
> This supersedes the *scope* of the prior render-only redesign
> ([`2026-06-02-render-pipeline-redesign-design.md`](2026-06-02-render-pipeline-redesign-design.md)) by
> adding the membership + collision + camera halves that the render rides on. The retail-pipeline
> reference [`../../research/2026-06-02-retail-render-pipeline-full-reference.md`](../../research/2026-06-02-retail-render-pipeline-full-reference.md)
> (the `PView` decomp + the CL-A…CL-G checklist) is the render half's spine; this doc is the superset.
---
## 0. Why a hybrid can't work here (the root lesson)
Retail makes **one** decision flow, shared across physics and render:
```
sphere sweep (CTransition) carries curr_cell THROUGH collision over a UNIFORM candidate set
→ membership = find_cell_list pick over that set (one point_in_cell criterion)
→ the camera runs its OWN sweep → viewer_cell
→ render roots at viewer_cell, floods portals (PView), seals (DrawCells)
```
acdream split this into independent reimplementations that use **different criteria** at the seams:
- membership exit uses `point_in_cell` (center) but entry uses `SphereIntersectsCellBsp` (overlap) → a hysteresis gap → **threshold ping-pong** (`0031↔0170↔0171`);
- collision forks indoor-cell-BSP vs outdoor-terrain → the candidate set isn't uniform → push-back bounce at the doorway;
- the render PVS is a home-grown `ProjectToNdc`/`ScreenPolygonClip` that approximates `GetClip` → near-the-portal projection blows up → **void / transparent walls / flaps**.
Every patch moves the failure from one seam to the next. **Only a verbatim port closes all seams at once**, because retail's correctness is *structural* (one criterion, one candidate set, one viewpoint, one flood), not tuned.
---
## 1. KEEP / REPLACE / DELETE (be precise about the blast radius)
**KEEP (infrastructure — not the algorithm):**
- The WB mesh pipeline: global VAO/VBO/IBO, `ObjectMeshManager`, `WbMeshAdapter`, GfxObj/Setup decode, `TextureCache`, bindless/MDI. (Geometry storage + upload.)
- `EnvCellRenderer`'s **mesh/MDI/texture** path (the actual cell-shell draw), `TerrainModernRenderer`, `SkyRenderer`, `ParticleRenderer` — the GL draw primitives.
- `DatCollection` + the dat readers; `PhysicsDataCache` as the cache (its CONTENTS get the new graph).
- The `Transition`/`SpherePath`/`CollisionInfo`/`ObjectInfo`/`BSPQuery` engine **core** (the sphere-sweep math, the 6-path BSP dispatcher) — it is already a faithful port; the GAP is how it's *driven* (forked vs uniform).
- The V1 win: render keys on the **viewer** (camera), lighting on the **player** — that invariant is retail and stays.
**REPLACE (the algorithm, verbatim):**
- Membership driver: `CellTransit.BuildCellSetAndPickContaining` + `FindCellList`/`FindCellSet` → faithful `find_cell_list` + `find_transit_cells` (with intrinsic building entry).
- `PhysicsEngine.ResolveCellId` → demoted to spawn/teleport seed only; per-frame membership comes from the swept `curr_cell` (already true post-Stage-1, finish it).
- Collision driver: `Transition.FindEnvCollisions` forked indoor/outdoor branches → one uniform `find_env_collisions` over the candidate set (land cells sweep terrain tris; env cells sweep the cell BSP) — same loop, per cell.
- Render PVS: `PortalVisibilityBuilder` + `PortalProjection.ProjectToNdc` + `ScreenPolygonClip` + the `CellView`/`ClipFrame` NDC model → retail `PView` (`InitCell`/`ClipPortals`/`GetClip`/`AddViewToPortals`/`AddToCell`/`DrawCells`) + `portal_view_type`/`view_type`/`update_count`.
- Camera viewer-cell: the sweep-`CurCellId` approximation → `find_visible_child_cell` (graph/BSP) seeded at the player cell, per retail `update_viewer`.
**DELETE (the bandaids — once their faithful replacement lands):**
- `CellTransit.CheckBuildingTransit` (the building-entry bridge, #5).
- `ResolveCellId`'s indoor `SphereIntersectsCellBsp` verify + the ad-hoc outdoor `CheckBuildingTransit` branch + any remaining `#90` stickiness.
- `PortalVisibilityBuilder`, `PortalProjection`, `ScreenPolygonClip`, `ClipFrame`/`ClipFrameAssembler`/`CellView`/`PortalView` (the NDC clip model) — replaced by `view_type`/portal_view + stencil/scissor per retail `DrawCells`.
- `MinW` near-clip approximation (this session) — subsumed by `GetClip`'s real near-clipping.
- The dormant WB-two-pipe scaffolding (`Building`/`BuildingLoader` stencil, occlusion-query, `IsShellScopedSet`) — already mostly dead.
---
## 2. Scope — "EVERYTHING," enumerated with decomp anchors
> Anchors are `Class::method @ 0xADDR (pc:LINE)` in `docs/research/named-retail/acclient_2013_pseudo_c.txt`;
> structs are `acclient.h:LINE`. All verified this session or in the full-reference doc. Port each
> **verbatim** (same control flow, same predicates, same constants); write pseudocode first
> (workflow step 3) for the gnarly ones (`ClipPortals`, `DrawCells`, `find_transit_cells`).
### A. Membership — the shared cell graph (physics owns the cell)
| # | Retail | Anchor | acdream now |
|---|---|---|---|
| A1 | `CObjCell::find_cell_list` (set build + interior-wins pick) | `0x52b4e0` pc:308742 | partial (Stage 1 pick); finish set-build |
| A2 | `CEnvCell::find_transit_cells` (portal crossing → neighbors; exit→`add_all_outside`; **building portals intrinsic**) | `0x52c820` pc:309968 | hybrid (`CheckBuildingTransit` bridge) |
| A3 | `CEnvCell::check_building_transit` / `find_building_transit_cells` (intrinsic entry) | `0x52c5d0` pc:309827 / pc:318309 | bridge (#5) |
| A4 | `CLandCell::add_all_outside_cells` | `0x533630` pc:317499 | present (verify coord-convention) |
| A5 | `CObjCell::GetVisible` / `CEnvCell::GetVisible` (graph resolve, ≥0x100 split) | `0x52ad40` pc:308209 / `0x52dc10` pc:311378 | present (`CellGraph`) |
| A6 | `point_in_cell` (CEnvCell cell-BSP, CLandCell XY) — the ONE containment criterion | (vtable 0x84) | present (`PointInsideCellBsp`); make it the sole criterion |
| A7 | `CTransition::transitional_insert` / `validate_transition` / `check_other_cells` (collide-then-pick) | `0x50aa70` pc:272547 / `0x50ae50` pc:272717 | partial (Stage 1 `RunCheckOtherCellsAndAdvance`) |
| A8 | `CPhysicsObj::SetPositionInternal` / `change_cell` (commit on diff) | `0x515330` pc:283399 / `0x513390` pc:281192 | present (`UpdateCellId`) |
| A9 | `enter_cell` / `leave_cell` (per-cell `object_list` + `shadow_object_list` — the shared graph) | `0x510ed0` / `0x510f50` | acdream uses a landblock-wide `ShadowObjectRegistry` — port per-cell lists |
### B. Collision — one uniform sphere-sweep (no fork)
| # | Retail | Anchor | acdream now |
|---|---|---|---|
| B1 | `CObjCell::find_env_collisions` (CLandCell sweeps terrain tris; CEnvCell sweeps cell BSP) — **uniform per candidate cell** | CLandCell + `CEnvCell::find_env_collisions` `0x52c130` pc:309573 | **forked** `cellLow>=0x0100` branch (#4) |
| B2 | `BSPTREE::find_collisions` (the 6-path dispatcher) | `0x323924` | present (`BSPQuery`) — keep |
| B3 | `CPhysicsObj::FindObjCollisions` + binary BSP-vs-cyl dispatch | `pc:274435`+ | partial (A6.P7/P8) |
| B4 | door / building-shell collision (the push-back **bounce** at the threshold) | `CBuildingObj`/`CBldPortal` collision path | **buggy** (3 failing Core door tests, #97) |
| B5 | `pos_hits_sphere` / `polygon_hits_sphere_slow_but_sure` (per-poly static tests) | pc:322974 | present (A6.P4) |
### C. Camera / viewer (the ONE viewpoint)
| # | Retail | Anchor | acdream now |
|---|---|---|---|
| C1 | `SmartBox::update_viewer` (spring-arm sweep → `viewer` + `viewer_cell = sphere_path.curr_cell`; `AdjustPosition` fallbacks; snap-to-player) | `0x453ce0` pc:92761 | V1 ported the sweep + ViewerCellId; ADD the AdjustPosition fallbacks + faithful start-cell |
| C2 | `CameraManager::UpdateCamera` (desired eye = `viewer_sought_position`) | `0x456660` | `RetailChaseCamera` reimplements (damping) — KEEP, it feeds C1 |
| C3 | `CEnvCell::find_visible_child_cell` (viewer's child cell via portals/stab_list, NOT the sweep approximation, NOT AABB) | `0x52dc50` pc:311397 | not ported (uses sweep `CurCellId`) |
| C4 | `CameraSet::UpdateCamera` player-fade | `0x458ae0` pc:97703 | ported (`ComputeTranslucency`) — keep |
### D. Render — the full PView (the big one)
| # | Retail | Anchor | acdream now |
|---|---|---|---|
| D1 | `SmartBox::RenderNormalMode` (binary decision on viewer cell → `DrawInside(viewer_cell)` vs `LScape::draw`) | `0x453aa0` pc:92635 | V1 ported the keying — keep, wire to PView |
| D2 | `PView::DrawInside` → `ConstructView(CEnvCell)` (the BFS) | `0x5a5860` pc:433793 / `0x5a57b0` pc:433750 | `PortalVisibilityBuilder` (REPLACE) |
| D3 | `PView::InitCell` (per-portal sidedness vs `viewer.viewpoint`; `update_count = view_count`) | `0x5a4b70` pc:432896 | `CameraOnInteriorSide` (REPLACE) |
| D4 | `PView::ClipPortals` (exit→`outside_view`; interior→`OtherPortalClip` into neighbor) | `0x5a5520` pc:433572 | inline in builder (REPLACE) |
| D5 | `PView::GetClip` (portal poly → screen clip, **proper near-clipping**, honor Sidedness) | `0x5a4320` pc:432344 | `ProjectToNdc`+`ScreenPolygonClip` (REPLACE — this is the void/flap source) |
| D6 | `PView::AddViewToPortals` / `AddToCell` / `SetOtherSeen` / `InsCellTodoList` (worklist + watermark) | `0x5a52d0` pc:433446 / `0x5a4d90` pc:433050 / `0x5a4e30` / `0x5a4f50` | HashSet-seen BFS (REPLACE) |
| D7 | `PView::DrawCells` (LScape-thru-door + conditional Z-clear + 3 per-cell loops: stencil exit portals, draw closed mesh, draw per-cell objects/particles) | `0x5a4840` pc:432709 | `InteriorRenderer` per-cell loop (PARTIAL — re-port the seal verbatim) |
| D8 | `PView::ConstructView(CBldPortal)` / `DrawPortal` (outside-looking-in) | `0x5a59a0` pc:433827 / `0x5a5ab0` pc:433895 | not built (residual C) |
| D9 | `portal_view_type` / `view_type` / `update_count` watermark (per-cell view accumulation; #95/#102) | acclient.h:32346 / 32338 | `CellView`/`ClipFrame` NDC model (REPLACE) |
| D10 | `CellManager::ChangePosition` (keep/release landscape + sunlight on `seen_outside` of the PLAYER cell) | `0x4559b0` pc:94601 | V1 lighting-on-player — keep, verify against this |
| D11 | `CEnvCell::grab_visible_cells` (load landscape iff `seen_outside`) | `0x52e220` pc:311878 | partial |
| D12 | `LScape::draw` (terrain+sky, clipped via `Render::PortalList`) | `0x506330` | acdream splits terrain/sky/scenery — clip each to `outside_view` per D7 |
---
## 3. Sequence (foundation-up; each phase ends at a user visual gate)
The render rides on stable membership + a stable viewer. Port bottom-up so each phase has a solid base.
**P0 first** because it's the apparatus that makes "verbatim" checkable.
- **P0 — Conformance apparatus (before any port).** Headless fixtures of the Holtburg cottage neighborhood (cells `0xA9B4003x` + `0xA9B4017x`, the building stab, the cellar) loaded from the real dats. Golden tests that assert *retail* outcomes: `find_cell_list` returns the same cell as a captured retail trace at the threshold; `point_in_cell` matches; the PVS visible-set for a given (cell, eye) matches. Use the existing `ACDREAM_CAPTURE_RESOLVE` + cdb retail traces. **This is how we know a port is verbatim, not vibes.**
- **P1 — Membership (A) + uniform collision (B1).** Port `find_cell_list`/`find_transit_cells`/`find_building_transit_cells`/`add_all_outside_cells` intrinsic; delete `CheckBuildingTransit`. Port uniform `find_env_collisions` (no fork). One `point_in_cell` criterion everywhere. **Gate:** stand in the cottage doorway — the cell does NOT ping-pong (`[cell-transit]` DELTA=0 standing still, no `0031↔0170↔0171`); walk in/out is a clean monotonic cell sequence.
- **P2 — Door/building-shell collision (B3/B4).** Fix the push-back bounce (the 3 failing Core door tests go green). **Gate:** stand in the doorway — no position oscillation (foot Y stable); walk through cleanly; walls block.
- **P3 — Camera viewer-cell (C1/C3).** Port `find_visible_child_cell` + the faithful `update_viewer` start-cell/fallbacks. **Gate:** `viewerCell` is stable + correct as the camera orbits across boundaries (no `[flap-cam]` thrash).
- **P4 — PView render (D2–D9), the core.** Replace `PortalVisibilityBuilder`/`ProjectToNdc`/`ScreenPolygonClip` with `ConstructView`/`InitCell`/`ClipPortals`/`GetClip`/`AddViewToPortals` + `portal_view_type`/`update_count`; re-port `DrawCells`' seal verbatim. **Gate:** cottage interior sealed (opaque walls, no transparent/flap, no void), sky/terrain through the door only.
- **P5 — Outside-looking-in (D8).** `DrawPortal` + `ConstructView(CBldPortal)`. **Gate:** from the street the interior renders through the door (no see-through box).
- **P6 — Dungeons + cleanup (D11/D12).** Validate the all-EnvCell path (`seen_outside==0`, watermark converges, #95 closed). Delete all dead hybrids (§1 DELETE list). **Gate:** a real dungeon sealed, no terrain/sky, no FPS collapse.
Each phase: grep-named → pseudocode → port verbatim → P0 conformance test green → `dotnet build`/`dotnet test` → **user visual gate**. No phase batched past its gate.
---
## 4. No-shortcuts rules (enforced every task)
1. Every ported behavior cites its decomp anchor (address + pc:line) in a comment.
2. No suppression flags, grace periods, stickiness, or `if (problem) return` guards. If retail doesn't do it, we don't.
3. When retail's behavior is unclear, read the decomp / attach cdb (Step -1) — never guess.
4. A faithful port that breaks an acdream test means the acdream test encoded a hybrid assumption — fix the test to the retail truth, don't bend the port.
5. Each phase ends GREEN + at a user visual gate; the seal/membership is verified on screen + probes, never off the unit suite alone.
## 5. References (read before each half)
- Render half spine: [`../../research/2026-06-02-retail-render-pipeline-full-reference.md`](../../research/2026-06-02-retail-render-pipeline-full-reference.md) (CL-A…CL-G).
- Membership: [`../../research/2026-06-03-membership-and-bluehole-shipped-handoff.md`](../../research/2026-06-03-membership-and-bluehole-shipped-handoff.md) + the four 2026-06-02 decomp studies.
- The V1 single-viewpoint win (keep): [`2026-06-03-single-viewpoint-render-design.md`](2026-06-03-single-viewpoint-render-design.md).
- Oracle: `docs/research/named-retail/acclient_2013_pseudo_c.txt` + `acclient.h`; live trace via cdb (CLAUDE.md "Retail debugger toolchain").
---
## 6. KICKOFF PROMPT (copy-paste for the execution session)
```
VERBATIM PORT of the retail spatial pipeline — membership + collision + camera + render. NO hybrids,
NO bandaids; full retail-faithful behavior; breaking intermediate states is fine. The doorway saga
(void → transparent walls → flaps) proved patching the hybrid is hopeless: retail does membership +
collision + camera + render as ONE coupled pipeline and acdream reimplemented pieces of each with
mismatched criteria at the seams. Branch: claude/thirsty-goldberg-51bb9b (do NOT branch/worktree; do
NOT push without asking; NEVER git stash/gc). PowerShell on Windows; launch logs are UTF-16.
READ FIRST:
1. docs/superpowers/specs/2026-06-03-verbatim-spatial-pipeline-port-master-plan.md (THIS plan — scope
§2, KEEP/REPLACE/DELETE §1, sequence §3, no-shortcuts §4).
2. docs/research/2026-06-02-retail-render-pipeline-full-reference.md (the PView decomp + CL-A…CL-G).
3. The V1 single-viewpoint design (keep that invariant): render on the viewer cell, lighting on the
player. Already shipped this session (commits 832001d / d03fe84 / 1e9a9ca) + the void near-clip
fix (0cc561c, which P4's GetClip subsumes).
THE JOB — port EVERYTHING in §2 verbatim, bottom-up per §3: P0 conformance apparatus → P1 membership +
uniform collision → P2 door collision → P3 camera viewer-cell → P4 PView render (the core) → P5
outside-looking-in → P6 dungeons + delete the hybrids (§1). Workflow per task: grep-named → pseudocode
→ port verbatim (cite the anchor) → P0 conformance test → build/test → USER VISUAL GATE. Use
superpowers:writing-plans to turn each phase into a TDD plan; superpowers:executing-plans to run it.
DELETE (once replaced): PortalVisibilityBuilder, PortalProjection, ScreenPolygonClip, CellView/ClipFrame;
CheckBuildingTransit, the forked FindEnvCollisions branches, ResolveCellId's ad-hoc outdoor branch + #90
stickiness. KEEP: WB mesh pipeline + EnvCellRenderer mesh/MDI + TerrainModernRenderer + SkyRenderer (GL
draw primitives), the Transition/SpherePath/BSPQuery engine core, DatCollection, the V1 viewer-keying.
START AT P0 (the conformance apparatus) — it is how we prove "verbatim" instead of guessing. Do NOT
start P1 code before P0 fixtures + at least one golden retail-trace assertion exist.
TEST BASELINE going in: Core 1295 pass / 5 fail (2 BSPStepUp + 3 door-collision — the door ones are
P2's target); App 177 green.
```