diff --git a/docs/superpowers/specs/2026-06-03-verbatim-spatial-pipeline-port-master-plan.md b/docs/superpowers/specs/2026-06-03-verbatim-spatial-pipeline-port-master-plan.md new file mode 100644 index 0000000..47e7406 --- /dev/null +++ b/docs/superpowers/specs/2026-06-03-verbatim-spatial-pipeline-port-master-plan.md @@ -0,0 +1,184 @@ +# Master Plan — VERBATIM port of the retail spatial pipeline (membership + collision + camera + render) + +> **Mandate (user, 2026-06-03):** *"No hybrids, no bandaids. I want FULLY ported behavior. Don't be +> afraid to break stuff. Port everything now."* The doorway saga (void → transparent walls → flaps) +> proved that patching the hybrid produces a new break for every fix. **Stop patching. Port retail's +> integrated spatial pipeline verbatim** from `docs/research/named-retail/acclient_2013_pseudo_c.txt` +> + `acclient.h`. The code is modern C#; the behavior is retail, line-for-line. Breaking intermediate +> states is acceptable — correctness-by-construction is the goal, not incremental safety. +> +> This supersedes the *scope* of the prior render-only redesign +> ([`2026-06-02-render-pipeline-redesign-design.md`](2026-06-02-render-pipeline-redesign-design.md)) by +> adding the membership + collision + camera halves that the render rides on. The retail-pipeline +> reference [`../../research/2026-06-02-retail-render-pipeline-full-reference.md`](../../research/2026-06-02-retail-render-pipeline-full-reference.md) +> (the `PView` decomp + the CL-A…CL-G checklist) is the render half's spine; this doc is the superset. + +--- + +## 0. Why a hybrid can't work here (the root lesson) + +Retail makes **one** decision flow, shared across physics and render: + +``` +sphere sweep (CTransition) carries curr_cell THROUGH collision over a UNIFORM candidate set + → membership = find_cell_list pick over that set (one point_in_cell criterion) + → the camera runs its OWN sweep → viewer_cell + → render roots at viewer_cell, floods portals (PView), seals (DrawCells) +``` + +acdream split this into independent reimplementations that use **different criteria** at the seams: +- membership exit uses `point_in_cell` (center) but entry uses `SphereIntersectsCellBsp` (overlap) → a hysteresis gap → **threshold ping-pong** (`0031↔0170↔0171`); +- collision forks indoor-cell-BSP vs outdoor-terrain → the candidate set isn't uniform → push-back bounce at the doorway; +- the render PVS is a home-grown `ProjectToNdc`/`ScreenPolygonClip` that approximates `GetClip` → near-the-portal projection blows up → **void / transparent walls / flaps**. + +Every patch moves the failure from one seam to the next. **Only a verbatim port closes all seams at once**, because retail's correctness is *structural* (one criterion, one candidate set, one viewpoint, one flood), not tuned. + +--- + +## 1. KEEP / REPLACE / DELETE (be precise about the blast radius) + +**KEEP (infrastructure — not the algorithm):** +- The WB mesh pipeline: global VAO/VBO/IBO, `ObjectMeshManager`, `WbMeshAdapter`, GfxObj/Setup decode, `TextureCache`, bindless/MDI. (Geometry storage + upload.) +- `EnvCellRenderer`'s **mesh/MDI/texture** path (the actual cell-shell draw), `TerrainModernRenderer`, `SkyRenderer`, `ParticleRenderer` — the GL draw primitives. +- `DatCollection` + the dat readers; `PhysicsDataCache` as the cache (its CONTENTS get the new graph). +- The `Transition`/`SpherePath`/`CollisionInfo`/`ObjectInfo`/`BSPQuery` engine **core** (the sphere-sweep math, the 6-path BSP dispatcher) — it is already a faithful port; the GAP is how it's *driven* (forked vs uniform). +- The V1 win: render keys on the **viewer** (camera), lighting on the **player** — that invariant is retail and stays. + +**REPLACE (the algorithm, verbatim):** +- Membership driver: `CellTransit.BuildCellSetAndPickContaining` + `FindCellList`/`FindCellSet` → faithful `find_cell_list` + `find_transit_cells` (with intrinsic building entry). +- `PhysicsEngine.ResolveCellId` → demoted to spawn/teleport seed only; per-frame membership comes from the swept `curr_cell` (already true post-Stage-1, finish it). +- Collision driver: `Transition.FindEnvCollisions` forked indoor/outdoor branches → one uniform `find_env_collisions` over the candidate set (land cells sweep terrain tris; env cells sweep the cell BSP) — same loop, per cell. +- Render PVS: `PortalVisibilityBuilder` + `PortalProjection.ProjectToNdc` + `ScreenPolygonClip` + the `CellView`/`ClipFrame` NDC model → retail `PView` (`InitCell`/`ClipPortals`/`GetClip`/`AddViewToPortals`/`AddToCell`/`DrawCells`) + `portal_view_type`/`view_type`/`update_count`. +- Camera viewer-cell: the sweep-`CurCellId` approximation → `find_visible_child_cell` (graph/BSP) seeded at the player cell, per retail `update_viewer`. + +**DELETE (the bandaids — once their faithful replacement lands):** +- `CellTransit.CheckBuildingTransit` (the building-entry bridge, #5). +- `ResolveCellId`'s indoor `SphereIntersectsCellBsp` verify + the ad-hoc outdoor `CheckBuildingTransit` branch + any remaining `#90` stickiness. +- `PortalVisibilityBuilder`, `PortalProjection`, `ScreenPolygonClip`, `ClipFrame`/`ClipFrameAssembler`/`CellView`/`PortalView` (the NDC clip model) — replaced by `view_type`/portal_view + stencil/scissor per retail `DrawCells`. +- `MinW` near-clip approximation (this session) — subsumed by `GetClip`'s real near-clipping. +- The dormant WB-two-pipe scaffolding (`Building`/`BuildingLoader` stencil, occlusion-query, `IsShellScopedSet`) — already mostly dead. + +--- + +## 2. Scope — "EVERYTHING," enumerated with decomp anchors + +> Anchors are `Class::method @ 0xADDR (pc:LINE)` in `docs/research/named-retail/acclient_2013_pseudo_c.txt`; +> structs are `acclient.h:LINE`. All verified this session or in the full-reference doc. Port each +> **verbatim** (same control flow, same predicates, same constants); write pseudocode first +> (workflow step 3) for the gnarly ones (`ClipPortals`, `DrawCells`, `find_transit_cells`). + +### A. Membership — the shared cell graph (physics owns the cell) +| # | Retail | Anchor | acdream now | +|---|---|---|---| +| A1 | `CObjCell::find_cell_list` (set build + interior-wins pick) | `0x52b4e0` pc:308742 | partial (Stage 1 pick); finish set-build | +| A2 | `CEnvCell::find_transit_cells` (portal crossing → neighbors; exit→`add_all_outside`; **building portals intrinsic**) | `0x52c820` pc:309968 | hybrid (`CheckBuildingTransit` bridge) | +| A3 | `CEnvCell::check_building_transit` / `find_building_transit_cells` (intrinsic entry) | `0x52c5d0` pc:309827 / pc:318309 | bridge (#5) | +| A4 | `CLandCell::add_all_outside_cells` | `0x533630` pc:317499 | present (verify coord-convention) | +| A5 | `CObjCell::GetVisible` / `CEnvCell::GetVisible` (graph resolve, ≥0x100 split) | `0x52ad40` pc:308209 / `0x52dc10` pc:311378 | present (`CellGraph`) | +| A6 | `point_in_cell` (CEnvCell cell-BSP, CLandCell XY) — the ONE containment criterion | (vtable 0x84) | present (`PointInsideCellBsp`); make it the sole criterion | +| A7 | `CTransition::transitional_insert` / `validate_transition` / `check_other_cells` (collide-then-pick) | `0x50aa70` pc:272547 / `0x50ae50` pc:272717 | partial (Stage 1 `RunCheckOtherCellsAndAdvance`) | +| A8 | `CPhysicsObj::SetPositionInternal` / `change_cell` (commit on diff) | `0x515330` pc:283399 / `0x513390` pc:281192 | present (`UpdateCellId`) | +| A9 | `enter_cell` / `leave_cell` (per-cell `object_list` + `shadow_object_list` — the shared graph) | `0x510ed0` / `0x510f50` | acdream uses a landblock-wide `ShadowObjectRegistry` — port per-cell lists | + +### B. Collision — one uniform sphere-sweep (no fork) +| # | Retail | Anchor | acdream now | +|---|---|---|---| +| B1 | `CObjCell::find_env_collisions` (CLandCell sweeps terrain tris; CEnvCell sweeps cell BSP) — **uniform per candidate cell** | CLandCell + `CEnvCell::find_env_collisions` `0x52c130` pc:309573 | **forked** `cellLow>=0x0100` branch (#4) | +| B2 | `BSPTREE::find_collisions` (the 6-path dispatcher) | `0x323924` | present (`BSPQuery`) — keep | +| B3 | `CPhysicsObj::FindObjCollisions` + binary BSP-vs-cyl dispatch | `pc:274435`+ | partial (A6.P7/P8) | +| B4 | door / building-shell collision (the push-back **bounce** at the threshold) | `CBuildingObj`/`CBldPortal` collision path | **buggy** (3 failing Core door tests, #97) | +| B5 | `pos_hits_sphere` / `polygon_hits_sphere_slow_but_sure` (per-poly static tests) | pc:322974 | present (A6.P4) | + +### C. Camera / viewer (the ONE viewpoint) +| # | Retail | Anchor | acdream now | +|---|---|---|---| +| C1 | `SmartBox::update_viewer` (spring-arm sweep → `viewer` + `viewer_cell = sphere_path.curr_cell`; `AdjustPosition` fallbacks; snap-to-player) | `0x453ce0` pc:92761 | V1 ported the sweep + ViewerCellId; ADD the AdjustPosition fallbacks + faithful start-cell | +| C2 | `CameraManager::UpdateCamera` (desired eye = `viewer_sought_position`) | `0x456660` | `RetailChaseCamera` reimplements (damping) — KEEP, it feeds C1 | +| C3 | `CEnvCell::find_visible_child_cell` (viewer's child cell via portals/stab_list, NOT the sweep approximation, NOT AABB) | `0x52dc50` pc:311397 | not ported (uses sweep `CurCellId`) | +| C4 | `CameraSet::UpdateCamera` player-fade | `0x458ae0` pc:97703 | ported (`ComputeTranslucency`) — keep | + +### D. Render — the full PView (the big one) +| # | Retail | Anchor | acdream now | +|---|---|---|---| +| D1 | `SmartBox::RenderNormalMode` (binary decision on viewer cell → `DrawInside(viewer_cell)` vs `LScape::draw`) | `0x453aa0` pc:92635 | V1 ported the keying — keep, wire to PView | +| D2 | `PView::DrawInside` → `ConstructView(CEnvCell)` (the BFS) | `0x5a5860` pc:433793 / `0x5a57b0` pc:433750 | `PortalVisibilityBuilder` (REPLACE) | +| D3 | `PView::InitCell` (per-portal sidedness vs `viewer.viewpoint`; `update_count = view_count`) | `0x5a4b70` pc:432896 | `CameraOnInteriorSide` (REPLACE) | +| D4 | `PView::ClipPortals` (exit→`outside_view`; interior→`OtherPortalClip` into neighbor) | `0x5a5520` pc:433572 | inline in builder (REPLACE) | +| D5 | `PView::GetClip` (portal poly → screen clip, **proper near-clipping**, honor Sidedness) | `0x5a4320` pc:432344 | `ProjectToNdc`+`ScreenPolygonClip` (REPLACE — this is the void/flap source) | +| D6 | `PView::AddViewToPortals` / `AddToCell` / `SetOtherSeen` / `InsCellTodoList` (worklist + watermark) | `0x5a52d0` pc:433446 / `0x5a4d90` pc:433050 / `0x5a4e30` / `0x5a4f50` | HashSet-seen BFS (REPLACE) | +| D7 | `PView::DrawCells` (LScape-thru-door + conditional Z-clear + 3 per-cell loops: stencil exit portals, draw closed mesh, draw per-cell objects/particles) | `0x5a4840` pc:432709 | `InteriorRenderer` per-cell loop (PARTIAL — re-port the seal verbatim) | +| D8 | `PView::ConstructView(CBldPortal)` / `DrawPortal` (outside-looking-in) | `0x5a59a0` pc:433827 / `0x5a5ab0` pc:433895 | not built (residual C) | +| D9 | `portal_view_type` / `view_type` / `update_count` watermark (per-cell view accumulation; #95/#102) | acclient.h:32346 / 32338 | `CellView`/`ClipFrame` NDC model (REPLACE) | +| D10 | `CellManager::ChangePosition` (keep/release landscape + sunlight on `seen_outside` of the PLAYER cell) | `0x4559b0` pc:94601 | V1 lighting-on-player — keep, verify against this | +| D11 | `CEnvCell::grab_visible_cells` (load landscape iff `seen_outside`) | `0x52e220` pc:311878 | partial | +| D12 | `LScape::draw` (terrain+sky, clipped via `Render::PortalList`) | `0x506330` | acdream splits terrain/sky/scenery — clip each to `outside_view` per D7 | + +--- + +## 3. Sequence (foundation-up; each phase ends at a user visual gate) + +The render rides on stable membership + a stable viewer. Port bottom-up so each phase has a solid base. +**P0 first** because it's the apparatus that makes "verbatim" checkable. + +- **P0 — Conformance apparatus (before any port).** Headless fixtures of the Holtburg cottage neighborhood (cells `0xA9B4003x` + `0xA9B4017x`, the building stab, the cellar) loaded from the real dats. Golden tests that assert *retail* outcomes: `find_cell_list` returns the same cell as a captured retail trace at the threshold; `point_in_cell` matches; the PVS visible-set for a given (cell, eye) matches. Use the existing `ACDREAM_CAPTURE_RESOLVE` + cdb retail traces. **This is how we know a port is verbatim, not vibes.** +- **P1 — Membership (A) + uniform collision (B1).** Port `find_cell_list`/`find_transit_cells`/`find_building_transit_cells`/`add_all_outside_cells` intrinsic; delete `CheckBuildingTransit`. Port uniform `find_env_collisions` (no fork). One `point_in_cell` criterion everywhere. **Gate:** stand in the cottage doorway — the cell does NOT ping-pong (`[cell-transit]` DELTA=0 standing still, no `0031↔0170↔0171`); walk in/out is a clean monotonic cell sequence. +- **P2 — Door/building-shell collision (B3/B4).** Fix the push-back bounce (the 3 failing Core door tests go green). **Gate:** stand in the doorway — no position oscillation (foot Y stable); walk through cleanly; walls block. +- **P3 — Camera viewer-cell (C1/C3).** Port `find_visible_child_cell` + the faithful `update_viewer` start-cell/fallbacks. **Gate:** `viewerCell` is stable + correct as the camera orbits across boundaries (no `[flap-cam]` thrash). +- **P4 — PView render (D2–D9), the core.** Replace `PortalVisibilityBuilder`/`ProjectToNdc`/`ScreenPolygonClip` with `ConstructView`/`InitCell`/`ClipPortals`/`GetClip`/`AddViewToPortals` + `portal_view_type`/`update_count`; re-port `DrawCells`' seal verbatim. **Gate:** cottage interior sealed (opaque walls, no transparent/flap, no void), sky/terrain through the door only. +- **P5 — Outside-looking-in (D8).** `DrawPortal` + `ConstructView(CBldPortal)`. **Gate:** from the street the interior renders through the door (no see-through box). +- **P6 — Dungeons + cleanup (D11/D12).** Validate the all-EnvCell path (`seen_outside==0`, watermark converges, #95 closed). Delete all dead hybrids (§1 DELETE list). **Gate:** a real dungeon sealed, no terrain/sky, no FPS collapse. + +Each phase: grep-named → pseudocode → port verbatim → P0 conformance test green → `dotnet build`/`dotnet test` → **user visual gate**. No phase batched past its gate. + +--- + +## 4. No-shortcuts rules (enforced every task) +1. Every ported behavior cites its decomp anchor (address + pc:line) in a comment. +2. No suppression flags, grace periods, stickiness, or `if (problem) return` guards. If retail doesn't do it, we don't. +3. When retail's behavior is unclear, read the decomp / attach cdb (Step -1) — never guess. +4. A faithful port that breaks an acdream test means the acdream test encoded a hybrid assumption — fix the test to the retail truth, don't bend the port. +5. Each phase ends GREEN + at a user visual gate; the seal/membership is verified on screen + probes, never off the unit suite alone. + +## 5. References (read before each half) +- Render half spine: [`../../research/2026-06-02-retail-render-pipeline-full-reference.md`](../../research/2026-06-02-retail-render-pipeline-full-reference.md) (CL-A…CL-G). +- Membership: [`../../research/2026-06-03-membership-and-bluehole-shipped-handoff.md`](../../research/2026-06-03-membership-and-bluehole-shipped-handoff.md) + the four 2026-06-02 decomp studies. +- The V1 single-viewpoint win (keep): [`2026-06-03-single-viewpoint-render-design.md`](2026-06-03-single-viewpoint-render-design.md). +- Oracle: `docs/research/named-retail/acclient_2013_pseudo_c.txt` + `acclient.h`; live trace via cdb (CLAUDE.md "Retail debugger toolchain"). + +--- + +## 6. KICKOFF PROMPT (copy-paste for the execution session) + +``` +VERBATIM PORT of the retail spatial pipeline — membership + collision + camera + render. NO hybrids, +NO bandaids; full retail-faithful behavior; breaking intermediate states is fine. The doorway saga +(void → transparent walls → flaps) proved patching the hybrid is hopeless: retail does membership + +collision + camera + render as ONE coupled pipeline and acdream reimplemented pieces of each with +mismatched criteria at the seams. Branch: claude/thirsty-goldberg-51bb9b (do NOT branch/worktree; do +NOT push without asking; NEVER git stash/gc). PowerShell on Windows; launch logs are UTF-16. + +READ FIRST: +1. docs/superpowers/specs/2026-06-03-verbatim-spatial-pipeline-port-master-plan.md (THIS plan — scope + §2, KEEP/REPLACE/DELETE §1, sequence §3, no-shortcuts §4). +2. docs/research/2026-06-02-retail-render-pipeline-full-reference.md (the PView decomp + CL-A…CL-G). +3. The V1 single-viewpoint design (keep that invariant): render on the viewer cell, lighting on the + player. Already shipped this session (commits 832001d / d03fe84 / 1e9a9ca) + the void near-clip + fix (0cc561c, which P4's GetClip subsumes). + +THE JOB — port EVERYTHING in §2 verbatim, bottom-up per §3: P0 conformance apparatus → P1 membership + +uniform collision → P2 door collision → P3 camera viewer-cell → P4 PView render (the core) → P5 +outside-looking-in → P6 dungeons + delete the hybrids (§1). Workflow per task: grep-named → pseudocode +→ port verbatim (cite the anchor) → P0 conformance test → build/test → USER VISUAL GATE. Use +superpowers:writing-plans to turn each phase into a TDD plan; superpowers:executing-plans to run it. + +DELETE (once replaced): PortalVisibilityBuilder, PortalProjection, ScreenPolygonClip, CellView/ClipFrame; +CheckBuildingTransit, the forked FindEnvCollisions branches, ResolveCellId's ad-hoc outdoor branch + #90 +stickiness. KEEP: WB mesh pipeline + EnvCellRenderer mesh/MDI + TerrainModernRenderer + SkyRenderer (GL +draw primitives), the Transition/SpherePath/BSPQuery engine core, DatCollection, the V1 viewer-keying. + +START AT P0 (the conformance apparatus) — it is how we prove "verbatim" instead of guessing. Do NOT +start P1 code before P0 fixtures + at least one golden retail-trace assertion exist. + +TEST BASELINE going in: Core 1295 pass / 5 fail (2 BSPStepUp + 3 door-collision — the door ones are +P2's target); App 177 green. +```