acdream/docs/superpowers/specs/2026-06-03-verbatim-spatial-pipeline-port-master-plan.md
Erik a859116d5f docs(spatial): master plan — VERBATIM port of the retail spatial pipeline (no hybrids)
The doorway saga (void -> transparent walls -> flaps) proved patching the hybrid is hopeless:
retail does membership + collision + camera + render as ONE coupled pipeline; acdream
reimplemented pieces with mismatched criteria at the seams. Master plan to port ALL of it
verbatim: A membership (find_cell_list/find_transit_cells/find_building_transit_cells intrinsic,
no bridge), B uniform collision (no indoor/outdoor fork) + door collision, C camera
(update_viewer + find_visible_child_cell), D the full PView render (ConstructView/InitCell/
ClipPortals/GetClip/DrawCells/DrawPortal + the update_count watermark). KEEP/REPLACE/DELETE
lists, decomp anchors per function, P0-P6 sequence (apparatus-first, foundation-up, visual gate
each), and the kickoff prompt. Supersedes the render-only redesign's scope.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 13:57:25 +02:00

17 KiB
Raw Blame History

Master Plan — VERBATIM port of the retail spatial pipeline (membership + collision + camera + render)

Mandate (user, 2026-06-03): "No hybrids, no bandaids. I want FULLY ported behavior. Don't be afraid to break stuff. Port everything now." The doorway saga (void → transparent walls → flaps) proved that patching the hybrid produces a new break for every fix. Stop patching. Port retail's integrated spatial pipeline verbatim from docs/research/named-retail/acclient_2013_pseudo_c.txt

  • acclient.h. The code is modern C#; the behavior is retail, line-for-line. Breaking intermediate states is acceptable — correctness-by-construction is the goal, not incremental safety.

This supersedes the scope of the prior render-only redesign (2026-06-02-render-pipeline-redesign-design.md) by adding the membership + collision + camera halves that the render rides on. The retail-pipeline reference ../../research/2026-06-02-retail-render-pipeline-full-reference.md (the PView decomp + the CL-A…CL-G checklist) is the render half's spine; this doc is the superset.


0. Why a hybrid can't work here (the root lesson)

Retail makes one decision flow, shared across physics and render:

sphere sweep (CTransition) carries curr_cell THROUGH collision over a UNIFORM candidate set
   → membership = find_cell_list pick over that set (one point_in_cell criterion)
   → the camera runs its OWN sweep → viewer_cell
   → render roots at viewer_cell, floods portals (PView), seals (DrawCells)

acdream split this into independent reimplementations that use different criteria at the seams:

  • membership exit uses point_in_cell (center) but entry uses SphereIntersectsCellBsp (overlap) → a hysteresis gap → threshold ping-pong (0031↔0170↔0171);
  • collision forks indoor-cell-BSP vs outdoor-terrain → the candidate set isn't uniform → push-back bounce at the doorway;
  • the render PVS is a home-grown ProjectToNdc/ScreenPolygonClip that approximates GetClip → near-the-portal projection blows up → void / transparent walls / flaps.

Every patch moves the failure from one seam to the next. Only a verbatim port closes all seams at once, because retail's correctness is structural (one criterion, one candidate set, one viewpoint, one flood), not tuned.


1. KEEP / REPLACE / DELETE (be precise about the blast radius)

KEEP (infrastructure — not the algorithm):

  • The WB mesh pipeline: global VAO/VBO/IBO, ObjectMeshManager, WbMeshAdapter, GfxObj/Setup decode, TextureCache, bindless/MDI. (Geometry storage + upload.)
  • EnvCellRenderer's mesh/MDI/texture path (the actual cell-shell draw), TerrainModernRenderer, SkyRenderer, ParticleRenderer — the GL draw primitives.
  • DatCollection + the dat readers; PhysicsDataCache as the cache (its CONTENTS get the new graph).
  • The Transition/SpherePath/CollisionInfo/ObjectInfo/BSPQuery engine core (the sphere-sweep math, the 6-path BSP dispatcher) — it is already a faithful port; the GAP is how it's driven (forked vs uniform).
  • The V1 win: render keys on the viewer (camera), lighting on the player — that invariant is retail and stays.

REPLACE (the algorithm, verbatim):

  • Membership driver: CellTransit.BuildCellSetAndPickContaining + FindCellList/FindCellSet → faithful find_cell_list + find_transit_cells (with intrinsic building entry).
  • PhysicsEngine.ResolveCellId → demoted to spawn/teleport seed only; per-frame membership comes from the swept curr_cell (already true post-Stage-1, finish it).
  • Collision driver: Transition.FindEnvCollisions forked indoor/outdoor branches → one uniform find_env_collisions over the candidate set (land cells sweep terrain tris; env cells sweep the cell BSP) — same loop, per cell.
  • Render PVS: PortalVisibilityBuilder + PortalProjection.ProjectToNdc + ScreenPolygonClip + the CellView/ClipFrame NDC model → retail PView (InitCell/ClipPortals/GetClip/AddViewToPortals/AddToCell/DrawCells) + portal_view_type/view_type/update_count.
  • Camera viewer-cell: the sweep-CurCellId approximation → find_visible_child_cell (graph/BSP) seeded at the player cell, per retail update_viewer.

DELETE (the bandaids — once their faithful replacement lands):

  • CellTransit.CheckBuildingTransit (the building-entry bridge, #5).
  • ResolveCellId's indoor SphereIntersectsCellBsp verify + the ad-hoc outdoor CheckBuildingTransit branch + any remaining #90 stickiness.
  • PortalVisibilityBuilder, PortalProjection, ScreenPolygonClip, ClipFrame/ClipFrameAssembler/CellView/PortalView (the NDC clip model) — replaced by view_type/portal_view + stencil/scissor per retail DrawCells.
  • MinW near-clip approximation (this session) — subsumed by GetClip's real near-clipping.
  • The dormant WB-two-pipe scaffolding (Building/BuildingLoader stencil, occlusion-query, IsShellScopedSet) — already mostly dead.

2. Scope — "EVERYTHING," enumerated with decomp anchors

Anchors are Class::method @ 0xADDR (pc:LINE) in docs/research/named-retail/acclient_2013_pseudo_c.txt; structs are acclient.h:LINE. All verified this session or in the full-reference doc. Port each verbatim (same control flow, same predicates, same constants); write pseudocode first (workflow step 3) for the gnarly ones (ClipPortals, DrawCells, find_transit_cells).

A. Membership — the shared cell graph (physics owns the cell)

# Retail Anchor acdream now
A1 CObjCell::find_cell_list (set build + interior-wins pick) 0x52b4e0 pc:308742 partial (Stage 1 pick); finish set-build
A2 CEnvCell::find_transit_cells (portal crossing → neighbors; exit→add_all_outside; building portals intrinsic) 0x52c820 pc:309968 hybrid (CheckBuildingTransit bridge)
A3 CEnvCell::check_building_transit / find_building_transit_cells (intrinsic entry) 0x52c5d0 pc:309827 / pc:318309 bridge (#5)
A4 CLandCell::add_all_outside_cells 0x533630 pc:317499 present (verify coord-convention)
A5 CObjCell::GetVisible / CEnvCell::GetVisible (graph resolve, ≥0x100 split) 0x52ad40 pc:308209 / 0x52dc10 pc:311378 present (CellGraph)
A6 point_in_cell (CEnvCell cell-BSP, CLandCell XY) — the ONE containment criterion (vtable 0x84) present (PointInsideCellBsp); make it the sole criterion
A7 CTransition::transitional_insert / validate_transition / check_other_cells (collide-then-pick) 0x50aa70 pc:272547 / 0x50ae50 pc:272717 partial (Stage 1 RunCheckOtherCellsAndAdvance)
A8 CPhysicsObj::SetPositionInternal / change_cell (commit on diff) 0x515330 pc:283399 / 0x513390 pc:281192 present (UpdateCellId)
A9 enter_cell / leave_cell (per-cell object_list + shadow_object_list — the shared graph) 0x510ed0 / 0x510f50 acdream uses a landblock-wide ShadowObjectRegistry — port per-cell lists

B. Collision — one uniform sphere-sweep (no fork)

# Retail Anchor acdream now
B1 CObjCell::find_env_collisions (CLandCell sweeps terrain tris; CEnvCell sweeps cell BSP) — uniform per candidate cell CLandCell + CEnvCell::find_env_collisions 0x52c130 pc:309573 forked cellLow>=0x0100 branch (#4)
B2 BSPTREE::find_collisions (the 6-path dispatcher) 0x323924 present (BSPQuery) — keep
B3 CPhysicsObj::FindObjCollisions + binary BSP-vs-cyl dispatch pc:274435+ partial (A6.P7/P8)
B4 door / building-shell collision (the push-back bounce at the threshold) CBuildingObj/CBldPortal collision path buggy (3 failing Core door tests, #97)
B5 pos_hits_sphere / polygon_hits_sphere_slow_but_sure (per-poly static tests) pc:322974 present (A6.P4)

C. Camera / viewer (the ONE viewpoint)

# Retail Anchor acdream now
C1 SmartBox::update_viewer (spring-arm sweep → viewer + viewer_cell = sphere_path.curr_cell; AdjustPosition fallbacks; snap-to-player) 0x453ce0 pc:92761 V1 ported the sweep + ViewerCellId; ADD the AdjustPosition fallbacks + faithful start-cell
C2 CameraManager::UpdateCamera (desired eye = viewer_sought_position) 0x456660 RetailChaseCamera reimplements (damping) — KEEP, it feeds C1
C3 CEnvCell::find_visible_child_cell (viewer's child cell via portals/stab_list, NOT the sweep approximation, NOT AABB) 0x52dc50 pc:311397 not ported (uses sweep CurCellId)
C4 CameraSet::UpdateCamera player-fade 0x458ae0 pc:97703 ported (ComputeTranslucency) — keep

D. Render — the full PView (the big one)

# Retail Anchor acdream now
D1 SmartBox::RenderNormalMode (binary decision on viewer cell → DrawInside(viewer_cell) vs LScape::draw) 0x453aa0 pc:92635 V1 ported the keying — keep, wire to PView
D2 PView::DrawInsideConstructView(CEnvCell) (the BFS) 0x5a5860 pc:433793 / 0x5a57b0 pc:433750 PortalVisibilityBuilder (REPLACE)
D3 PView::InitCell (per-portal sidedness vs viewer.viewpoint; update_count = view_count) 0x5a4b70 pc:432896 CameraOnInteriorSide (REPLACE)
D4 PView::ClipPortals (exit→outside_view; interior→OtherPortalClip into neighbor) 0x5a5520 pc:433572 inline in builder (REPLACE)
D5 PView::GetClip (portal poly → screen clip, proper near-clipping, honor Sidedness) 0x5a4320 pc:432344 ProjectToNdc+ScreenPolygonClip (REPLACE — this is the void/flap source)
D6 PView::AddViewToPortals / AddToCell / SetOtherSeen / InsCellTodoList (worklist + watermark) 0x5a52d0 pc:433446 / 0x5a4d90 pc:433050 / 0x5a4e30 / 0x5a4f50 HashSet-seen BFS (REPLACE)
D7 PView::DrawCells (LScape-thru-door + conditional Z-clear + 3 per-cell loops: stencil exit portals, draw closed mesh, draw per-cell objects/particles) 0x5a4840 pc:432709 InteriorRenderer per-cell loop (PARTIAL — re-port the seal verbatim)
D8 PView::ConstructView(CBldPortal) / DrawPortal (outside-looking-in) 0x5a59a0 pc:433827 / 0x5a5ab0 pc:433895 not built (residual C)
D9 portal_view_type / view_type / update_count watermark (per-cell view accumulation; #95/#102) acclient.h:32346 / 32338 CellView/ClipFrame NDC model (REPLACE)
D10 CellManager::ChangePosition (keep/release landscape + sunlight on seen_outside of the PLAYER cell) 0x4559b0 pc:94601 V1 lighting-on-player — keep, verify against this
D11 CEnvCell::grab_visible_cells (load landscape iff seen_outside) 0x52e220 pc:311878 partial
D12 LScape::draw (terrain+sky, clipped via Render::PortalList) 0x506330 acdream splits terrain/sky/scenery — clip each to outside_view per D7

3. Sequence (foundation-up; each phase ends at a user visual gate)

The render rides on stable membership + a stable viewer. Port bottom-up so each phase has a solid base. P0 first because it's the apparatus that makes "verbatim" checkable.

  • P0 — Conformance apparatus (before any port). Headless fixtures of the Holtburg cottage neighborhood (cells 0xA9B4003x + 0xA9B4017x, the building stab, the cellar) loaded from the real dats. Golden tests that assert retail outcomes: find_cell_list returns the same cell as a captured retail trace at the threshold; point_in_cell matches; the PVS visible-set for a given (cell, eye) matches. Use the existing ACDREAM_CAPTURE_RESOLVE + cdb retail traces. This is how we know a port is verbatim, not vibes.
  • P1 — Membership (A) + uniform collision (B1). Port find_cell_list/find_transit_cells/find_building_transit_cells/add_all_outside_cells intrinsic; delete CheckBuildingTransit. Port uniform find_env_collisions (no fork). One point_in_cell criterion everywhere. Gate: stand in the cottage doorway — the cell does NOT ping-pong ([cell-transit] DELTA=0 standing still, no 0031↔0170↔0171); walk in/out is a clean monotonic cell sequence.
  • P2 — Door/building-shell collision (B3/B4). Fix the push-back bounce (the 3 failing Core door tests go green). Gate: stand in the doorway — no position oscillation (foot Y stable); walk through cleanly; walls block.
  • P3 — Camera viewer-cell (C1/C3). Port find_visible_child_cell + the faithful update_viewer start-cell/fallbacks. Gate: viewerCell is stable + correct as the camera orbits across boundaries (no [flap-cam] thrash).
  • P4 — PView render (D2D9), the core. Replace PortalVisibilityBuilder/ProjectToNdc/ScreenPolygonClip with ConstructView/InitCell/ClipPortals/GetClip/AddViewToPortals + portal_view_type/update_count; re-port DrawCells' seal verbatim. Gate: cottage interior sealed (opaque walls, no transparent/flap, no void), sky/terrain through the door only.
  • P5 — Outside-looking-in (D8). DrawPortal + ConstructView(CBldPortal). Gate: from the street the interior renders through the door (no see-through box).
  • P6 — Dungeons + cleanup (D11/D12). Validate the all-EnvCell path (seen_outside==0, watermark converges, #95 closed). Delete all dead hybrids (§1 DELETE list). Gate: a real dungeon sealed, no terrain/sky, no FPS collapse.

Each phase: grep-named → pseudocode → port verbatim → P0 conformance test green → dotnet build/dotnet testuser visual gate. No phase batched past its gate.


4. No-shortcuts rules (enforced every task)

  1. Every ported behavior cites its decomp anchor (address + pc:line) in a comment.
  2. No suppression flags, grace periods, stickiness, or if (problem) return guards. If retail doesn't do it, we don't.
  3. When retail's behavior is unclear, read the decomp / attach cdb (Step -1) — never guess.
  4. A faithful port that breaks an acdream test means the acdream test encoded a hybrid assumption — fix the test to the retail truth, don't bend the port.
  5. Each phase ends GREEN + at a user visual gate; the seal/membership is verified on screen + probes, never off the unit suite alone.

5. References (read before each half)


6. KICKOFF PROMPT (copy-paste for the execution session)

VERBATIM PORT of the retail spatial pipeline — membership + collision + camera + render. NO hybrids,
NO bandaids; full retail-faithful behavior; breaking intermediate states is fine. The doorway saga
(void → transparent walls → flaps) proved patching the hybrid is hopeless: retail does membership +
collision + camera + render as ONE coupled pipeline and acdream reimplemented pieces of each with
mismatched criteria at the seams. Branch: claude/thirsty-goldberg-51bb9b (do NOT branch/worktree; do
NOT push without asking; NEVER git stash/gc). PowerShell on Windows; launch logs are UTF-16.

READ FIRST:
1. docs/superpowers/specs/2026-06-03-verbatim-spatial-pipeline-port-master-plan.md  (THIS plan — scope
   §2, KEEP/REPLACE/DELETE §1, sequence §3, no-shortcuts §4).
2. docs/research/2026-06-02-retail-render-pipeline-full-reference.md  (the PView decomp + CL-A…CL-G).
3. The V1 single-viewpoint design (keep that invariant): render on the viewer cell, lighting on the
   player. Already shipped this session (commits 832001d / d03fe84 / 1e9a9ca) + the void near-clip
   fix (0cc561c, which P4's GetClip subsumes).

THE JOB — port EVERYTHING in §2 verbatim, bottom-up per §3: P0 conformance apparatus → P1 membership +
uniform collision → P2 door collision → P3 camera viewer-cell → P4 PView render (the core) → P5
outside-looking-in → P6 dungeons + delete the hybrids (§1). Workflow per task: grep-named → pseudocode
→ port verbatim (cite the anchor) → P0 conformance test → build/test → USER VISUAL GATE. Use
superpowers:writing-plans to turn each phase into a TDD plan; superpowers:executing-plans to run it.

DELETE (once replaced): PortalVisibilityBuilder, PortalProjection, ScreenPolygonClip, CellView/ClipFrame;
CheckBuildingTransit, the forked FindEnvCollisions branches, ResolveCellId's ad-hoc outdoor branch + #90
stickiness. KEEP: WB mesh pipeline + EnvCellRenderer mesh/MDI + TerrainModernRenderer + SkyRenderer (GL
draw primitives), the Transition/SpherePath/BSPQuery engine core, DatCollection, the V1 viewer-keying.

START AT P0 (the conformance apparatus) — it is how we prove "verbatim" instead of guessing. Do NOT
start P1 code before P0 fixtures + at least one golden retail-trace assertion exist.

TEST BASELINE going in: Core 1295 pass / 5 fail (2 BSPStepUp + 3 door-collision — the door ones are
P2's target); App 177 green.