From e8c7164ad9ffa679716b4ee1eb64db27bef38647 Mon Sep 17 00:00:00 2001 From: Erik Date: Tue, 2 Jun 2026 08:27:00 +0200 Subject: [PATCH] =?UTF-8?q?docs(render):=20Unified=20Cell=20Graph=20pivot?= =?UTF-8?q?=20=E2=80=94=20evidence=20model=20+=20Stage=201=20spec?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pixel-grounded investigation concluded the indoor 'world from below' is a cell-MEMBERSHIP disagreement between render-side CellVisibility and physics-side ResolveCellId, not any single draw gate (terrain has one gated draw path; it leaks only on render null-root frames). Decision with user: full migration onto one retail CObjCell graph across physics+collision+render+streaming, staged in 5 verify-each cycles. This lands the evidence model + the Stage 1 (ObjCell scaffold) design. No code yet. - docs/research/2026-06-02-render-cell-membership-evidence.md (the why, from pixels) - docs/superpowers/specs/2026-06-02-unified-cell-graph-stage1-design.md (Stage 1) Co-Authored-By: Claude Opus 4.8 (1M context) --- ...6-06-02-render-cell-membership-evidence.md | 120 +++++++++ ...-06-02-unified-cell-graph-stage1-design.md | 227 ++++++++++++++++++ 2 files changed, 347 insertions(+) create mode 100644 docs/research/2026-06-02-render-cell-membership-evidence.md create mode 100644 docs/superpowers/specs/2026-06-02-unified-cell-graph-stage1-design.md diff --git a/docs/research/2026-06-02-render-cell-membership-evidence.md b/docs/research/2026-06-02-render-cell-membership-evidence.md new file mode 100644 index 0000000..d3c47c4 --- /dev/null +++ b/docs/research/2026-06-02-render-cell-membership-evidence.md @@ -0,0 +1,120 @@ +# Render Cell-Membership — evidence model (2026-06-02) + +> Canonical "why" for the **Unified Cell Graph (CObjCell)** pivot. This is the +> pixel-grounded investigation that ended the week-long indoor-render saga. +> Companion design spec: `docs/superpowers/specs/2026-06-02-unified-cell-graph-stage1-design.md`. +> Predecessors: `docs/research/2026-05-31-render-architecture-reset-handoff.md`, +> `docs/research/2026-06-01-render-reset-session2-handoff.md`. + +## 0. One-line conclusion + +The render pipeline maintains its **own** indoor-cell system (`CellVisibility` + +`PortalVisibilityBuilder` + 3 gates) that is **separate from, and out of sync with, +physics cell tracking**. When the render side's answer is "outdoor/null" — which +happens on spawn-in, on within-building flicker, and on building entry — terrain +draws ungated and you see the **world from below**. Point-fixing any one gate cannot +win because the *visibility input* is unreliable and disagrees with physics. The fix +is to unify onto **one cell membership** all systems share (the retail `CObjCell` +model), executed as a staged migration (see spec). + +## 1. Symptom + +Standing inside the Holtburg cottage (room or cellar) the interior does not seal: +the full outdoor world (terrain hills, water, scenery) renders and the player +appears to float in it ("world from below"). The exterior renders correctly. + +## 2. Method + +Live client against local ACE, four launches, character `+Acdream`. Existing +baseline probes only — **no code added** (investigation was report-only): +`ACDREAM_PROBE_SHELL` (`[shell]`), `ACDREAM_PROBE_VIS` (`[vis]`), +`ACDREAM_PROBE_FLAP` (`[flap]`/`[flap-cam]`), `ACDREAM_PROBE_ENVCELL`, +`ACDREAM_PROBE_CELL` (`[cell-transit]`). Screenshots via PowerShell +`CopyFromScreen` on the `AcDream.App` window → PNG → read. + +**Gotcha (recorded so it isn't re-hit):** `Tee-Object` writes the log as **UTF-16LE**. +GNU `grep` (Bash) silently matches nothing on it; the ripgrep-based Grep tool and +PowerShell `Get-Content` decode it correctly. Early "0 probes" Bash reads were false +negatives — always read these logs with ripgrep/PowerShell. + +## 3. Findings (per waypoint) + +1. **Exterior — correct.** Opaque textured cottage walls, building sitting on dirt, + no holes. No indoor probes fire (outdoor root). The outdoor case is sound and the + migration must preserve it. +2. **Room, entered from outside (run 1) — indoor path NEVER engaged.** + `[vis]=[shell]=[envcells]=0`. The "interior" we saw was the building-shell GfxObj + + interior statics drawn by the *outdoor* entity pass; dark void where nothing covered. + ⇒ **outdoor→indoor building entry did not transit the player into the cell.** +3. **Cellar, walked in (run 1) — indoor path engaged but leaks.** + `[vis] root=0xA9B40174 cells=3 ids=[0174,0175,0171] outside(polys=0,planes=0)` — + visibility computed correctly (windowless ⇒ empty OutsideView). `[shell]` shows all + three shells render fine (`gfx=1 tr=0 zh=0`). `[flap-cam] terrain=Skip outVisible=False` + — terrain *decision* is correct. Yet the screenshot is the full outdoor world. +4. **First-person test (user-confirmed) — terrain shows even with the eye inside the + cell.** ⇒ the leak is **not** the 3rd-person camera; it's the gate/root. +5. **Spawn directly into the cellar (run 3) — the cleanest proof.** Only one + `[cell-transit] 0x0 → 0xA9B40174 reason=teleport`. **Physics knew the indoor cell + instantly.** Render fired **zero** `[vis]/[shell]` while standing still ⇒ world from + below. ⇒ **render and physics disagree: physics indoor, render outdoor.** +6. **Move from the cellar (run 3) — render re-engages on motion.** `[cell-transit]` + flickers `0174↔0175↔0171` (`reason=resolver`); `[vis]`/`[shell]` start firing. + `[vis] root=0xA9B40174 cells=1` vs `root=0xA9B40175 cells=3` — the portal walk from + `0174` reaches **only itself** (incomplete traversal); the root flickers, so the + sealed set flickers 1↔3 and gaps show. +7. **Room from the cellar side (run 3).** The cottage interior is **one connected + EnvCell group: `0171`(room/Z94) ↔ `0175`(stairs/Z93) ↔ `0174`(cellar/Z90)**; the + cell flickers among the three while standing still. Render engaged (came up through + the graph) yet still showed terrain at room level. + +## 4. Causal model + +- **Terrain has exactly one draw path** (`_terrain.Draw`, `GameWindow.cs:7415/7420`), + both call sites behind the `TerrainClipMode` gate; **no second terrain/water/skybox + source exists** (code-confirmed). So terrain can only appear when `terrainClipMode + != Skip`, which happens when the render **root resolves outdoor** (`clipRoot==null` + → the `else` branch at `GameWindow.cs:7352` → default `Planes` → ungated terrain). +- **Two independent cell resolvers.** Render root = `CellVisibility.ComputeVisibility + (visRootPos)` → `FindCameraCell` (AABB `PointInCell`, render-side registry). + Physics = `PhysicsEngine.ResolveCellId` → `PlayerMovementController.CellId` + (BSP-based, separate registry). They disagree. +- **Render registration lags/misses.** Render cells enter `CellVisibility` via + `_pendingCells` → render-thread `AddCell` (`GameWindow.cs:5749`), a *different + lifecycle* from physics' worker-thread `CacheCellStruct`. On spawn-in the render + side has nothing registered yet ⇒ `clipRoot==null` ⇒ terrain. +- **The `[flap-cam]` probe can't see the failure frames** — it only logs inside + `if (clipRoot is not null)`. The `null`-root frames (where terrain draws) are + invisible to it, which is why prior log-archaeology missed this. +- **Two failure variants, one disease:** building **entry** (outdoor→indoor) leaves + you outdoor (finding 2); **within-building** the membership flickers and the portal + walk is incomplete (findings 6–7). Both are "cell membership recomputed from + geometry, unreliable, and disagreeing across systems." + +## 5. Verdict → the pivot + +Unify onto **one cell membership** every system shares — the retail `CObjCell` model +(one `curr_cell`, `GetVisible` magnitude dispatch, portal graph). Scope decided with +the user (2026-06-02): **full migration** across physics + collision + render + +streaming, freeze lifted as needed, executed as a 5-stage chain that keeps the client +runnable + visually verified at each step. Design: the companion Stage 1 spec. + +## 6. Disproven / do-not-repeat (this session + predecessors) + +- Camera/eye as the root cause (FP test: terrain shows with eye inside the cell). +- A second terrain renderer (code-confirmed there is none). +- "Render-gate consolidation only" as sufficient (the broken input is the *membership*, + upstream of the gate). +- Earlier disproven: H1 PVS grounding, H2 PortalSide, cull mode, shell + geometry/texture missing, zoom-confound, the screen-space stencil-mask rule + (breaks outdoors), flag-based per-entity gate routing (= patchwork). + +## 7. Grounding research (full reports archived in session transcript) + +Two research agents produced the retail structure + acdream inventory that ground the +design; their key facts are folded into the spec's "Retail grounding" and "acdream +current state" sections with citations. Decomp anchors: +`CObjCell` (`acclient.h:30915`), `CEnvCell` (`:32072`), `CLandCell` (`:31886`), +`CSortCell` (`:31880`), `CCellPortal` (`:32300`), `CBldPortal` (`:32094`), +`CellStruct` (`:32275`); `GetVisible` (`pseudo_c:308209`), `change_cell` (`:281192`), +`find_cell_list` (`:308742`), `ConstructView` (`:433750`/`:433827`), +`InitCell` (`:432896`), `find_visible_child_cell` (`:311397`). diff --git a/docs/superpowers/specs/2026-06-02-unified-cell-graph-stage1-design.md b/docs/superpowers/specs/2026-06-02-unified-cell-graph-stage1-design.md new file mode 100644 index 0000000..6b1d7ff --- /dev/null +++ b/docs/superpowers/specs/2026-06-02-unified-cell-graph-stage1-design.md @@ -0,0 +1,227 @@ +# Unified Cell Graph (CObjCell) — Stage 1: ObjCell Scaffold + +**Status:** design approved 2026-06-02 (brainstorm). Implementation pending plan. +**Branch:** `claude/thirsty-goldberg-51bb9b` (unpushed). +**Evidence:** `docs/research/2026-06-02-render-cell-membership-evidence.md`. +**Supersedes:** the abandoned A8 two-pipe (issue #103) and the render-only framing of +"Phase U"; expands that intent to the full retail cell model. +**Roadmap:** register this program as a phase in `docs/plans/2026-04-11-roadmap.md` +when Stage 1 implementation begins (id TBD with user — referred to here as "UCG"). + +--- + +## 1. Why (motivation) + +acdream renders the world through **two independent cell systems** that disagree: +a render-side one (`CellVisibility` + `PortalVisibilityBuilder` + 3 inconsistent +draw gates) and a physics-side one (`PhysicsEngine.ResolveCellId` + +`PhysicsDataCache`). Standing inside the Holtburg cottage, the interior fails to +seal and the outdoor world renders ("world from below"). Pixel-grounded evidence +(see evidence doc) proved the root cause is **membership disagreement**, not any +single gate: physics knows you're in indoor cell `0xA9B40174` the instant you spawn, +while the render side fires zero visibility probes; terrain (which has exactly one, +gated draw path) leaks only on frames where the render root resolves outdoor/null. + +Retail has no such split. It has **one** abstraction — `CObjCell` — with a single +`curr_cell` per object; outdoor terrain cells (`CLandCell`) and indoor rooms +(`CEnvCell`) are the same kind of thing in one graph, and crossing a doorway is a +pointer swap, not a mode switch. **Seamlessness is the absence of a second system.** + +## 2. The program (full migration, staged) + +Scope decided with the user (2026-06-02): **full migration** onto one `CObjCell` +graph across physics + collision + render + streaming; the M0 freeze is lifted for +whatever this needs. Executed as an ordered chain — each stage its own +spec→plan→implement→verify cycle, each leaving the client runnable + visually +verified, never one long broken stretch: + +| Stage | What | Verify | Risk | +|---|---|---|---| +| **1 — `ObjCell` scaffold (THIS spec)** | New Core model + graph + `GetVisible`, built alongside today's systems, consumed by nobody. | Builds green; **zero behavior change**; Core unit tests. | Low | +| 2 — One membership | Player `curr_cell` via retail `find_cell_list` + `change_cell` + doorway hysteresis; collapses `ResolveCellId` and `FindCameraCell` into one answer. | No inn-doorway ping-pong. | Med | +| 3 — Render on the graph | PView walk from `curr_cell`; **one gate** for terrain/shells/entities; bounded traversal. Retires `CellVisibility`/`LoadedCell` + the 3 gates. | Cottage + dungeon seal (the M1.5 win). | Med | +| 4 — Collision on the graph | Physics collision queries the same `ObjCell`s; retire parallel `CellPhysics`. | A6 conformance set green. | High | +| 5 — Streaming → `ObjCell`s | Streaming loads/unloads cells uniformly; terrain becomes `LandCell`s; wrappers removed. | Streaming + N.6 perf baseline hold. | High | + +The visible indoor fix lands at **Stage 3**; Stages 4–5 finish the foundation so the +seam is gone by construction. + +## 3. Retail grounding (the model we port) + +From the decomp survey (anchors cited). Hierarchy: +`CObjCell → CSortCell → CLandCell` and `CObjCell → CEnvCell`. + +- **`CObjCell`** (`acclient.h:30915`): id (`m_DID.id` — **magnitude is the type + discriminator**: `<0x100` land, `≥0x100` env), `pos`, `object_list`, + `shadow_object_list` (the per-cell **collision** membership list — Stage 4), + `stab_list` (visibility-reachable ids; doubles as the hysteresis keep-set), + `seen_outside`, `myLandBlock_`. Vtable seams: `+0x80 find_transit_cells`, + `+0x84 point_in_cell`, `+0x88 find_collisions`. +- **`CSortCell`** (`acclient.h:31880`): adds `building` — the outdoor→indoor bridge. +- **`CLandCell`** (`acclient.h:31886`): `polygons`, `in_view`. Resolved positionally + via `LScape::get_landcell`. +- **`CEnvCell`** (`acclient.h:32072`): `structure` (a `CellStruct`, `:32275`, holding + vertices + **three distinct BSPs**: drawing / physics / cell-containment), + `portals` (`CCellPortal[]`), `static_objects`, `portal_view` (per-frame recursive + clip stack — Stage 3). Resolved via a hash table keyed by id. +- **`CCellPortal`** (`acclient.h:32300`): `other_cell_id`, `other_cell_ptr` (cache), + `portal` (the doorway polygon/plane), `portal_side`, `other_portal_id`, + `exact_match`. `GetOtherCell` = `CEnvCell::GetVisible(other_cell_id)`. +- **`CBldPortal`** (`acclient.h:32094`) on `CBuildingObj`: outdoor→interior entry + portals (`other_cell_id`, `other_portal_id`, `stab_list`, `sidedness`). +- **`GetVisible(id)`** (`pseudo_c:308209`): `id==0`→null; `≥0x100`→`CEnvCell` hash; + else→`CLandCell` positional. The one resolver the whole engine uses. +- **`change_cell`** (`pseudo_c:281192`): `leave_cell(old)` then `enter_cell(new)` — + a pointer swap maintaining `object_list` + `PhysicsObj.cell`. (Stage 2.) +- **`find_cell_list`** (`pseudo_c:308742`): position+sphere → cell set. **Anti-ping-pong + is NOT a remembered prev-cell** — it's the persistent `CellArray` (lives on the + `Transition`) + the `do_not_load_cells` prune to `{anchor} ∪ {anchor.stab_list}`. + (Stage 2.) +- **`ConstructView`/`InitCell`** (`pseudo_c:433750`/`433827`/`432896`): recursive + per-portal clipping (root → InitCell builds clip region → for each visible portal, + clip parent view ∩ portal polygon → `GetVisible(other_cell_id)` → push clip onto + neighbor's `portal_view` → recurse). Exit-portal-to-outside via `outdoor_portal_list`. + This is the *real* PView, NOT WB's flat stencil. (Stage 3.) + +## 4. acdream current state (what we wrap/absorb) + +From the inventory (file:line cited). **No `ObjCell` exists today.** Two worlds: + +- **Render:** `LoadedCell` (`src/AcDream.App/Rendering/CellVisibility.cs:24-105`) — + `CellId`, `WorldTransform`/`InverseWorldTransform`, `LocalBoundsMin/Max`, `Portals` + (`CellPortalInfo`: `OtherCellId,PolygonId,Flags,OtherPortalId`), `ClipPlanes`, + `PortalPolygons`, `BuildingId`, `VisibleCells` (stab-list), `SeenOutside`. Stored in + `_cellLookup` (full id) + `_cellsByLandblock` (upper-16). **Closest match to retail + `CEnvCell`** but lives in the App layer. +- **Physics:** `CellPhysics` (`src/AcDream.Core/Physics/PhysicsDataCache.cs:511-564`) — + `BSP`, `PhysicsPolygons`, `CellBSP` (`DatReaderWriter.Types.CellBSPTree`), `Portals` + (`PortalInfo` — **no `OtherPortalId`**), `PortalPolygons`, transforms. Has the BSPs, + lacks bounds/`SeenOutside`/`OtherPortalId`. Plus `BuildingPhysics`+`BldPortalInfo` + (`BuildingPhysics.cs`) and legacy `CellSurface` (`CellSurface.cs` — to retire). +- **Terrain:** no per-landcell object. `TerrainSurface` + (`src/AcDream.Core/Physics/TerrainSurface.cs`) is **one per landblock** (8×8 cells) + + `ComputeOutdoorCellId(localX,localY)` (`:510`). A `LandCell` must be **synthesized**. +- **Membership:** `PlayerMovementController.CellId` (bare `uint`, + `PlayerMovementController.cs:133`) re-resolved each tick by + `PhysicsEngine.ResolveCellId` (`PhysicsEngine.cs:253`), branching at `0x0100`. +- **Registration divergence:** render `LoadedCell` is staged to `_pendingCells` and + committed on the **render thread** (`AddCell`, `GameWindow.cs:5749`); physics + `CacheCellStruct` commits on the **worker thread** (`GameWindow.cs:5489`). Plus a + **+2 cm render Z-lift** (`GameWindow.cs:5463-5464`) and a **null-BSP drop** in + physics (`PhysicsDataCache.cs:158-159`) → the render cell set is a *superset*. + +**Layer constraint (decisive):** `AcDream.Core` must not depend on `AcDream.App`/GL +(Code Structure Rule 2). `LoadedCell` is App; `CellPhysics`/`TerrainSurface` are Core; +neither alone is a complete `CEnvCell`. Therefore the unified model **must live in Core +and cannot literally wrap the App `LoadedCell`.** + +## 5. Stage 1 design + +### 5.1 Layer decision (approved: Approach A) +The `ObjCell` graph lives in **`AcDream.Core`** and owns the **CPU cell model**; +**App keeps GL** mesh handles keyed by cell id and reads the Core graph to decide what +to draw + clip (Stage 3). GL never enters Core. The two legacy objects are absorbed +over stages (`LoadedCell` CPU role retired at Stage 3; `CellPhysics` folded at Stage 4). +Rejected alternatives: **B** (`ObjCell` as a Core interface both objects adapt to — keeps +two objects bridged, never yields one `curr_cell`); **C** (move `LoadedCell` into Core — +drags render concepts `ClipPlanes`/stencil polys into Core). + +This also resolves three inventory divergences: the **+2 cm lift** (Core holds one +physics-verbatim transform; render applies the lift downstream), the **null-BSP drop** +(the graph includes *all* cells; `PointInCell` on a BSP-less cell falls back to AABB), +and the **three keying schemes** (one graph, one key). + +### 5.2 Types (`namespace AcDream.Core.World.Cells`) +Sketch (final signatures in the plan): + +```csharp +abstract class ObjCell { + uint Id { get; } + bool IsEnv => (Id & 0xFFFF) >= 0x100; // retail magnitude dispatch + Matrix4x4 WorldTransform, InverseWorldTransform; // physics-verbatim (no render lift) + Vector3 LocalBoundsMin, LocalBoundsMax; + IReadOnlyList Portals; + IReadOnlyList StabList; + bool SeenOutside; + abstract bool PointInCell(Vector3 worldPoint); // retail +0x84 seam +} + +sealed class EnvCell : ObjCell { // built from the dat CellStruct + CellBSPTree? ContainmentBsp; // PointInCell → existing BSPQuery; null → AABB + // physics-BSP reference reserved for Stage 4; static objects stay entity-side +} + +sealed class LandCell : ObjCell { // SYNTHESIZED from TerrainSurface + TerrainSurface Terrain; int Cx, Cy; // PointInCell = 24 m quad XY test + uint? BuildingCellId; // CSortCell bridge ref (logic = Stage 2) +} + +readonly struct CellPortal { // unified superset of the 3 portal types + uint OtherCellId; ushort OtherPortalId, PolygonId, Flags; bool PortalSide; + Vector3[] PolygonLocal; // carried now; consumed by PView at Stage 3 +} + +sealed class CellGraph { + void Add(EnvCell); + void RegisterTerrain(uint landblockId, TerrainSurface); + void RemoveLandblock(uint landblockId); + ObjCell? GetVisible(uint id); // 0→null; env→hash; land→synthesize + ObjCell? Neighbor(ObjCell cell, in CellPortal p); // = GetVisible(p.OtherCellId) + ObjCell? CurrCell { get; } // holder defined here; INERT until Stage 2 +} +``` + +### 5.3 Population +Built **alongside** existing registration from the **same dat data**, in Core: +- In `CacheCellStruct` (worker thread) also build + `CellGraph.Add(envCell)` — derive + bounds + full portals (`OtherPortalId`) + `SeenOutside` from the same `CellStruct`. +- In `AddLandblock` also `CellGraph.RegisterTerrain(lbId, terrainSurface)`. +- `LandCell`s are **synthesized on lookup** (not stored), like retail's positional + `LScape` resolution. +- **Every** cell is included — no null-BSP drop. + +### 5.4 What Stage 1 does NOT do (YAGNI boundary) +- No membership driving — `CurrCell` has no writer (Stage 2). +- No render consumption — `CellVisibility`/`LoadedCell` untouched (Stage 3). +- No collision consumption — `ResolveCellId`/`PhysicsDataCache` untouched (Stage 4). +- No streaming-production change — built beside existing registration (Stage 5). +- No PView/clipping, no `find_transit_cells`/`find_collisions`/shadow lists. +- **The graph is built and tested but consumed by nobody.** Behavior is unchanged. + +## 6. Testing (pure Core unit tests, no GL) +Use the **real cottage cell fixtures from the #98 harness** (`0xA9B4014x`) + the +cottage GfxObj fixture (`0x01000A2B`): +- `GetVisible` dispatch: `0`→null; `0xA9B40174`→the `EnvCell`; an outdoor id→a + synthesized `LandCell`. +- Topology: portals resolve to neighbors; `OtherPortalId` reciprocity holds. +- `PointInCell`: a point inside the cottage cell → true, just outside → false + (delegating to the containment BSP); `LandCell` 24 m quad test; BSP-less cell falls + back to AABB. +- Bounds/transform round-trip (world↔local). +- Population/eviction: registering a landblock builds the expected set; + `RemoveLandblock` clears it. + +## 7. Acceptance criteria +- `dotnet build` green; `dotnet test` green (new Core tests pass; existing suite + unchanged — the baseline static-leak flakiness noted in CLAUDE.md is the only allowed + variance). +- **Zero behavior change**: launching the client renders identically to baseline + `9bff2b0` (the graph is consumed by nobody). Spot-check the cottage looks the same + (still broken — that's expected; Stage 3 fixes it). +- Code-structure: all new types in `AcDream.Core.World.Cells`; no App/GL dependency + added to Core. + +## 8. Risks +- **Leaky half-migration** (the standing risk for the whole program): mitigated by the + crisp `ObjCell` interface as the seam — render + membership go through it; collision + + streaming stay behind their current adapters until Stages 4–5. +- **Dat-derivation drift**: deriving bounds/portals/`SeenOutside` in Core must match what + the App `BuildLoadedCell` computes today, or Stage 3's switch-over changes behavior. + Mitigation: a conformance test comparing Core-derived fields against the App values for + the fixtures (one-off, can be deleted after Stage 3). +- **Scope creep into Stage 2+**: enforce §5.4 ruthlessly — Stage 1 ships an inert graph. + +## 9. Next +On approval → `writing-plans` produces the Stage 1 implementation plan +(subagent-friendly tasks, RED→GREEN tests, small sequential commits). Stages 2–5 each +get their own spec at the start of their cycle.