diff --git a/docs/research/2026-05-24-a6-p4-pickup-handoff.md b/docs/research/2026-05-24-a6-p4-pickup-handoff.md new file mode 100644 index 0000000..2f4adf0 --- /dev/null +++ b/docs/research/2026-05-24-a6-p4-pickup-handoff.md @@ -0,0 +1,330 @@ +# A6.P4 — Retail-faithful per-cell shadow_object_list port — pickup handoff + +**Date:** 2026-05-24 (end of A6.P3 session, start of A6.P4 plan) +**Status:** Ready to start. Design committed (b55ae83). Pre-flight pending in slice 1's first moves. +**Worktree:** `C:\Users\erikn\source\repos\acdream\.claude\worktrees\strange-albattani-3fc83c` +**Branch:** `claude/strange-albattani-3fc83c` +**Milestone:** M1.5 — "Indoor world feels right" (active) +**Predecessor:** A6.P3 (issue #98 cellar-up) — closed 2026-05-24 by `b3ce505` as a behavioral stopgap. A6.P4 ships the full architectural port and removes the stopgap. + +--- + +## TL;DR for the next session + +1. **State both altitudes** in your first message: M1.5 active; current phase A6.P4; first concrete step is the slice-1 pre-flight reads (Q1 + Q2 below). +2. **Read these three documents first** (in this order, ~15 min): + - `docs/superpowers/specs/2026-05-24-phase-a6-p4-retail-shadow-architecture.md` — the design (slices, anchors, risks) + - `docs/research/2026-05-23-a6-p3-issue98-comparison-harness-findings.md` — the Resolution section at the bottom (architectural divergence + b3ce505 stopgap + door regression) + - `docs/ISSUES.md` — #98 (DONE, contextual), #99 (OPEN — what slice 1 closes), #100 (OPEN — separate phase after A6.P4) +3. **Resolve the two pre-flight questions** (~20 min total) before touching code. +4. **Slice 1 implements** in ~30 min. Test + visual + commit. +5. **Slices 2-3** follow in subsequent sessions (one per session ideally). +6. **Then #100** (transparent ground around houses) — separate phase. + +--- + +## What's already done (DO NOT REDO) + +### Commits on this branch (recent, A6.P3 + handoff) +- `b3ce505` — fix(phys): A6.P3 #98 — gate outdoor shadow radial sweep on indoor primary cell. **Stopgap; slice 3 of A6.P4 removes it.** +- `b55ae83` — docs: A6.P3 #98 resolution + A6.P4 design + #99/#100 filed. **Includes the design doc you'll execute against.** + +### Memory entries (out-of-tree at `C:\Users\erikn\.claude\projects\C--Users-erikn-source-repos-acdream\memory\`) +- `feedback_retail_per_cell_shadow_list.md` — the architectural lesson + decomp anchors +- `feedback_apparatus_for_physics_bugs.md` — the apparatus pattern (live capture + dump + harness) +- `MEMORY.md` index updated + +### Apparatus in tree (REUSE; don't rebuild) +- `PhysicsResolveCapture` ([`src/AcDream.Core/Physics/PhysicsResolveCapture.cs`](../../src/AcDream.Core/Physics/PhysicsResolveCapture.cs)) — env var `ACDREAM_CAPTURE_RESOLVE=` writes JSON Lines per `ResolveWithTransition` call +- `GfxObjDump` / `GfxObjDumpSerializer` ([`src/AcDream.Core/Physics/GfxObjDump.cs`](../../src/AcDream.Core/Physics/GfxObjDump.cs)) — env var `ACDREAM_DUMP_GFXOBJS=0xHHH,0xHHH,...` +- `CellDump` / `CellDumpSerializer` ([`src/AcDream.Core/Physics/CellDump.cs`](../../src/AcDream.Core/Physics/CellDump.cs)) — env var `ACDREAM_DUMP_CELLS=0xHHH,...` +- Harness: [`tests/AcDream.Core.Tests/Physics/CellarUpTrajectoryReplayTests.cs`](../../tests/AcDream.Core.Tests/Physics/CellarUpTrajectoryReplayTests.cs) — `LiveCompare_*` test pattern +- Fixtures at `tests/AcDream.Core.Tests/Fixtures/issue98/` — 16 cell dumps + cottage GfxObj `0x01000A2B.gfxobj.json` + 3-record `live-capture.jsonl` + +--- + +## Direction: A6.P4 full (slices 1–3), then #100 + +**Why this order** (user decision 2026-05-24): #99 (doors) is a regression from b3ce505 that needs prompt fix; slices 2-3 close it architecturally and likely fold in #97 (phantom collisions) + Finding 3 family (sling-out); doing the full port in one phase preserves apparatus + decomp context that would degrade if we paused for #100 in the middle. #100 is cosmetic (visual ground) and doesn't block any demo target. + +**User's stated value driving the choice:** "I want retail parity on collision." Quoted in `feedback_no_patching_collision.md`. The b3ce505 stopgap is, by my own commit message, "the smallest behavioral patch matching retail's effect at the query level" — A6.P4 is the actual port. + +--- + +## Slice 1 — query-side portal expansion (1-2 hours) + +### Goal +Close issue #99 (run-through doors) by extending the query side of `GetNearbyObjects` to include portal-reachable outdoor cells when the primary cell is indoor. **Minimal change; sets up slice 2's registration-side refactor.** + +### Pre-flight (~20 min — answer BEFORE writing code) + +**Q1: Does `CellPhysics.VisibleCellIds` include the outdoor cell on the other side of a building doorway?** + +- Read [`src/AcDream.Core/Physics/CellPhysics.cs`](../../src/AcDream.Core/Physics/CellPhysics.cs) — find what populates `VisibleCellIds` +- Read [`src/AcDream.Core/World/LandblockLoader.cs`](../../src/AcDream.Core/World/LandblockLoader.cs) — find where portal data hydrates into CellPhysics +- Cross-ref against a real loaded EnvCell — `tests/AcDream.Core.Tests/Fixtures/issue98/0xA9B40143.json` has the cottage main floor; does its CellBSP / portal data list any outdoor cell? +- **Decision branch:** + - If `VisibleCellIds` DOES include outdoor neighbors → slice 1 is straightforward; walk that list, filter by `< 0x0100u` (outdoor), include in indoor query + - If `VisibleCellIds` is indoor-only → walk the cell's `Portals` directly (each `PortalInfo` has an `OtherCellId`); collect those that resolve outdoor + +**Q2: Are doors actually registered with outdoor cellScope today?** + +- Find the door spawn path. Likely candidates: + - [`src/AcDream.App/Rendering/GameWindow.cs:3139`](../../src/AcDream.App/Rendering/GameWindow.cs:3139) — server-spawned entities register here (Cylinder collision) + - `EntitySpawnAdapter` or `WorldEntityFactory` — the construction path +- Check what `cellScope` is passed. Default: `cellScope = entity.ParentCellId ?? 0u`. For a door at a doorway, `ParentCellId` might be: + - **null** → cellScope=0u → landblock-wide registration → currently registered via outdoor 24m grid → the b3ce505 gate now skips it from indoor queries → walk-through + - **the indoor cell** → cellScope=that-cell-id → registered indoor-scoped → indoor query already finds it (no #99 bug from this door) + - **the outdoor cell** → cellScope=that-cell-id → indoor-scoped registration with an outdoor cellId (an A1.5 corner case) → behavior depends on how `GetNearbyObjects` handles outdoor cellScope (likely treats it as indoor branch and skips it via the `< 0x0100u` filter — needs verification) +- **If Q2 reveals doors aren't outdoor-registered**, the diagnosis is wrong. Stop coding, re-trace the regression via launch + `ACDREAM_CAPTURE_RESOLVE` + the door scenario. + +**If Q1 + Q2 both confirm the design**, proceed to implementation. Otherwise adjust slice 1. + +### Implementation (~30 min) + +Files to touch: +- `src/AcDream.Core/Physics/ShadowObjectRegistry.cs` — `GetNearbyObjects` gains a new parameter `IReadOnlyCollection? portalReachableOutdoorCells = null`. When primary is indoor and this is non-null, iterate the outdoor cells listed (each is a regular cell key into `_cells`) and merge into results. +- `src/AcDream.Core/Physics/TransitionTypes.cs:2180+` — in `FindObjCollisions`, after computing `indoorCellIds` via `CellTransit.FindCellSet`, build a `portalReachableOutdoorCells` set by walking each indoor cell's `VisibleCellIds` (or `Portals` per Q1 answer) and filtering outdoor ids (`< 0x0100u` low byte). Pass to `GetNearbyObjects`. + +Test: +- New `LiveCompare_DoorThroughDoorway_*` test. Two options: + - **(preferred)** Capture a live tick where a door blocks the player at a Holtburg doorway. `ACDREAM_CAPTURE_RESOLVE=` set. Walk into the inn doorway with door closed. Find the tick where the engine detected the door (`obj=0x...` in the `[resolve]` probe). Add the record to a new fixture. + - **(fallback)** Synthetic harness test: register a fake door Cylinder shadow at a known doorway portal position with the right outdoor cellScope, verify `FindObjCollisions` from the indoor cell returns it. Same shape as the existing harness tests. + +Tests must pass: +- 11/11 `CellarUpTrajectoryReplayTests` continue passing +- 19+ `ShadowObjectRegistryTests` continue passing +- New door test passes + +Visual verification: +- Launch acdream (use the `Run-WithLogout` pattern from `CLAUDE.md` to avoid 3-minute stuck-session) +- Walk into a Holtburg cottage — door blocks from outside ✓ +- Walk inside, walk back toward the doorway — door blocks from inside ✓ (this was the regression) +- Walk into the cellar — cellar climb still works ✓ (no #98 regression) +- Bump into a chair / fireplace inside — still blocks ✓ (no indoor-static regression) +- Bump into a building exterior wall from outside — still blocks ✓ (no outdoor-static regression) + +Commit shape: +``` +feat(phys): A6.P4 slice 1 — portal-reachable outdoor cells in indoor shadow query + +Closes #99. The b3ce505 stopgap (gate outdoor sweep on indoor primary cell) +correctly closes #98 but blocks doors registered to outdoor cells from +being seen by spheres in the adjacent indoor cell. Mirrors retail's +behavior via query-side portal expansion: when primary cell is indoor, +walk indoor cells' VisibleCellIds (or Portals), include any portal- +reachable outdoor cells in the iteration set. + +This is slice 1 of A6.P4. Slice 2 ports retail's full Register-side cell- +set computation; slice 3 removes the b3ce505 gate entirely. + +Pre-flight Q1+Q2 verified before coding: +- Q1: VisibleCellIds is populated with [populate with answer] +- Q2: doors register with cellScope=[populate] + +Verification: +- 11/11 CellarUpTrajectoryReplayTests pass +- new LiveCompare_DoorThroughDoorway test passes +- ShadowObjectRegistry tests pass +- visual: doors block both sides, cellar still climbable, indoor + outdoor + statics unaffected + +Co-Authored-By: Claude Opus 4.7 (1M context) +``` + +--- + +## Slice 2 — registration-side BuildShadowCellSet (~half day with verification) + +### Goal +Port retail's `CObjCell::find_cell_list` indoor/outdoor branch + portal-visible recursion into `ShadowObjectRegistry.Register`. After slice 2, objects are placed in retail-faithful per-cell shadow lists at registration time — the query side becomes pure per-cell list iteration. + +### Plan +- New helper `ShadowObjectRegistry.BuildShadowCellSet(boundingSphere, m_positionCellId, landblockContext)` returns the set of cellIds the object should be registered in. + - If `m_positionCellId` is indoor (≥ 0x0100): include that cell, recurse via the cell's portal-visible neighbors (use `VisibleCellIds` or walk `Portals.OtherCellId`) + - If outdoor: enumerate outdoor cells the bounding sphere overlaps — current behavior for cellScope=0 +- `Register` deprecates `cellScope` param (Obsolete attribute kept for slice 2). New required param `m_positionCellId`. +- All 6 production registration sites in [`GameWindow.cs`](../../src/AcDream.App/Rendering/GameWindow.cs) updated to pass the entity's m_position cellId: + - `:3139` server-spawned entities — pass `spawn.Position.Value.LandblockCellId` (or analog) + - `:5893` landblock-baked statics — pass the static's resolved cellId (compute from world XY if no `ParentCellId`) + - `:5963, :5999, :6024, :6211` setup-derived primitive shapes — same as 5893 + +### Tests +- `Register_OutdoorPosition_RegistersInOutdoorCellsOnly` — outdoor m_position, indoor cell list is empty for that entity +- `Register_IndoorPosition_RegistersInThatCellAndPortalNeighbors` — indoor m_position, the cell + portal-visible cells are in the list +- Existing 11/11 harness tests + 19+ ShadowObjectRegistry tests continue passing +- Slice 1's `LiveCompare_DoorThroughDoorway` continues passing + +### Risks (call-outs from design doc §5) +- **Two-tier streaming order:** if far-tier cells load BEFORE their portal-visible neighbors are loaded, `BuildShadowCellSet` might miss portal cells that arrive later. Mitigation: verify the streaming order in `StreamingController` + `LandblockStreamer`. Possibly re-register on cell load if a portal-neighbor arrives late. +- **Live entity perf:** `UpdatePosition` runs at 5-10 Hz per visible entity. `BuildShadowCellSet`'s portal-traversal is O(portal_count_per_cell). Measure before/after — should still be sub-microsecond. + +### Commit shape +``` +feat(phys): A6.P4 slice 2 — BuildShadowCellSet for retail-faithful Register +refactor(phys): A6.P4 slice 2 — production call sites pass m_positionCellId +``` + +(Two commits — feat for the registry change, refactor for the GameWindow.cs site updates. Keep them in separate commits so a future bisect can attribute regressions cleanly.) + +--- + +## Slice 3 — remove b3ce505 stopgap (~few hours) + +### Goal +Delete the `primaryCellId` parameter on `ShadowObjectRegistry.GetNearbyObjects` and the indoor-primary skip gate. After slice 2, the architecture no longer needs query-time gating — the right shadows are returned by per-cell iteration alone. + +### Plan +- `ShadowObjectRegistry.GetNearbyObjects`: remove `primaryCellId` param + the `if ((primaryCellId & 0xFFFFu) >= 0x0100u) return;` block +- `TransitionTypes.cs:2180` (`Transition.FindObjCollisions`): drop the `primaryCellId: sp.CheckCellId` argument +- `LiveCompare_FirstCap_FixClosesCottageFloorCap` test docstring: update to attribute the fix to registration-side cell-set computation instead of query-side gate +- Remove slice-1's `portalReachableOutdoorCells` parameter too if slice 2's registration-side fix obsoletes it (verify by running slice 3 without it and confirming doors still work) + +### Verification — the load-bearing check +After slice 3, the fix is supposed to live at the registration side, not the query side. Visual verify that: +- Cellar still climbable (#98 still closed) +- Doors still block both sides (#99 still closed) +- Indoor statics still block (chair, fireplace) +- Outdoor statics still block (building walls from outside) + +If anything regresses after removing the stopgap, slice 2 didn't fully port the registration-side architecture — investigate before declaring slice 3 done. + +### Commit shape +``` +refactor(phys): A6.P4 slice 3 — remove b3ce505 indoor-primary gate (stopgap retired) +docs: A6.P4 ship — #98 architectural close, #99 close, likely-closes #97 + Finding 3 family +``` + +--- + +## After A6.P4: #100 (transparent ground around houses) + +### What we know +- Bisected to commit `35b37df` ("chore(phys): A6.P3 #98 triage") +- Introduced the `hiddenTerrainCells` mechanism in `src/AcDream.Core/Terrain/LandblockMesh.cs:178` — collapses terrain triangles in outdoor cells where buildings sit +- Granularity is 24m × 24m outdoor cell; cottage footprint is ~12m × 12m → entire 24m cell hidden but cottage only fills part of it → dark rectangle around every house +- The hide list comes from `LandblockLoader.BuildBuildingTerrainCells` reading `LandBlockInfo.Buildings` + +### Three fix paths (from `docs/ISSUES.md` #100) +1. **Polygon-level terrain occlusion** — build per-building convex-hull cutouts, modify mesh to have a polygon-precise hole. Retail-faithful (probably) but real engineering work in `LandblockMesh.Build` +2. **Drop the hiddenTerrainCells mechanism + Z lift** — accept that buildings sit on terrain and use a render-only Z lift on building floors (same trick env cell floors already use at `GameWindow.cs:5363 + Vector3(0,0,0.02f)`) +3. **Render the building's "yard" mesh** — if retail has a stone-foundation mesh around each building, render it. Need retail visual research + +Option 2 is the smallest and probably right; option 1 is the most faithful. Decide via retail visual cross-check at session start. + +### Phase shape +File as A6.P5 or N.7 (it's rendering, not physics — should be in a separate phase letter). Likely 1 session (small change + visual verification). + +--- + +## Decomp anchors (one stop reference) + +All from `docs/research/named-retail/acclient_2013_pseudo_c.txt`: + +| Line | Function | Role | +|---|---|---| +| 308742+ | `CObjCell::find_cell_list(Position, ...)` | Cell list at registration | +| 308751-308769 | (within) indoor/outdoor branch | Indoor adds 1; outdoor calls `add_all_outside_cells` | +| 308773-308825 | (within) visible-cells recursion | Portal traversal via vtable offset 0x80 | +| 282819+ | `CPhysicsObj::add_shadows_to_cells(CELLARRAY)` | Adds to each cell's list | +| 283322, 283369, 283389 | call sites | Build cell array, then add_shadows_to_cells | +| 308584+ | `CObjCell::add_shadow_object` | Per-cell list append | +| 308916 | `CObjCell::find_obj_collisions(this, ...)` | Per-cell iteration at query time | +| 309560 | `CEnvCell::find_collisions` | Indoor entry — env then obj | +| 316951 | `CLandCell::find_collisions` | Outdoor entry — env then sort then obj | + +--- + +## CLAUDE.md rules that apply + +- **No workarounds without approval** — A6.P4's purpose IS removing a workaround (b3ce505). Don't add new ones. If slice 2 reveals an architectural mismatch that needs a band-aid, STOP and file an issue with full repro notes. +- **Retail-faithful first; cleaner second** — if a retail-port decision conflicts with a modern-design preference, retail wins. +- **Visual verification belongs to the user** — at the end of each slice, request a launch. Don't claim "fix verified" without it. +- **Work-order autonomy** — Claude picks the next step; user reviews. Don't ask "should I start slice 2?"; do it after slice 1 verifies. +- **Apparatus-first for physics divergences** — if any slice surfaces a new bug, build apparatus before guessing (per `feedback_apparatus_for_physics_bugs.md`). + +--- + +## Pickup prompt for next session + +``` +A6.P4 — retail-faithful per-cell shadow_object_list port. Three slices, +then issue #100. Worktree open: + + C:\Users\erikn\source\repos\acdream\.claude\worktrees\strange-albattani-3fc83c + +Read FIRST (in order, ~15 min): + 1. docs/research/2026-05-24-a6-p4-pickup-handoff.md — this handoff + (the canonical pickup; everything else expands from it) + 2. docs/superpowers/specs/2026-05-24-phase-a6-p4-retail-shadow-architecture.md + — the design doc (slices, anchors, risks) + 3. docs/research/2026-05-23-a6-p3-issue98-comparison-harness-findings.md + — Resolution section at the bottom (the saga that led here) + +State both altitudes at the start: + Currently working toward: M1.5 — Indoor world feels right + Current phase: A6.P4 slice 1 — query-side portal expansion to close #99 + (run-through doors regression from b3ce505) + +Direction (user-approved 2026-05-24): + Option B — A6.P4 full (slices 1-3) then issue #100 (transparent ground). + Slice 1 closes #99 fast. Slices 2-3 port retail's Register-side cell-set + computation and remove the b3ce505 stopgap. Likely closes #97 + Finding 3 + family as side effects. #100 is a separate phase after A6.P4 (rendering, + not physics). + +DO NOT REDO: + b3ce505 — issue #98 cellar fix (visual-verified by user 2026-05-24) + b55ae83 — design doc + #98 resolution + #99/#100 filed + memory entries + Apparatus already in tree: PhysicsResolveCapture, GfxObjDump, CellDump, + CellarUpTrajectoryReplayTests harness + fixtures + +Slice 1 first moves (in order): + + (1) PRE-FLIGHT Q1 (~10 min): Does CellPhysics.VisibleCellIds include + the outdoor cell on the other side of a building doorway? Read + src/AcDream.Core/Physics/CellPhysics.cs + LandblockLoader.cs. + Cross-ref with tests/AcDream.Core.Tests/Fixtures/issue98/0xA9B40143.json + (cottage main floor cell). If yes, slice 1 walks VisibleCellIds. + If no, slice 1 walks Portals.OtherCellId directly. + + (2) PRE-FLIGHT Q2 (~10 min): Are doors actually registered with + outdoor cellScope today? Find the door spawn path (likely + GameWindow.cs:3139 + EntitySpawnAdapter), trace cellScope passed. + If doors aren't outdoor-registered, the #99 diagnosis is wrong; + stop and re-investigate via ACDREAM_CAPTURE_RESOLVE at a Holtburg + doorway. + + (3) IMPLEMENT (~30 min if Q1+Q2 confirm): + - ShadowObjectRegistry.GetNearbyObjects gains an optional + portalReachableOutdoorCells parameter + - TransitionTypes.cs:2180 (FindObjCollisions) computes the set + from indoorCellIds + VisibleCellIds/Portals + - New LiveCompare_DoorThroughDoorway_* test (live capture + preferred; synthetic fallback) + - 11/11 CellarUpTrajectoryReplayTests must still pass + + (4) VERIFY (user-side): launch acdream, walk cottage cellar (still + climbable), test doors from both sides (block from both sides + now), bump indoor furniture (still blocks), bump outdoor walls + (still blocks). + + (5) COMMIT (per slice 1 commit shape in the handoff doc). + +Slices 2-3 plans + #100 plan in the handoff doc — execute one slice +per session, visual-verify between, file follow-ups as discovered. + +CLAUDE.md rules apply: + - No workarounds (the b3ce505 stopgap is what slice 3 retires; don't + add new ones) + - Apparatus-first if a new bug surfaces (3+ failed attempts = stop) + - Visual verification belongs to user + - Work-order autonomy — keep going through slices without asking + "should I continue?" + +Test baseline: 11/11 CellarUpTrajectoryReplayTests + 19+ +ShadowObjectRegistry + 4 GfxObjDumpRoundTrip + 4 CellDumpRoundTrip ++ 1 PhysicsDiagnosticsTests pass in isolation. Maintain. Pre-existing +8-19 static-state-leakage failures in serial physics suite are +unchanged from baseline (verified by stash+retest pattern). +```