docs: #100 ship + indoor-cell culling investigation handoff

Session-end documentation for the issue #100 ship and the visibility- culling investigation handoff for the next session. Three documents land together: - docs/superpowers/plans/2026-05-25-issue-100-terrain-cutout.md (the 3-task plan that drove this session's f48c74a / a64e6f2 / 84e3b72 — never committed by Tasks 1-2) - docs/research/2026-05-25-issue-100-terrain-cutout-handoff.md (the predecessor session's smoking-gun research that drove the #100 fix — never committed by the prior session) - docs/research/2026-05-25-issue-100-shipped-and-culling-handoff.md (THIS session's handoff: what shipped, what visual-verification surfaced, the issue family map for #78 + #95 + the new cellar- stairs finding, root-cause hypothesis, retail anchors, WB references, do-not-retry list, and pickup prompt for the next session's investigation + plan + implementation) Plus two updates to existing files: - CLAUDE.md — adds a ship paragraph for #100 to the M1.5 progress block. References the new handoff doc as the next-session pickup point. - docs/ISSUES.md #78 — broadens scope from "outdoor stabs visible through floor" to "outdoor stabs + terrain mesh visible inside EnvCells". Adds the 2026-05-25 cellar-stairs evidence (per user direction: not filed as new issue; treated as evidence reinforcing #78's hypothesis #2). Promotes hypothesis #2 to "high confidence as of 2026-05-25" and adds the retail anchor (acclient_2013_pseudo_c.txt:311397 CEnvCell::find_visible_child_cell). Acceptance criteria broadened to include the cellar-stairs case. Next session: pickup prompt at the bottom of the new handoff doc drives a /investigate → writing-plans → subagent-driven-development pass on indoor-cell visibility culling — the work that closes #78 + cellar-stairs together, and possibly #95 if the infrastructure overlaps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 22:17:51 +02:00 · 2026-05-25 22:17:51 +02:00 · 4cbfbf98af
commit 4cbfbf98af
parent 84e3b72b27
5 changed files with 1431 additions and 11 deletions
--- a/docs/research/2026-05-25-issue-100-terrain-cutout-handoff.md
+++ b/docs/research/2026-05-25-issue-100-terrain-cutout-handoff.md
@ -0,0 +1,406 @@
+# Issue #100 — Transparent ground around buildings — investigation handoff
+
+**Date:** 2026-05-25 PM (end of A6.P8 session)
+**Status:** Initial research done; **next session is fix-design + implement**. The smoking gun is retail's per-draw `zFightTerrainAdjust = 0.01`. The current acdream code uses a wrong mechanism (cell-level terrain collapse) that creates the transparent rectangles around every Holtburg house.
+**Predecessor issue entry:** [`docs/ISSUES.md` #100](../ISSUES.md) (filed 2026-05-24).
+
+---
+
+## TL;DR
+
+The transparent rectangles around every Holtburg house are caused by acdream's
+`hiddenTerrainCells` mechanism — a misfire on the Z-fighting problem. The
+mechanism collapses entire 24m × 24m outdoor terrain cells to a zero-area
+degenerate when any building's `Frame.Origin` lies in them, but cottages are
+only ~12m × 12m, so ~75% of each "hidden" cell is bare framebuffer-clear
+showing through.
+
+**Retail's mechanism is different and almost trivially small:** retail
+**always renders the full terrain mesh, then nudges every terrain vertex Z
+down by `0.00999999978 m` (= ~0.01 m) at draw time.** That makes terrain
+always lose the depth test against a coplanar building floor — Z-fight
+solved, no cells hidden, no cutout polygon needed. Verbatim from the
+2013 EoR retail decomp:
+
+| Source | What |
+|---|---|
+| `docs/research/named-retail/acclient_2013_pseudo_c.txt:1120769` | `float zFightTerrainAdjust = 0.00999999978;` |
+| `docs/research/named-retail/acclient_2013_pseudo_c.txt:430113` | `DrawLandCell(esi_3)` — per-cell terrain draw |
+| `docs/research/named-retail/acclient_2013_pseudo_c.txt:430124` | `DrawSortCell(esi_3)` — per-cell building draw, **same iteration** |
+| `docs/research/named-retail/acclient_2013_pseudo_c.txt:427867` | `ACRender::landPolysDraw(arg2->polygons, 2)` — the `arg2=2` path |
+| `docs/research/named-retail/acclient_2013_pseudo_c.txt:006b6402` | `edi_4[1] = (float)((long double)esi_1[2] - (long double)zFightTerrainAdjust);` — the terrain-Z nudge |
+
+**WorldBuilder also renders full terrain** — it does **not** hide cells.
+WB has a known Z-fighting issue in the editor view that nobody noticed
+because it's editor-only.
+[`references/WorldBuilder/Chorizite.OpenGLSDLBackend/Lib/TerrainGeometryGenerator.cs:123-141`](../../references/WorldBuilder/Chorizite.OpenGLSDLBackend/Lib/TerrainGeometryGenerator.cs) iterates all 64 cells unconditionally.
+
+**The fix is path 2 from the issue #100 entry**, refined: drop
+`hiddenTerrainCells` entirely + apply `gl_Position.z -= 0.01` (or
+equivalent world-Z nudge) in `src/AcDream.App/Rendering/Shaders/terrain_modern.vert`
+at line 139. Estimated change: ~15 LOC across 1-2 commits, including
+removal of the dead `BuildingTerrainCells` / `hiddenTerrainCells`
+plumbing.
+
+---
+
+## Symptom (concrete evidence)
+
+User screenshot 2026-05-25: standing next to a Holtburg cottage. The ground
+in a rectangular footprint around the building appears as a flat dark
+pink/light patch (the framebuffer clear color) instead of cobblestone /
+grass terrain. Visible as a sharp-edged rectangle the size of the
+**outdoor terrain cell** (24 × 24 m), not the size of the **cottage's
+building footprint** (~12 × 12 m). Same shape on every house observed.
+
+User wording from 2026-05-24 report: "around every house now I missing
+the ground texture, it is transparent. I can see through the ground."
+
+---
+
+## Root cause (now confirmed via decomp cross-reference)
+
+### The acdream code that produces the bug
+
+Commit `35b37df` (2026-05-23, A6.P3 #98 triage) kept the
+`hiddenTerrainCells` mechanism. The path:
+
+1. **`LandblockLoader.BuildBuildingTerrainCells(LandBlockInfo info)`**
+   ([`src/AcDream.Core/World/LandblockLoader.cs:39-50`](../../src/AcDream.Core/World/LandblockLoader.cs:39))
+   reads `info.Buildings`, computes
+   `int cx = clamp(building.Frame.Origin.X / 24f, 0, 7)`,
+   `int cy = clamp(building.Frame.Origin.Y / 24f, 0, 7)`, and emits
+   `cy * 8 + cx` per building. Granularity: **one 24m cell per building**.
+2. **`LandblockMesh.Build`**
+   ([`src/AcDream.Core/Terrain/LandblockMesh.cs:175-185`](../../src/AcDream.Core/Terrain/LandblockMesh.cs:175))
+   replaces every index in those cells with the cell's first-vertex index,
+   producing degenerate (zero-area) triangles that the GPU rasterizer skips.
+3. Result: a **24m × 24m hole** in the terrain mesh per building, regardless
+   of the building's actual size.
+
+A cottage at, say, world `(110, 26)` has `Frame.Origin` at landblock-local
+`(110, 26)` → `cx = 4`, `cy = 1` → outdoor cell index `12`. The hidden
+area is `(cx*24, cy*24)` to `((cx+1)*24, (cy+1)*24)` = `(96, 24)` to
+`(120, 48)` — a 24×24m square. The cottage footprint is closer to
+~12×12m centred near `(110, 26)`. ~75% of the hidden area has no
+building geometry to cover it → framebuffer-clear visible.
+
+### What the existing comments said the intent was
+
+[`src/AcDream.Core/Terrain/LandblockMesh.cs:171-174`](../../src/AcDream.Core/Terrain/LandblockMesh.cs:171):
+
+> Indices are trivial 0..383 since we don't deduplicate verts. When a
+> building owns an outdoor terrain cell, **keep the fixed 384-index
+> contract but collapse its two triangles so the building/stair mesh can
+> visually own the hole.**
+
+[`src/AcDream.Core/World/LandblockLoader.cs:33-37`](../../src/AcDream.Core/World/LandblockLoader.cs:33):
+
+> Map LandBlockInfo.Buildings to 8x8 terrain mesh cells (cy * 8 + cx).
+> **Retail attaches each CBuildingObj to its outside landcell during
+> CLandBlock::init_buildings;** keep this signal separate from stabs so
+> ordinary static props do not punch holes in terrain.
+
+The first comment shows the intent: avoid Z-fighting between the building
+floor and the terrain below. The second is correct but irrelevant — retail
+attaches buildings to a cell for render-order (the `DrawSortCell` step),
+NOT to hide that cell's terrain. Our author misread the retail intent.
+
+---
+
+## Retail mechanism (verbatim)
+
+Per the research-agent dispatch this session, the full retail render
+sequence is at `RenderDeviceD3D::DrawBlock`
+([`acclient_2013_pseudo_c.txt:430027`](../research/named-retail/acclient_2013_pseudo_c.txt)
+onwards):
+
+```
+for each CLandCell in draw_array (all 64 cells):                    // line 430113
+    DrawLandCell(esi_3)         // → ACRender::landPolysDraw(polygons, 2)
+    DrawSortCell(esi_3)         // → DrawBuilding(...) for any CBuildingObj attached
+                                //   to this cell + the cell's object list
+```
+
+`landPolysDraw(polygons, 2)` selects the path that subtracts
+`zFightTerrainAdjust` from every terrain vertex Z at upload time. The
+constant:
+
+```c
+float zFightTerrainAdjust = 0.00999999978;   // acclient_2013_pseudo_c.txt:1120769
+```
+
+And the application
+([`acclient_2013_pseudo_c.txt:006b6402`](../research/named-retail/acclient_2013_pseudo_c.txt)):
+
+```c
+edi_4[1] = ((float)(((long double)esi_1[2]) - ((long double)zFightTerrainAdjust)));
+```
+
+Where `edi_4[1]` is the output vertex Z and `esi_1[2]` is the source
+vertex Z. So every terrain vertex's `Z` becomes `Z - 0.01` at draw time.
+
+**Result:** terrain is uniformly 1 cm lower than its physical height (the
+physics path uses the un-nudged Z; only the render path nudges). Building
+floors at the physically-correct height always win the depth test
+because they're 1 cm higher than the rendered terrain. No cells are
+hidden. No cutout is computed. The world reads as one continuous surface.
+
+### Retail's `CLandBlock::init_buildings`
+
+[`acclient_2013_pseudo_c.txt:313854`](../research/named-retail/acclient_2013_pseudo_c.txt)
+iterates `lbi->buildings`, calls
+`CBuildingObj::makeBuilding(building_id, ...)`, then
+`CBuildingObj::add_to_cell(eax_4, landcell)` — attaches the building to
+whichever `CLandCell` it physically belongs to. **This is for render
+ordering (sort) and physics scoping, not for terrain cutout.** No terrain
+modification happens here.
+
+### `BuildInfo` data fields (acclient.h:32035)
+
+```c
+struct __cppobj BuildInfo {
+    IDClass<_tagDataID,32,0> building_id;   // Setup DID (0x02xxxxxx)
+    Frame building_frame;                   // position + rotation
+    unsigned int num_leaves;                // portal leaf count
+    unsigned int num_portals;
+    CBldPortal **portals;
+};
+```
+
+**There is no explicit footprint polygon, AABB, or terrain-cell list.**
+The only geometric anchor is `building_frame.Origin`. Building footprint
+must be derived from the Setup's `parts[0]` GfxObj geometry if you needed
+it — retail never does, because the depth-nudge mechanism makes it
+unnecessary.
+
+---
+
+## Recommended fix shape
+
+### Path 2 (refined) — retail-faithful terrain Z-nudge
+
+**Site:** [`src/AcDream.App/Rendering/Shaders/terrain_modern.vert`](../../src/AcDream.App/Rendering/Shaders/terrain_modern.vert) line 139.
+
+**Change:** replace
+
+```glsl
+gl_Position = uProjection * uView * vec4(aPos, 1.0);
+```
+
+with
+
+```glsl
+// Retail zFightTerrainAdjust (acclient_2013_pseudo_c.txt:1120769, value
+// 0.00999999978). Lower terrain by 1 cm so coplanar building floors
+// (at the un-nudged physically-correct Z) always win the depth test.
+// Cross-ref: docs/research/2026-05-25-issue-100-terrain-cutout-handoff.md.
+vec3 terrainPos = vec3(aPos.xy, aPos.z - 0.01);
+gl_Position = uProjection * uView * vec4(terrainPos, 1.0);
+```
+
+**Cleanup (same commit or follow-up):**
+
+1. Delete `hiddenTerrainCells` parameter and the collapse block at
+   `LandblockMesh.cs:175-185`.
+2. Delete `LoadedLandblock.BuildingTerrainCells` field at
+   `src/AcDream.Core/World/LoadedLandblock.cs`.
+3. Delete `BuildBuildingTerrainCells` at
+   `LandblockLoader.cs:33-50`.
+4. Delete the threading through `GameWindow.cs:1808, 5366, 8761` and
+   `src/AcDream.App/Streaming/{GpuWorldState,LandblockStreamer}.cs`.
+5. Delete `tests/AcDream.Core.Tests/Terrain/LandblockMeshTests.cs`'s
+   hiddenTerrainCells test cases. Delete or rewrite
+   `tests/AcDream.Core.Tests/World/LandblockLoaderTests.cs`'s
+   `BuildBuildingTerrainCells_*` cases.
+
+**Test plan:**
+
+- Add a tiny shader-vertex unit test if there's a precedent (look in
+  `tests/AcDream.App.Tests/Rendering/` for any shader-correctness tests).
+- Visual verification at Holtburg: terrain renders continuously under
+  cottages, no transparent rectangles. Z-fighting between building floor
+  and terrain not visible.
+- Run the full focused test suite (now 23 tests, will likely shrink by 2-4
+  when the dead `BuildBuildingTerrainCells` / `LandblockMesh.hiddenTerrainCells`
+  tests are removed) and confirm green.
+
+**Why this is right:**
+
+- Matches retail mechanism verbatim (1 cm Z nudge on terrain at draw time).
+- Removes ~50 LOC of dead plumbing (`BuildingTerrainCells` threading
+  through 5 files).
+- Avoids the per-building-footprint computation that the current code
+  cannot do correctly without loading the Setup mesh.
+
+### Why NOT path 1 (polygon-level cutout)
+
+- Retail doesn't do this — there is no precedent in the named decomp.
+- Building footprint isn't in `BuildInfo` — would require loading the
+  Setup AND computing a 2D XY footprint polygon from `parts[0]`'s
+  geometry. Engineering-heavy.
+- Even if computed, mesh modifications break the fixed 384-index contract
+  in `LandblockMesh.Build`.
+
+### Why NOT path 3 (building yard mesh)
+
+- Retail doesn't have this. `BuildInfo` carries no yard polygon.
+- Cottage Setups don't appear to include a yard mesh in their geometry
+  (would need confirmation by dumping a cottage Setup, but the retail
+  mechanism makes this question moot).
+
+---
+
+## Do-not-retry list
+
+1. **Don't try to compute the building's tight footprint** from
+   `LandBlockInfo.Buildings`. The struct doesn't carry one. Retail doesn't
+   either. Any computation would require loading the Setup mesh and
+   building an XY hull from `parts[0]` — pure engineering with no retail
+   anchor.
+2. **Don't shift the 0.02 m EnvCell render lift** at
+   `GameWindow.cs:5400` (or equivalent). That lift is for indoor-cell
+   floor rendering and is correct as-is. The terrain Z nudge is the
+   reverse direction (lower terrain) and is independent.
+3. **Don't disable depth testing** on terrain or building draws. Retail
+   uses standard depth test (`GL_LESS` equivalent); the Z nudge alone is
+   the disambiguator.
+4. **Don't apply `glPolygonOffset`** to terrain. Retail uses a vertex Z
+   nudge, not GPU-side polygon offset. Polygon offset has hardware-specific
+   slope-dependent behavior; the constant 1 cm world-Z is uniform and
+   well-defined.
+5. **Don't keep `hiddenTerrainCells` and add the Z nudge as a "belt and
+   suspenders"** safety. The hidden-cells path is wrong and should be
+   deleted in the same commit. Two mechanisms for the same problem is
+   future technical debt.
+6. **Don't touch the physics path.** The Z nudge is render-only. Physics
+   already uses the un-nudged terrain Z. This is the same render-vs-physics
+   split that `35b37df` correctly introduced for the `0.02m` EnvCell render
+   lift (kept item in that commit's "Kept" list).
+
+---
+
+## Files involved (for the next session)
+
+| File | What's there | Action |
+|---|---|---|
+| `src/AcDream.Core/Terrain/LandblockMesh.cs:175-185` | `hiddenTerrainCells` collapse block | Delete |
+| `src/AcDream.Core/Terrain/LandblockMesh.cs:Build` signature | `IReadOnlySet<int>? hiddenTerrainCells` param | Delete param |
+| `src/AcDream.Core/World/LoadedLandblock.cs` | `BuildingTerrainCells` field | Delete |
+| `src/AcDream.Core/World/LandblockLoader.cs:33-50` | `BuildBuildingTerrainCells` method | Delete |
+| `src/AcDream.Core/World/LandblockLoader.cs:Load` | `buildingTerrainCells` local + threading into `LoadedLandblock` ctor | Delete locals + simplify ctor call |
+| `src/AcDream.App/Rendering/GameWindow.cs` ~lines 1808, 5366, 8761 | `LandblockMesh.Build(..., lb.BuildingTerrainCells)` call sites | Drop the `hiddenTerrainCells` argument |
+| `src/AcDream.App/Streaming/GpuWorldState.cs` | `BuildingTerrainCells` threading | Drop |
+| `src/AcDream.App/Streaming/LandblockStreamer.cs` | `BuildingTerrainCells` threading | Drop |
+| `src/AcDream.App/Rendering/Shaders/terrain_modern.vert:139` | `gl_Position = ...` | Insert `aPos.z - 0.01` nudge above |
+| `tests/AcDream.Core.Tests/Terrain/LandblockMeshTests.cs` | `hiddenTerrainCells` test cases | Delete |
+| `tests/AcDream.Core.Tests/World/LandblockLoaderTests.cs` | `BuildBuildingTerrainCells_*` cases | Delete |
+
+---
+
+## Open questions
+
+1. **Old terrain shader removed?** There's a `terrain_modern.vert` and the
+   build-output mirrors. Confirm there's no older `terrain.vert` that
+   also needs the nudge applied (the comment at line 4-5 says "Math
+   identical to terrain.vert"; check whether the legacy shader is still
+   compiled into the binary or has been fully retired post-N.5b).
+2. **Sky / water shaders** — confirm the Z-nudge doesn't accidentally
+   affect anything else. Should be limited to the terrain shader only.
+3. **Building floor render order** — retail also relies on the
+   `DrawSortCell` per-cell building draw happening after `DrawLandCell`.
+   Does acdream's current draw order put buildings after terrain? If yes,
+   nothing else needed. If the order is reversed, the depth-nudge still
+   works because depth-test is positional, not order-dependent. Just
+   verify for completeness.
+4. **Does WB have a different shader Z nudge we should crib?** The
+   research agent says no — WB renders full terrain without nudge and
+   has Z-fighting in the editor view. So we should NOT crib from WB
+   here; this is one of the cases where WB and retail diverge and
+   retail wins.
+
+---
+
+## Pickup prompt for next session
+
+```
+Issue #100 — Transparent ground around buildings.
+
+Initial research is done by the prior session (the smoking gun is
+retail's zFightTerrainAdjust = 0.01). This session: VALIDATE the
+research first, then plan, then implement.
+
+Read first (in this order):
+  1. docs/research/2026-05-25-issue-100-terrain-cutout-handoff.md
+     (the handoff doc — symptom, retail mechanism, proposed fix
+     shape, do-not-retry list, files involved)
+  2. docs/ISSUES.md #100
+  3. CLAUDE.md — search "currently working toward" to refresh state
+
+State both altitudes:
+  Currently working toward: M1.5 — Indoor world feels right
+  Current phase: A6 follow-up — fix issue #100 visual regression
+
+## Session flow (three phases, in order)
+
+### Phase 1 — Investigate (use the /investigate skill)
+
+Independently verify the handoff's claims before committing to the
+fix shape. Specifically:
+
+  a. Confirm zFightTerrainAdjust = 0.00999999978 at
+     docs/research/named-retail/acclient_2013_pseudo_c.txt:1120769
+     and the nudge-application at line 006b6402. The handoff cites
+     these — read them yourself and cross-check the surrounding
+     context.
+  b. Confirm WorldBuilder renders all 64 cells unconditionally at
+     references/WorldBuilder/Chorizite.OpenGLSDLBackend/Lib/
+     TerrainGeometryGenerator.cs (handoff says lines 123-141).
+  c. Read src/AcDream.App/Rendering/Shaders/terrain_modern.vert in
+     full and confirm line 139 is the right injection point. Check
+     for any older terrain shader still compiled into the binary
+     (the handoff flags this as an open question).
+  d. Check that physics uses the un-nudged Z. Render-vs-physics
+     split must hold; we cannot let the Z nudge leak into collision.
+  e. Confirm there's no precedent for glPolygonOffset on terrain
+     in our codebase (handoff says no, but verify).
+
+Output of this phase: a short report in chat — either "research
+confirmed, fix shape stands" or "found X divergence, here's the
+revised fix shape." If the research holds, proceed to Phase 2.
+
+### Phase 2 — Plan (use the superpowers:writing-plans skill)
+
+Draft the implementation plan. Expect 3-4 tasks:
+
+  Task 1: terrain_modern.vert Z nudge (the one substantive change).
+  Task 2: delete hiddenTerrainCells / BuildingTerrainCells plumbing
+          (LandblockMesh.cs, LoadedLandblock.cs, LandblockLoader.cs,
+          GameWindow.cs call sites, GpuWorldState.cs,
+          LandblockStreamer.cs). Pure removal — no behavioral
+          change beyond what Task 1 introduces.
+  Task 3: delete corresponding tests in LandblockMeshTests +
+          LandblockLoaderTests that exercise the dead plumbing.
+  Task 4: visual verification — terrain renders continuously at
+          Holtburg cottages, no transparent rectangles, no obvious
+          Z-fighting at building floors.
+
+The handoff doc has a file-by-file action table to seed the plan.
+
+### Phase 3 — Implement (use superpowers:subagent-driven-development)
+
+Execute the plan with fresh subagents per task, two-stage review
+between (spec + code quality), final review across all commits.
+
+Pre-flight verification: full focused test suite green. Build clean.
+
+## Constraints
+
+Do-not-retry list in the handoff doc (6 items). Read it before
+starting Phase 2.
+
+Visual verification is the acceptance test — the M1.5 milestone is
+at stake and any new visual regression in this area would be
+obvious. Be honest about what visual verification shows; don't
+declare success on partial regressions.
+```