acdream/docs/superpowers/specs/2026-06-08-portal-flood-enqueue-once-port-design.md
Erik 3fd71a123c docs(render): R-A2b spec — revive bounded-propagation, churn confirmed at flap-time
The indoor doorway flap is the portal flood's re-enqueue churn (0171<->0173 mutual re-contribution; drifted near-duplicate regions AddRegion won't dedup -> grew -> re-enqueue, capped at MaxReprocessPerCell=16 -> eye-sensitive flood depth -> grey flash). Confirmed live: launch-churn-confirm.log shows maxPop=16 on 44% of frames during a doorway walk-through. The 2026-06-08 'maxPop=1, churn refuted' verdict was a camera-turn-at-rest capture (wrong reproduction); its DO-NOT is overturned.

Fix (Option A, user-approved): contributions already covered by the neighbour's accumulated view don't grow it (no re-enqueue); only the uncovered remainder propagates -- retail's 'redundant -> empty before copy_view' (copy_view confirmed to just append). Remove MaxReprocessPerCell; keep re-processing of genuinely-new slices. Scope: PortalVisibilityBuilder only. Revives 2026-06-08 spec+plan (banners redirected).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 09:48:53 +02:00

278 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

> **✅ REVIVED 2026-06-09 — the REVISION banner (bounded propagation) is the live design; see
> [`2026-06-09-portal-flood-bounded-propagation-r-a2b-design.md`](2026-06-09-portal-flood-bounded-propagation-r-a2b-design.md).**
> The `⛔` banner below was wrong: the `maxPop=1` "refutation" was a **camera-turn-at-rest** capture (the
> calm position, root `0172`), NOT a doorway crossing. A 2026-06-09 walk-through re-capture
> (`launch-churn-confirm.log`) measured `maxPop=16` on 44 % of frames — the churn is confirmed at
> flap-time. The "enqueue-once" half (§4/§5) stays dead (re-processing IS retail-faithful); the
> **bounded-propagation** half (REVISION banner) is what ships, as R-A2b.
>
> ---
>
> **⛔ (HISTORICAL — corrected above) SUPERSEDED / REFUTED 2026-06-08 (evening).** Both this spec's directions
> (enqueue-once AND the revised bounded-propagation) were called dead: live measurement found ZERO portal
> re-enqueue churn at the flap (`maxPop=1`) — **but that sample was the wrong reproduction (see the revival
> note above).** The flap is a STRUCTURAL divergence — retail renders
> inside+outside through ONE `DrawInside(viewer_cell)` path with no inside/outside branch, and is
> robust to its own ~36 µm eye jitter via many small per-building floods. Decision: full retail port
> (Option A). Canonical:
> [`docs/research/2026-06-08-full-retail-render-port-OPTION-A-handoff.md`](../../research/2026-06-08-full-retail-render-port-OPTION-A-handoff.md).
# Portal-Flood Bounded-Propagation Port — the indoor "flap" fix (verified design)
**Date:** 2026-06-08 (revised — enqueue-once superseded)
**Branch:** `claude/thirsty-goldberg-51bb9b`
**Status:** Design revised after the writing-plans decomp pass; pending re-review.
> ## ⚠️ REVISION (2026-06-08 PM): "enqueue-once" REFUTED — corrected to "bounded propagation"
> The original approach below (§4 enqueue-once, §5 correct-the-test) is **WRONG** and is retained only
> for the audit trail. The writing-plans decomp pass read `FixCellList` (decomp 433407) →
> `AdjustCellView` (433741) → `ClipPortals(update_count)` + `AddViewToPortals`, which proves **retail
> DOES re-process a grown-after-drawn cell**. So:
> - **Re-processing on growth is retail-faithful and STAYS.** Pure enqueue-once is wrong; it would break
> `Build_ViewGrowthAfterDoneCell_PropagatesNewSlicesToExit` for the *right* reason (the test is correct
> — do NOT "correct" it; §5 below is VOID).
> - **The real divergence is BOUNDING.** Retail's re-processing terminates *structurally* — each view
> slice is processed once (the `update_count` watermark is monotonic) and a redundant **reciprocal**
> back-contribution clips to **empty** (`OtherPortalClip` → no `copy_view` → no new slice; decomp
> 433654/433711-712). acdream's reciprocal (`ApplyReciprocalClip`+`ClipToRegion`) instead yields a
> **drifted non-empty sliver** → `grew` → re-enqueue → churn, bounded only by the
> `MaxReprocessPerCell=16` **hack**. The churn's fixpoint is eye-sensitive → the flap.
> - **Corrected fix:** port retail's **bounded propagation** — make redundant reciprocal/re-clip
> contributions NOT generate new propagatable slices (match retail's empty-reciprocal + monotonic
> `update_count` watermark), and remove the `MaxReprocessPerCell` cap. Keep re-processing.
> - **Scope:** still `PortalVisibilityBuilder` only; no rooting/camera/clip-math-rewrite/seal change. The
> user approved this corrected "faithful moderate port" direction (over a non-faithful epsilon-dedup
> band-aid) on 2026-06-08.
> - **One open precision (→ plan Task 1):** exactly *where* acdream's reciprocal sliver becomes non-empty
> is float-drift-dependent on real doorway geometry — a runtime fact. The implementation plan's first
> task **instruments `PortalVisibilityBuilder` (per-pop re-pop count + reciprocal-clip in/out + `grew`),
> captures at the doorway, and pins the exact line** before the fix, rather than guessing from decomp.
> - **Corrected retail grounding (the full traversal):** `ConstructView` 433750 (pop-once → draw-list →
> `ClipPortals(cell,0)` → `AddViewToPortals`); `ClipPortals` 433572 (slices `[update_count,view_count)`,
> `GetClip` per portal, exit→`copy_view`/OutsideView, neighbour→`OtherPortalClip`); `AddViewToPortals`
> 433446 (first-discovery→`InitCell`+`InsCellTodoList`; growth→`AddToCell`+`FixCellList`);
> `FixCellList` 433407 = `AdjustCellPlace` + `AdjustCellView` 433741 (=`ClipPortals(update_count)` +
> `AddViewToPortals` — **the re-process**); `OtherPortalClip` 433524 (reciprocal, empty-for-redundant).
>
> Everything below this banner is the ORIGINAL (superseded) enqueue-once design — kept for the record.
> ---
# (SUPERSEDED) Portal-Flood Enqueue-Once Port — original design
**Date:** 2026-06-08
**Branch:** `claude/thirsty-goldberg-51bb9b`
**Status:** SUPERSEDED by the revision banner above.
> **Supersedes** the enqueue logic in `docs/superpowers/specs/2026-06-08-portal-flood-membership-stability-design.md`
> (whose §4 enqueue-once was an *incomplete* attempt — it dropped the `AddToCell` growth half) and the
> physics-rest direction in `docs/research/2026-06-08-flap-rootcause-physics-rest-handoff.md` (refuted).
> **Diagnosis evidence:** `docs/research/2026-06-08-flap-physics-diagnosis-REFUTED-its-render-membership.md`
> + this session's two adversarial verification agents (retail decomp + acdream code/data).
---
## 1. Summary
The indoor render **flap** (interior textures battling / popping in and out at a building doorway) is a
**render-side portal-flood membership instability**: as the camera **eye** moves (turning the camera, or
the camera's smoothing-glide after a turn), the set of cells the flood deems visible **oscillates**
(e.g. `8↔3`) even though the eye sweeps **monotonically**. The root is acdream's **re-enqueue-on-growth
"drift"** in `PortalVisibilityBuilder.Build` (`cs:322`, `MaxReprocessPerCell = 16`): a cell whose view
grows is re-enqueued and its portals **re-clipped from the grown (drifted) view** each round; under
sub-cm eye motion each frame re-clips slightly differently → the visible set flips.
The fix is a **verbatim port of retail's enqueue-once portal traversal** (`PView::ConstructView` +
`AddViewToPortals`): a cell is enqueued **only on first discovery**; its portals are clipped **exactly
once** (at pop); later growth into an already-discovered cell is unioned **incrementally in place**
(`AddToCell`) and its draw-list slot re-ordered (`FixCellList`) — **never re-enqueued, never re-clipped
from scratch**. This makes the visible set a deterministic function of the **root + geometry**, so it no
longer drifts with eye jitter. Localized to `PortalVisibilityBuilder`. No camera, physics, rooting, clip-
math, or seal change.
---
## 2. Root cause — verified this session
### 2.1 What the flap is NOT (refuted with primary evidence)
- **Not physics.** `door-recheck-capture.jsonl`: **216,300 standstill physics records, 0 position
re-snaps** — the body is byte-stable at rest. Deterministic tests (flat terrain + indoor cell, resolver
+ full controller) confirm: a resting body holds a byte-identical position. The 2026-06-08 AM
"physics rest µm-jitter" diagnosis is refuted.
- **Not the camera rooting or the inside/outside toggle.** Verified against retail (agent 1):
`SmartBox::RenderNormalMode` (0x453aa0) calls **`DrawInside(viewer_cell)`** (decomp 92675), and
`SmartBox::update_viewer` (0x453ce0) sets `viewer_cell` from a **swept `CTransition`** seeded at the
**player's cell** (`init_path(cell_1, …)` 92866 → `viewer_cell = sphere_path.curr_cell` 92871). So
rooting at the camera's `viewer_cell` and toggling `DrawInside`/`LScape::draw` are **retail-faithful**.
The locked-design claim "root at the player cell" (`2026-06-02 …redesign-design.md` §1.5) is **wrong**;
acdream's current `clipRoot = viewerRoot ?? _outdoorNode` (eye-cell rooting) is correct and stays.
- **Not camera drift at rest.** When the eye is byte-stable (hands-off idle), the flood is rock-stable
(203/181/178-frame byte-identical-eye runs hold a single flood value). The camera settles; the flap
fires **only while the eye moves**.
### 2.2 What the flap IS (verified — agent 2 + live capture)
- The flood oscillates **only when the eye moves**: across ~7,800 flood flips, **3** had a byte-identical
eye (all startup/streaming); **~87 %** of eye-motion flips have a **byte-identical player** position.
A clean burst (yaw byte-constant, eye gliding monotonically 18→5 mm/frame as the camera settles) shows
flood `8→3→8…`**non-monotonic membership under a monotonic eye sweep**.
- The mechanism is the **re-enqueue/re-clip drift**: `PortalVisibilityBuilder.cs:322`
`if (grew && popCounts.GetValueOrDefault(neighbourId) < MaxReprocessPerCell && queued.Add(neighbourId))
todo.Insert(neighbour, dist);` re-enqueues a grown neighbour up to 16×; each re-process re-clips the
cell's portals from its grown view, so sub-cm eye jitter walks `ClipToRegion`'s surviving-vertex count
across the empty/non-empty boundary → the deep cluster `{0172-0175}` drops/returns → the flap.
- **Sub-issue "C" (indoor flood=2 / "missing textures") is mostly a *symptom* of this drift**, not a
missing seal: the landscape-through-the-door seal **is** present in the indoor path
(`RetailPViewRenderer.DrawInside``DrawLandscapeThroughOutsideView`). When the flood drops `8→3`,
the `OutsideView`/terrain/cell clip shrinks → things vanish. Fixing the drift removes the symptom.
---
## 3. Retail grounding (the traversal being ported)
All from `docs/research/named-retail/acclient_2013_pseudo_c.txt`:
- **`PView::ConstructView`** (0x5a57b0, :433750): `InitCell(root)` + `InsCellTodoList(root)`, then a loop
that **pops one cell at a time** from the todo list, **appends it to the draw list** (← that is
membership), sets `cell_view_done = 1` (:433784), runs `ClipPortals` once, then `AddViewToPortals`.
- **`PView::AddViewToPortals`** (0x5a52d0, :433446): for each visible portal to a neighbour, three cases
keyed on the neighbour's stamps (`processed_stamp` = `*(view+0x44)`, `view_stamp` = `*(view+0x38)`):
- **First discovery** (`processed_stamp == 0`, :433478): `InitCell(neighbour)` + `InsCellTodoList`
(**enqueue once**).
- **Growth** (`processed_stamp != view_stamp`, :433492): `AddToCell(neighbour)` + if already drawn
`FixCellList`; then `processed_stamp = view_stamp`. **No re-enqueue. No re-clip from scratch.**
- **Already current** (`processed_stamp == view_stamp`): **nothing**.
- **`PView::AddToCell`** (0x5a4d90, :433050): clips the cell's portals against **only the newly-added
view slices** (`for i = esi[0x11]; i < esi[0xe]`) — an **incremental** union, not a full re-clip; it
does **not** re-contribute to `OutsideView`.
- **`PView::FixCellList`** (0x5a5250, :433407) → `AdjustDrawList` (:433107): **re-orders** the grown cell
in the draw list to preserve draw order. No re-flood.
- **`PView::InitCell`** (0x5a4b70, :432896): seeds the cell's view, clips its portals against the full
incoming view, stamps with `master_timestamp`; returns whether the cell is non-empty (→ enqueue).
So retail clips each cell's portals **exactly once** (at pop). Late growth refines a cell's own view +
draw order, never its downstream flood. This is the `cell_view_done` "process each cell once" guarantee.
---
## 4. The fix (design)
**Scope: `PortalVisibilityBuilder.Build` only.** Replace the re-enqueue-on-growth fixpoint with retail's
enqueue-once traversal. Concretely:
**Change A — enqueue-once (`Build` ~308-328).** Today:
var nview = GetOrCreate(frame.CellViews, neighbourId);
bool grew = AddRegion(nview, clippedRegion); // union in place (= retail AddToCell)
if (grew && popCounts.GetValueOrDefault(neighbourId) < MaxReprocessPerCell && queued.Add(neighbourId))
todo.Insert(neighbour, dist); // RE-ENQUEUE on growth the drift
New: enqueue a neighbour into `todo` **only on first discovery** i.e. when it has **no `CellViews`
entry yet** (retail `processed_stamp == 0` `InitCell` + `InsCellTodoList`). On growth into an
already-discovered neighbour, **keep `AddRegion`** (incremental union = `AddToCell`) and re-order it in
the draw list if already present (`FixCellList`, §Change C), but **do not** re-insert into `todo`.
**Change B — remove the re-enqueue machinery.** Delete `MaxReprocessPerCell`, `popCounts`, and the
per-pop re-enqueue / `queued`-reset logic in the pop loop. Termination is now by construction (each cell
enqueued 1, popped 1; N cells total), matching retail `cell_view_done`. The `MaxReprocessPerCell` cap
existed **only** as a termination band-aid for the re-enqueue with enqueue-once it is dead.
**Change C — draw-list re-order on growth (`FixCellList`).** When growth unions into an
already-discovered cell that is **already in `OrderedVisibleCells`**, re-position it to preserve
closest-first draw order (retail `AdjustDrawList` :433107). If acdream's `OrderedVisibleCells` is already
distance-sorted at assembly time and order is not load-bearing for correctness, this degrades to a no-op
confirm during implementation; do **not** add ordering machinery the renderer doesn't consume.
**Unchanged (explicitly):** the per-portal clip (`ProjectToClip`/`ClipToRegion`), the
`EyeInsidePortalOpening` degenerate-portal guard (`Build:235-244`), the reciprocal `OtherPortalClip`, the
`OutsideView` exit contribution, the rooting (`clipRoot = viewerRoot ?? _outdoorNode`), the camera, and
the landscape-through-door seal. No new predicate, no robustness heuristic, no hysteresis.
**Why this is the flap fix, not a band-aid:** with each cell's portals clipped once, the visible set is a
deterministic function of `(root, geometry)` independent of the per-round re-clip path. Sub-cm eye
jitter changes the *projection* (and thus what's drawn within each clipped cell, correctly) but no longer
changes *which cells are members*. The membership stops oscillating; the textures stop battling.
---
## 5. The `Build_ViewGrowthAfterDoneCell` question (open item, resolve during implementation)
The re-enqueue was added 2026-06-07 "to propagate late-discovered slices to exit portals," and
`PortalVisibilityBuilderTests.Build_ViewGrowthAfterDoneCell_PropagatesNewSlicesToExit` encodes that. But
the decomp shows retail's `AddToCell` (:433050) only clips the cell's **own** portals against new slices
+ re-orders draw position it does **not** re-contribute to `OutsideView` (the exit slice is emitted by
`ClipPortals` at pop, once). So "late growth reaches the exit/OutsideView" appears to be **non-retail**.
**Action:** read `PView::ClipPortals` (the OutsideView contribution site) during implementation to
confirm. If confirmed, this test encodes the non-faithful re-enqueue behavior and is **corrected to
match retail** (late growth refines the cell's view + draw order, not the OutsideView). It will **not**
be satisfied by reinstating the re-enqueue. If the OutsideView tests
(`Builder_Cellar_WindowClippedToStairwell`, look-in tests) shrink, that is the retail behavior, handled
retail's way not by re-adding the drift.
---
## 6. Testing (TDD)
The flap manifests only under live µm/mm eye motion at a specific grazing geometry, so the **visual gate
is acceptance**; the unit layer pins determinism + guards regressions.
1. **Deterministic eye-sweep stability (new, the RED→GREEN driver).** In `AcDream.App.Tests`
(alongside `PortalVisibilityBuilderTests`, since `PortalVisibilityBuilder` is an App-layer type), build
the flood at a sequence of eye positions stepping across the grazing door angle (sub-cm steps
reproducing the live sweep). **Assert each cell's membership across the sweep is a single contiguous
run** no `present→absent→present` (or `absent→present→absent`) flicker for any cell. That is the
precise anti-flap property (the live capture showed `8→3→8→3`, multiple transitions per cell). RED
under the re-enqueue drift; GREEN after enqueue-once. *Fixture note:* the captured dumps live at
`tests/AcDream.Core.Tests/Fixtures/flap-doorway/0xA9B4017{0..5}.json`; the test must reach them
(shared path or copied into `AcDream.App.Tests/Fixtures`) and the cells must carry the portal graph +
clip planes `Build` consumes. If the cell-dump format omits portals/clip-planes, the impl plan either
extends the dump or synthesizes a minimal doorway portal topology reproducing the grazing geometry
surface this as the first implementation step, do not silently weaken the test.
2. **Enqueue-once termination + dedup (new).** Diamond (a cell reachable from two parents) + cycle
fixtures: assert the flood terminates with `MaxReprocessPerCell` removed, `OrderedVisibleCells` is
deduped, each reachable cell present exactly once, and (if a per-cell pop counter is cheap to surface)
each cell popped 1.
3. **No membership regression.** `Build_IsDeterministic_*`, `Build_EyeStandingInInteriorPortal_FloodsNeighbour`,
`Build_CollapsedInteriorPortalNearEyeBeyondHalfMeter_FloodsNeighbour`,
`Build_DegeneratePortalToTheSide_NotFlooded_NoOverInclusion` (#95 guard), and the cellar/window/look-in
tests stay **green**. `Build_ViewGrowthAfterDoneCell_PropagatesNewSlicesToExit` is handled per §5.
4. **Visual gate (user) — acceptance.** At the cottage doorway: turn the camera back and forth and walk
through the interior rooms render steadily, no battling/popping; the `[pv-input]` flood is stable
for a given eye pose. Re-run with `launch-flap-capture.ps1`.
`dotnet build` + `dotnet test` green before the visual gate.
---
## 7. Scope / non-goals
- **In scope:** `PortalVisibilityBuilder.Build` enqueue logic (enqueue-once; remove
`MaxReprocessPerCell`/`popCounts`/re-enqueue; incremental union on growth; draw-order re-position) + the
new/updated tests; reading `ClipPortals` to settle §5.
- **Non-goals (deferred / untouched):**
- **No rooting change** eye-cell rooting (`clipRoot = viewerRoot ?? _outdoorNode`) is retail-faithful
2.1). The locked design's "root at player cell" is refuted, not implemented.
- **No clip-math change** (`ProjectToClip`/`ClipToRegion`), no `EyeInsidePortalOpening` change, no
overlap predicate, no hysteresis/robustness heuristic.
- **No camera, physics, or seal change.** The landscape-through-door seal already exists; C is a symptom
of the drift and resolves with it.
- The 4 GREEN physics rest-stability tests added this session stay as regression guards (they document
that physics rest is bit-stable the flap is not physics).
---
## 8. Apparatus + references
- **Diagnosis + verification:** `docs/research/2026-06-08-flap-physics-diagnosis-REFUTED-its-render-membership.md`;
this session's two adversarial verification agents (retail decomp CONFIRMED rooting/seal; acdream
code/data CONFIRMED physics-out + eye-driven + the `cs:322` drift).
- **Captured fixtures:** `tests/AcDream.Core.Tests/Fixtures/flap-doorway/0xA9B4017{0..5}.json`;
`flap-doorway-resolve.jsonl`. Apparatus: `launch-flap-capture.ps1`, `analyze_flap_live.py`,
`find_burst.py`, the `[pv-input]` probe (`ACDREAM_PROBE_PVINPUT`, now logs eye/player/rawPlayer/yaw).
- **Retail decomp anchors:** `ConstructView` :433750, `AddViewToPortals` :433446, `InitCell` :432896,
`AddToCell` :433050, `FixCellList` :433407 / `AdjustDrawList` :433107, `InsCellTodoList` :433183,
`SmartBox::update_viewer` :92761, `SmartBox::RenderNormalMode` :92635.
- **Superseded:** `2026-06-08-portal-flood-membership-stability-design.md` §4 (incomplete enqueue-once);
`2026-06-08-flap-rootcause-physics-rest-handoff.md` (physics direction, refuted).
- **Memory to correct after ship:** `project_indoor_flap_rootcause` (root = the `PortalVisibilityBuilder`
re-enqueue/re-clip **drift** under a moving eye; rooting/toggle is retail-faithful; physics + camera
exonerated).