diff --git a/docs/superpowers/specs/2026-08-30-campaign-w-retail-frame-walk-design.md b/docs/superpowers/specs/2026-08-30-campaign-w-retail-frame-walk-design.md new file mode 100644 index 00000000..665db04b --- /dev/null +++ b/docs/superpowers/specs/2026-08-30-campaign-w-retail-frame-walk-design.md @@ -0,0 +1,211 @@ +# Campaign W — the retail frame walk (rendering redesign design spec) + +**Status:** DRAFT — awaiting owner review (2026-08-30). +**Owner directive:** "I want this as solid as retail, no weird seams between +walls or floors, it should work as retail. As stable as retail." Scope +confirmed 2026-08-30: retail owns every frame decision, modern code owns +only GPU mechanics, and the camera behavior is ported too. +**Predecessor:** Campaign PV (halted 2026-08-30 by owner verdict — +`docs/plans/2026-08-30-pview-visibility-campaign.md`). PV tried to import +retail's visibility decisions piecemeal into our gather/regroup renderer +and failed. This campaign replaces the renderer's decision layer outright. + +--- + +## 1. Why (the disease, named once) + +Retail has no separate "visibility system." The world draws itself by +walking itself: the frame starts at the viewer's cell, recurses through +portals (or walks landcells outward when outdoors), and each cell draws its +own polygons at its turn in the walk. Every polygon is drawn exactly once, +by exactly one owner, in an order that resolves every depth tie, from +geometry the DATs author to be watertight per cell. Correctness is a +property of the structure, not of any check. + +acdream inverted that: gather everything visible into lists, regroup by +material for GPU efficiency, draw in batch order. Regrouping discards +retail's ordering and ownership guarantees, and we have spent months buying +them back one patch at a time. The complete patch inventory — the doorway +flap, the seam strips, the +0.02 z-fight lift, the exit-portal stencils, +the look-in seed heuristics, the seal-panel depth stamps, the #456 +cathedral bleed, and the 2026-08-29/30 transition breakage — is one disease +with one cure: make the walk own the frame. + +The precedent is physics. "No patching collision" led to a faithful retail +port, and physics has been stable since. The renderer is the last subsystem +violating the project thesis (modern code, retail behavior). + +## 2. Goal and non-goals + +**Goal.** Port retail's frame composition verbatim so that every defect +class above becomes structurally impossible: no seams between walls, +floors, or terrain; no visibility flap; no vista bleed at any position +retail can reach; no stale partial-world frames; retail camera behavior. + +**Non-goals.** +- NOT porting retail's per-polygon D3D submission. Our pre-built meshes + contain the same triangles (conformance-tested); porting 2002 D3D buys + zero correctness. +- NOT touching DAT decoding, mesh extraction, terrain math, texture + decode, or GPU residency (the WB-derived asset layer, ~19K lines, + healthy). +- NOT redesigning streaming. The only streaming change is one new + invariant (§4, atomic cell presence). +- NOT changing lighting/shading models already user-gated (VisualMaster). + +## 3. The ownership line + +**Retail owns every decision** — what draws, from where, in what order, +clipped by what. Ported verbatim from `docs/research/named-retail/` with +named-symbol citations in code comments: + +| Decision | Retail mechanism (port source) | +|---|---| +| Frame rooting | `SmartBox::RenderNormalMode` @0x00453aa0: viewer in an EnvCell → `CEnvCell` DrawInside recursion; truly outdoors → `set_default_view` + `LScape` draw | +| Per-portal views | `ConstructView` / `set_view` / `PView::GetClip`; the portal CHAIN's polygon is the only CPU-clipped geometry (`ACRender::polyClipFinish`); one view per chain, never merged (the PV1 finding stands) | +| Object gating | `Render::viewconeCheck` — coarse sphere-vs-cone; PARTIALLY_INSIDE draws the whole object; pixel exactness comes from recursion + z-buffer | +| Landscape walk | `LScape`/`CLandBlock`/`CLandCell` draw order and radius rules | +| Buildings | `CBuildingObj` draw (pre-punch flush @0x0059F2A0) + `BuildInfo.Portals` look-in views (`ConstructView(CBldPortal)`) | +| Stage interleave | The full retail frame sequence (landscape early/late, alpha farther/near, exit mask, shells, dynamics, particles) — owned BY the walk, not approximated from outside as `RetailPViewPassExecutor` does today | +| Depth semantics | Retail's depth-test function and write configuration verbatim, so submission-order tie resolution matches retail exactly (this is what retires the +0.02 lift) | +| LOD/degrade | The marker-class degrade admission (already ported at `RetailDegradePolicy`; relocates to the walk's object-gating step) | +| Particles/dynamics | Emitters draw at their owner cell's walk turn, unclipped (`ShouldDrawParticles` @0x0050FE60) | +| Camera | Retail camera behavior in full: chase, 0.3 m collision probe, slope alignment (already default-on), PLUS the retail zoom envelope. The 40 m development zoom is retired and its register row (AD-116) deleted. If a development free-camera is still wanted later it is a separate explicit dev tool, not a modification of the retail camera; out of scope here. | + +**Modern code owns only mechanics**, and each substitution carries a proof +of pixel-neutrality: + +| Modern mechanic | Neutrality proof | +|---|---| +| Pre-built vertex/index buffers (WB-derived) | Same triangles — existing conformance suites | +| Bindless textures, Vulkan RHI | Resource binding strategy; changes no order, no coverage | +| Merging **adjacent** same-state draws into one MDI call | Order-preserving by construction; a merge across a state or stage boundary is forbidden | +| Async landblock streaming | The atomic-presence invariant (§4) makes async loading indistinguishable from retail's synchronous loading, minus the hitch | + +**The rule:** if a modern component ever makes a decision retail did not +make, that is a campaign bug regardless of how the pixels look. + +## 4. Architecture + +### Components (each one purpose, testable alone) + +- **`RetailFrameWalk`** (new, App/Rendering) — the ported recursion + + landscape walk. Pure CPU: input = world snapshot + camera pose; output = + the ordered draw stream and the frame's visible-cell set. No GPU types. + Unit-testable headless against synthetic worlds and against captured + retail traces (§5). +- **`PViewSet`** (new) — retail `PView` construction, chain-polygon + clipping, cone tests. Owned by the walk; no consumer sees clip planes + except through it. +- **`OrderedDrawStream`** (new) — the append-only frame command list: + (mesh ref, transform, material state, stage) tuples in walk order. +- **`OrderPreservingSubmitter`** (new, replaces the regrouping half of + `WbDrawDispatcher`) — walks the stream, merges adjacent same-state + commands into MDI calls, submits through the existing RHI. May never + reorder. The bindless/SSBO machinery beneath is reused as-is. +- **`WalkWorldSnapshot`** (new seam over existing streaming) — the atomic + view: a cell/landblock is either present-and-complete or absent. Leans + on the existing typed completion admission (Modern Runtime Slice E); the + walk consumes only committed-complete cells. **Invariant: no frame ever + walks a partially published cell.** This is the one place modern differs + from retail by design (async vs synchronous load) and it gets a + divergence-register row. +- **`RetailCameraController`** (extend existing camera) — re-land the + retail zoom envelope on top of the already-default-on chase/collide/ + slope-align behaviors; delete the dev-zoom deviation. + +### What gets deleted (the patch apparatus) + +`PortalVisibilityBuilder` as production visibility, the look-in seed +heuristics, exit-portal stencils, `ClipPlaneSet`'s gating role beyond chain +polygons, the +0.02 lift, the seal-panel depth stamps (quarantined, +never merged), the `InViewCells` side-channel (consumers — radar, lights, +particle culling — read the walk's visible-cell output instead: one gate, +computed once, enforced once). `RetailPViewRenderer`/`RetailPViewPassExecutor` +are absorbed: their already-correct stage knowledge moves into the walk; +the shells are retired at cutover. + +### Data flow (one frame) + +1. Streaming publishes committed-complete cells → `WalkWorldSnapshot`. +2. `RetailCameraController` produces the eye pose (retail rules). +3. `RetailFrameWalk` roots at the eye's cell (or outdoors), recurses/walks, + consults `PViewSet` per portal chain, gates objects per view, and + appends draws to `OrderedDrawStream` in retail stage order. +4. `OrderPreservingSubmitter` merges adjacent runs and submits via RHI. +5. The walk's visible-cell set feeds radar/lights/particles/audio. + +## 5. Conformance and gates + +- **The walk oracle (new tooling, built FIRST).** cdb attaches to the + paired 2013 retail client and captures the per-frame cell-draw sequence + (cell id order + stage boundaries) at canonical positions: the Sanctuary + terrace and its edge, the cathedral portal-in, a Holtburg doorway + (flap scene), a Holtburg street + house exit, one deep dungeon corridor. + Our `RetailFrameWalk`, fed the same world state and pose, must produce + the identical sequence. This gates the port BEFORE any pixel is judged. + (Toolchain and safety rules: CLAUDE.md "Retail debugger toolchain" + the + PV campaign's cdb lessons — no inline `-c` attach, every `j` branch ends + in `gc`.) +- **Pixel/behavior gates (owner-judged):** cathedral bleed gone at the + edge AND at normal camera (#456 acceptance); doorway flap gone; no + wall/floor/terrain seams in dungeons; clean portal-in, house exit, and + teleport timing. **The transition checklist (portal-in, house exit, + teleport stopwatch) is a standing self-run gate on every candidate build + before it reaches the owner** — the binding lesson of 2026-08-29/30. +- **Perf checkpoint EARLY (stage W3, before full commitment):** dense + Arwic uncapped with walk-order submission, measured in Release. Target: + within 20% of the current production profile. A larger regression stops + the campaign for an explicit owner decision — we measure, not hope. +- **Suites:** walk unit tests on synthetic worlds; captured-trace replay + tests checked into `docs/research/` fixtures; the standard hermetic + suites stay green at every stage; every ported function cites its named + retail symbol. + +## 6. Staged cutover (no long-lived dual path) + +Campaign V precedent: port behind the contract, then delete. Each stage +lands build+test green; visual stages get the standing transition +checklist self-run plus owner gates. + +- **W0** — Oracle tooling: the cdb walk-capture harness + recorded traces + at the canonical positions. Deliverable: replayable trace fixtures. +- **W1** — `RetailFrameWalk` + `PViewSet` as pure CPU modules, conformant + to the traces. No production rendering change yet. +- **W2** — `OrderedDrawStream` + `OrderPreservingSubmitter` against the + existing RHI; walk drives a diagnostic scene. +- **W3** — Static world cutover (terrain, EnvCells, buildings) to the + walk; the old gather path for statics is deleted in the same stage. + **The perf checkpoint lives here.** +- **W4** — Objects, entities, particles, dynamics move into walk turns; + delete the look-in machinery, stencils, stamps, lift, and the + `InViewCells` side-channel. +- **W5** — Camera port completion (retail zoom envelope; AD-116 retired). +- **W6** — Closeout: divergence-register reconciliation, docs, the full + gate matrix, roadmap/milestones update. + +## 7. Risks + +- **Perf** (walk-order submission loses cross-cell batching). Mitigation: + adjacency merging + bindless; the W3 checkpoint decides with numbers. +- **Streaming scope creep.** Guard: only the atomic-presence contract may + touch streaming; anything more is out of scope. +- **Oracle fidelity.** Retail traces need live retail sessions (owner + assists per the established cdb workflow); traces are captured once and + checked in as fixtures. +- **The quarantined PV branch.** Nothing merges from it. Salvageable + pieces (the degrade port, PV1's merged-hull deletion) are RE-LANDED + fresh on this campaign's stages if and when their stage needs them, + each with the transition checklist. The unidentified back-half culprit + stays quarantined until its machinery is deleted wholesale at W4. +- **Camera/dev workflow.** Retiring the 40 m zoom changes the owner's dev + ergonomics; the optional separate flycam tool is deliberately deferred + to keep this campaign's scope honest. + +## 8. Register impact + +Added: the streaming atomic-presence adaptation (W3); order-preserving +MDI merging as intentional architecture. Retired at their stages: the ++0.02 lift row, AD-116 (camera), AD-117 (stamps — dies with the +quarantine), and every look-in/stencil-era row the deletions obsolete. +Rule 1 of the register applies to every stage commit.