diff --git a/docs/plans/2026-04-11-roadmap.md b/docs/plans/2026-04-11-roadmap.md index ea586a03..16306685 100644 --- a/docs/plans/2026-04-11-roadmap.md +++ b/docs/plans/2026-04-11-roadmap.md @@ -518,6 +518,26 @@ Plus polish that doesn't get its own phase number: ## Phases ahead — agreed order +### Campaign FW — the retail frame walk (ACTIVE 2026-08-30) + +**Tagline:** retail owns every frame decision; modern code only moves bytes. + +**Filed:** 2026-08-30 after Campaign PV's halt (the PView graft onto the +gather/regroup renderer failed; owner verdict). Replaces the ~4.7K-line +accumulated visibility/composition layer (`PortalVisibilityBuilder`, +`RetailPViewRenderer`/`PassExecutor`, `CellVisibility` BFS, `ClipPlaneSet` +gating) and the dispatcher's material regrouping with a verbatim port of +retail's frame walk (`SmartBox::RenderNormalMode` rooting, +`PView::DrawCells` recursion, `LScape::draw`, building look-ins, stage +interleave, depth semantics, camera incl. the retail zoom envelope). The +WB-derived asset/residency layer is untouched. Gated by a cdb walk oracle +(retail's captured per-frame draw sequence as replayable fixtures), a +standing transition checklist, and a dense-Arwic perf stop-rule at FW3. +Stages FW0–FW6; no long-lived dual path. + +**Spec:** [`docs/superpowers/specs/2026-08-30-campaign-w-retail-frame-walk-design.md`](../superpowers/specs/2026-08-30-campaign-w-retail-frame-walk-design.md). +**Plan + ledger:** [`docs/plans/2026-08-30-campaign-fw-frame-walk.md`](2026-08-30-campaign-fw-frame-walk.md). + ### Phase O — DatPath Unification — SHIPPED 2026-05-21 **Tagline:** ONE thing touches the DATs. diff --git a/docs/plans/2026-08-30-campaign-fw-frame-walk.md b/docs/plans/2026-08-30-campaign-fw-frame-walk.md new file mode 100644 index 00000000..2eaf5624 --- /dev/null +++ b/docs/plans/2026-08-30-campaign-fw-frame-walk.md @@ -0,0 +1,331 @@ +# Campaign FW — the retail frame walk (implementation plan) + +**Status:** ACTIVE (opened 2026-08-30; spec approved by owner 2026-08-30). +**Spec (approved):** +[`docs/superpowers/specs/2026-08-30-campaign-w-retail-frame-walk-design.md`](../superpowers/specs/2026-08-30-campaign-w-retail-frame-walk-design.md) +— note: the campaign letter is **FW** (frame walk); the spec predates the +rename (the roadmap already owns "Phase W" for the historical Unified Cell +Graph program). +**Branch:** `claude/campaign-w-retail-frame-walk` off main `e102fb36`. +**Predecessor:** Campaign PV, halted 2026-08-30 +(`docs/plans/2026-08-30-pview-visibility-campaign.md`); its back half is +quarantined on `claude/cathedral-rendering-debug-47a417` and nothing merges +from it. + +**One-sentence goal:** retail owns every frame decision (rooting, portal +views, cone gating, landscape walk, buildings, stage order, depth +semantics, degrade, particles, camera); modern code owns only pixel-neutral +GPU mechanics; the ~4.7K-line accumulated visibility layer and its patch +apparatus are deleted at their stages. + +--- + +## Binding rules (all stages, binding on subagents) + +1. **The ownership line is the campaign invariant.** If a modern component + makes a decision retail did not make, that is a campaign bug regardless + of pixels. Every ported function cites its named retail symbol. +2. **Grep named first.** `docs/research/named-retail/acclient_2013_pseudo_c.txt` + by `class::method` before any fresh decompilation; pseudocode notes in + `docs/research/` before C# (the mandatory workflow). +3. **Fail loud, never silently drop.** The PV3 post-mortem rule: any + admission/seeding/ordering filter that rejects work must log the + rejection under a diagnostic flag or fall open — silent drops produced + this week's hollow-world regressions. +4. **The transition checklist is a standing self-run gate.** Every + candidate build gets, before the owner sees it: portal-in arrival + frames, a Holtburg house exit, a teleport-to-Holtburg stopwatch, and a + steady-state fps read (drive.ps1 + screenshots). No terrace-only + verification, ever again. +5. **No long-lived dual path.** Campaign V precedent: a stage that cuts + over deletes the code it replaces in the same stage. Short-lived + scaffolding inside one stage is fine; a toggle that survives a stage + boundary is not. +6. **Register discipline.** Deviations add their row in the same commit; + ported mechanisms delete rows in the same commit + (`docs/architecture/retail-divergence-register.md`). +7. **Suites green at every stage close** (hermetic filter + `Lane!=InstalledDat&Lane!=Linux&Lane!=Manual&Lane!=Timing&Lane!=Live&Lane!=PreparedPackage`), + plus the stage's own gates. Sonnet implements; Opus reviews stage + boundaries (architectural + retail-faithfulness lenses). + +## Verified seam map (recon 2026-08-30; cite these, do not re-derive) + +**Frame chain today:** `GameWindow.OnRender` (GameWindow.cs:1785/1809) → +`GameFrameGraphSlot.Render` → `RenderFrameOrchestrator.Render` +(RenderFrameOrchestrator.cs:206/226) → `VulkanWorldScenePhase` +(VulkanCompositionFramePhases.cs:113, owns the backbuffer pass) → +`WorldSceneRenderer.Render` (WorldSceneRenderer.cs:128) → +`RetailPViewRenderer.DrawInside` (RetailPViewRenderer.cs:112), which builds +`PortalVisibilityFrame` internally (line 122) and drives +`RetailPViewPassExecutor` through the packed route contract +(`RenderFrameCandidateRoute`: LandscapeOutdoorStatic → LandscapeBuildingShell +→ LookInObject → LandscapeOutsideDynamic → CellStatic → DynamicLast; +`WbDrawDispatcher.PackedOracle.cs:108/171` enforces in-order consumption). + +**What the walk-order submitter REPLACES:** `GroupKey` material bucketing +(`ClassifyBatches` WbDrawDispatcher.cs:3292, `ClassifyPackedBatches` +PackedOracle.cs:610, `_groups`/`_packedGroups`), `PartitionInstanceGroups` +(:2232), and the two distance sorts (`CompareOpaqueSubmissionOrder` :2865, +`CompareTransparentSubmissionOrder` :2871). Note: the deferred-alpha path +(`DeferTransparentGroups` :2619 → `RetailAlphaQueue`) already reconstructs +draw-local order per instance — the transparent side is half-way to +walk-order today and is the SSBO-layout pattern to follow. + +**What is REUSED unchanged:** `GlobalMeshBuffer` arena (GlobalMeshBuffer.cs:84), +`WbMeshAdapter`/`ObjectMeshManager` mesh prep + self-heal +(WbMeshAdapter.cs:328/435), `TextureCache` GpuTextureSlot resolution +(WbDrawDispatcher.cs:3421), the ring-section SSBO writers + shared +`WorldTransformFrameArena` (Rhi.cs:913/945), `MeshPipelineSet` (Rhi.cs:49), +and the terminal MDI recorder `DrawIndirectRangeRhi` (Rhi.cs:798). + +**Caveats that shaped the stages:** three parallel classifiers exist +(classic, packed, directional-shadow — WbDrawDispatcher.DirectionalShadows.cs:1021) +and must agree until each is cut over; `EntityClassificationCache` and +`PackedProjectionClassificationCache` assume group-bucketed submission and +retire with their routes; `StageImmediateGroup` lays per-instance SSBOs by +group-order cursor, so the walk submitter re-derives layout from stream +order (start per-instance like deferred alpha; merge adjacent runs after +correctness). + +**Visibility consumers to re-point at the walk output** (the complete +production list — radar, audio, and picking do NOT consume visibility): +- Particles: `WorldSceneRenderer.cs:263` → `ParticleVisibilityController` + → `ParticleSystem.ApplyRetailView` (Core/Vfx/ParticleSystem.cs:313), fed + outdoors by `TerrainModernRenderer.VisibleCellIds`. +- Lights: `WorldSceneRenderer.cs:264` → `RuntimeWorldFrameEnvironmentPreparation` + (WorldRenderFrameBuilder.cs:543) → `LightManager.BuildPointLightSnapshot` + (Core/Lighting/LightManager.cs:284). +- EnvCell prepare filter: `RetailPViewPassExecutor.cs:276` → + `EnvCellRenderer.PrepareRenderBatches` (EnvCellRenderer.cs:456). +- Entity routes: `RenderScenePViewFrameProduct.cs:1330-1587`. +- Gated diagnostics only beyond that. + +**Streaming atomicity (what already holds vs what FW adds):** per-cell +publication is ALREADY atomic — one dictionary swap of a complete +immutable `EnvCellLandblock` (EnvCellRenderer.CommitPublication:347); +outdoor terrain commits at LandblockRenderPublisher.AdvanceBeginOne:267. +The real partial states are (a) landblock-stage terrain-before-interiors +across yielded publication stages (LandblockPresentationPipeline.cs:711/757 +— deliberate walking pop-in, hidden by the reveal hold during +portal/login), and (b) committed cells drawing before every contained +static mesh is resident (skip-if-missing, EnvCellRenderer.cs:720). FW keeps +both as EXPLICIT, register-rowed adaptations; the walk consumes only the +committed registry (`CellVisibility.TryGetCell` — production role is the +registry; its BFS is already dead code). + +**The patch apparatus to delete (exact sites):** +- `ShellDrawLiftZ` +0.02 lift, three consumers: PortalVisibilityBuilder.cs:353 + (const + Build's drawLiftZ), RetailPViewPassExecutor.cs:697 (depth fans), + LandblockBuildFactory.cs:523 (baked into draw cellTransform). +- Punch stencils (#117 adaptation): PortalDepthMaskRenderer.Rhi.cs:59-125. +- Look-in seeding heuristics: PortalVisibilityBuilder.BuildFromExterior:713-923 + + callers MergeNearbyBuildingFloods (RetailPViewRenderer.cs:411), + BuildInteriorRootLookIns (:471), DrawBuildingLookIns (:600-860), and the + barrier math (:515-559). +- The InViewCells side-channel (RetailPViewRenderer.cs:2187) once consumers + read the walk output. + +**Camera facts:** `RetailChaseCamera.cs` — `Distance` default 2.61 m, +clamps DistanceMin=2/DistanceMax=40 (:110-113); collision probe + slope +align + chase already default-on retail behaviors +(Core/Rendering/CameraDiagnostics.cs:28-50). Retail envelope (from the +d71f5ef1 diff, which lives ONLY on the quarantined branch — this branch has +no AD-116 row): `CameraSet::Farther` @0x00458890 scales the viewer offset +multiplicatively and REFUSES the write once |offset.y| ≥ 10 (gates |x|<10, +|y|<10, z<450, z>-1.8); the standard arc saturates near offset +(0, −10, +8.33) ≈ a 13.0 m boom. `CameraSet::Raise` @0x00457980 rotates at +preserved length. TS-56 records the non-retail adjustment scalars +(CameraAdjustmentSpeed=40/s etc.). + +**Retail anchors already established:** `SmartBox::RenderNormalMode` +@0x00453aa0 (frame rooting), `PView::DrawCells` @0x005A4840, +`LScape::draw` @0x00506330, `LScape::grab_visible_cells` @0x00504EC0, +`DrawBuilding` pre-punch flush @0x0059F2A0, `ShouldDrawParticles` +@0x0050FE60, the untextured skip in `D3DPolyRender::DrawMesh` @0x0059d4a0, +`ACRender::polyClipFinish` (two call sites: `PView::GetClip` + debug), +`Render::viewconeCheck`. + +--- + +## Stages + +### FW0 — the walk oracle (tooling + fixtures) + +**Goal:** retail's actual per-frame draw walk, captured and replayable, +BEFORE any port is judged. + +- Build the cdb capture harness (`tools/walk-oracle/`): breakpoint script + logging, per frame, the sequence of cell/building draw entries with + stage boundaries — hook `PView::DrawCells`, `CEnvCell` draw, + `CLandBlock`/`LScape::draw`, `DrawBuilding`; dump cell ids (`this` + offsets verified with one-shot `dt` first). cdb safety rules from the PV + campaign are binding: never inline `-c` attach to a client that matters, + every `j` branch ends in `gc`, `qd` threshold auto-detach, watch hit-rate + lag. +- Capture positions (owner assists — retail sessions): Sanctuary terrace + center + terrace EDGE (normal camera — the #456 acceptance pose), + cathedral portal-in arrival, a Holtburg doorway (the flap scene), a + Holtburg street + house exit, one deep dungeon corridor. Record camera + pose per capture. +- Also capture the same positions in acdream (world state: cell registry + contents) so W1 replays are apples-to-apples. +- Pin the port function list as a research note with pseudocode stubs: + `docs/research/2026-08-30-fw-walk-pseudocode.md` (grow it through FW1). +- **Deliverable/gate:** trace fixtures checked into + `docs/research/2026-08-30-fw-walk-oracle/`, replayable by a test + helper; harness scripts committed; no production code touched. + +### FW1 — `RetailFrameWalk` + `PViewSet` (pure CPU, conformant) + +**Goal:** the ported walk as a standalone module that reproduces the +oracle traces exactly. + +- New `src/AcDream.App/Rendering/Walk/`: `RetailFrameWalk` (rooting per + `SmartBox::RenderNormalMode`; EnvCell recursion per `PView::DrawCells`; + landscape walk per `LScape::draw`/`grab_visible_cells`; building + look-ins per `DrawBuilding` + `ConstructView(CBldPortal)`), + `PViewSet` (`ConstructView`/`set_view`/`PView::GetClip` — the chain + polygon is the only CPU-clipped geometry; one view per chain, never + merged), `IWalkWorldView` (the read seam: committed cells via the + registry, buildings via `BuildingRegistry`, camera pose). +- Output: an ordered walk event stream (cell id, view, stage) matching the + oracle trace format, plus the frame's visible-cell set. +- Pseudocode first (workflow step 3), C# port line-by-line (step 4), named + symbols in comments. +- **Gates:** oracle-trace replay tests pass position-for-position; + synthetic-world unit tests for rooting/recursion/cone edge cases; the + existing replay fixture suite (Issue95/113/120/124/127/130/131/176-177/ + 181, SanctuaryPortalSeam, HouseExitWalkReplay, TowerAscentReplay, + CornerFloodReplay) re-expressed against the walk — behavior differences + vs the old builder are adjudicated against the ORACLE, not against the + old builder. No production wiring; hermetic suites green. + +### FW2 — `OrderedDrawStream` + `OrderPreservingSubmitter` + +**Goal:** walk-order submission through the existing RHI, proven +pixel-equivalent on static content. + +- `OrderedDrawStream`: append-only (mesh ref, transform, material state, + stage, cell) commands in walk order. `OrderPreservingSubmitter`: binds + `GlobalMeshBuffer` once, reuses ring-section writers + `MeshPipelineSet` + + `DrawIndirectRangeRhi`; emits per-instance first (the deferred-alpha + SSBO pattern), then merges ADJACENT same-state commands into MDI runs — + a merge across a state or stage boundary is forbidden by construction + (assert it). +- Depth semantics: port retail's depth compare/write configuration for the + world passes verbatim in this stage (current set uses Less — verify + against retail's raster state and document the finding either way). +- Diagnostic harness: a dev scene (offscreen target) drawn twice — old + path vs walk path — framebuffer-compared (the Slice-7 comparison + pattern, with the pixel-diff repeat floor from + `feedback_pixel_diff_needs_repeat_floor`). +- **Gates:** submitter unit tests (order preservation, merge legality); + framebuffer equivalence on static-content scenes; hermetic suites green. + +### FW3 — static world cutover + THE perf checkpoint + +**Goal:** terrain, EnvCell shells, and buildings draw via the walk in +production; the statics gather path is deleted; perf is measured before +deeper commitment. + +- Root the production frame at the walk: `WorldSceneRenderer` drives + `RetailFrameWalk` + submitter for the static routes + (LandscapeOutdoorStatic, LandscapeBuildingShell, CellStatic); dynamic + routes (LookInObject, LandscapeOutsideDynamic, DynamicLast) stay on the + packed path within this stage only (the route-contract split makes the + seam clean). +- Retire the three `ShellDrawLiftZ` sites in this stage — walk order + + retail depth semantics replace the tie-break; prove it at the #130 + doorway-strip scene before deleting, and delete the register row in the + same commit. +- Terrain/sky/weather draw at their walk turns (the stage interleave from + `RetailPViewPassExecutor`'s ported knowledge moves inward). +- **Perf checkpoint (the spec's stop rule):** dense Arwic uncapped, + Release, exact binary: CPU/GPU p50/p99 + fps vs the current production + profile (519.7 fps / 1.869 / 1.096 ms baseline). Within 20% → proceed; + worse → STOP, owner decision with numbers on the table. +- **Gates:** transition checklist self-run; hermetic suites; owner visual + gate (statics look right indoors + outdoors, no seam strip at the + doorway, no z-fighting at shells); perf numbers recorded in this plan. + +### FW4 — entities, particles, dynamics into walk turns; delete the apparatus + +**Goal:** everything draws at its walk turn; the patch apparatus and the +old visibility layer are gone. + +- Entities gate per view via `Render::viewconeCheck` at their cell's walk + turn; dynamics-last and alpha interleave per the walk's stages + (`RetailAlphaQueue` becomes the stream's alpha stage or is absorbed — + decide at implementation with the retail stage list as the arbiter); + particles draw at their owner cell's turn (`ShouldDrawParticles`), + unclipped. +- Re-point the consumers at the walk's visible-cell output: particles + (`ParticleVisibilityController`), lights (`BuildPointLightSnapshot` + filter), EnvCell prepare filter. Directional-shadow caster selection + consumes the same set. +- DELETE in this stage (registry rows updated in the same commits): + look-in seeding + callers + barrier math; punch stencils + (`PortalDepthMaskRenderer` #117); the InViewCells side-channel; + `PortalVisibilityBuilder`'s production role; `RetailPViewRenderer` / + `RetailPViewPassExecutor` shells (their stage knowledge now lives in the + walk); the packed route machinery + `PackedOracle` + both + classification caches; `CellVisibility`'s dead BFS. +- **Gates:** the full re-expressed replay suite; transition checklist; + hermetic suites; the owner visual matrix — cathedral #456 acceptance + (far building, monsters, stalks hidden at the terrace edge at NORMAL + camera; no seam strip; no z-fighting; candle/falls/lake/portal-flood + intact), doorway flap, dungeon wall/floor seams, portal-in, house exit, + teleport timing. + +### FW5 — camera completion + +**Goal:** the camera is retail's, end to end. + +- Port `CameraSet::Farther`/`CameraSet::Raise` verbatim into + `RetailChaseCamera`: the multiplicative offset scaling with the + write-refusal gates (|x|<10, |y|<10, z<450, z>−1.8 — NOT a simple + distance clamp), the preserved-length pitch. The 40 m dev zoom goes + away with it (owner-approved in the spec round). +- Port retail's zoom/pitch adjustment rates if recoverable from decomp + (retires TS-56); if not recoverable, TS-56 stays with a note. +- **Gates:** camera unit tests against ported constants; owner + side-by-side feel gate vs retail (chase, zoom saturation ≈13 m boom, + collision, slope align). + +### FW6 — closeout + +- Divergence-register reconciliation (added: landblock-stage streaming + exposure rows if not already rowed, order-preserving MDI merging as + intentional architecture; retired: lift, #117 stencils, look-in-era + rows, TS-56 if ported). +- Docs: architecture doc render section rewritten around the walk; + `worldbuilder-inventory.md` updated (WB = asset layer only); + `launch-options.md` rows for any probes added/removed (probes die with + their investigations). +- Memory: `project_render_pipeline_digest.md` rewritten current-truth-on-top + around the walk; campaign memory closed out. +- Roadmap + milestones updated; the quarantined PV branch gets a + disposition note (kept as archive; nothing merges). +- Final gates: full hermetic suites, both standard connected routes, + the complete owner visual matrix, merge to main. + +## Rollbacks + +Each stage lands as its own commit series; record `git revert` anchors +here as stages close (the Modern Runtime plan's convention). FW3 and FW4 +are the cutover stages — their revert anchors are mandatory entries. + +## Risks (tracked, from the spec) + +- **Perf** — decided by FW3's checkpoint, numbers over hope. +- **Streaming scope creep** — only the two register-rowed exposure + adaptations may be touched; anything more is out of scope. +- **Oracle fidelity** — traces need owner-assisted retail sessions (FW0); + captured once, checked in. +- **Classifier drift during staging** — FW3's split runs walk statics + beside packed dynamics for one stage; the route contract seam is the + guard, and FW4 deletes the duality. +- **Camera ergonomics** — the dev zoom disappears at FW5; a separate dev + flycam is deliberately deferred (the Fly camera in `CameraController` + already exists for development framing). diff --git a/docs/superpowers/specs/2026-08-30-campaign-w-retail-frame-walk-design.md b/docs/superpowers/specs/2026-08-30-campaign-w-retail-frame-walk-design.md index 665db04b..9bece6bb 100644 --- a/docs/superpowers/specs/2026-08-30-campaign-w-retail-frame-walk-design.md +++ b/docs/superpowers/specs/2026-08-30-campaign-w-retail-frame-walk-design.md @@ -1,6 +1,10 @@ -# Campaign W — the retail frame walk (rendering redesign design spec) +# Campaign FW — the retail frame walk (rendering redesign design spec) -**Status:** DRAFT — awaiting owner review (2026-08-30). +**Status:** APPROVED by owner 2026-08-30. Implementation plan: +[`docs/plans/2026-08-30-campaign-fw-frame-walk.md`](../../plans/2026-08-30-campaign-fw-frame-walk.md). +**Naming note:** approved as "Campaign W"; renamed to **FW** because the +roadmap already owns "Phase W" (the historical Unified Cell Graph program). +Stage ids are FW0–FW6. **Owner directive:** "I want this as solid as retail, no weird seams between walls or floors, it should work as retail. As stable as retail." Scope confirmed 2026-08-30: retail owns every frame decision, modern code owns