phase(N.4): SHIP — flag default-on + finalize plan + roadmap
Phase N.4 (Rendering Pipeline Foundation) ships. WbFoundationFlag flips to default-on (== "1" → != "0"). WB's ObjectMeshManager is now acdream's production mesh pipeline; WbDrawDispatcher is the production draw path. Legacy InstancedMeshRenderer is retained as ACDREAM_USE_WB_FOUNDATION=0 escape hatch until N.6 retires it. Visual verification at Holtburg passed: - Scenery (trees / rocks / fences / buildings) renders correctly - Characters connected with full close-detail geometry (Issue #47 preserved — GfxObjDegradeResolver path intact) - FPS substantially improved by grouped instanced draws + per-entity AABB cull + opaque front-to-back sort + palette-hash memoization Three high-value WB API gotchas surfaced during Task 26 visual verification and are now documented in CLAUDE.md "WB integration cribs" + plan Adjustments 7-9 + memory project_phase_n4_state.md: 1. ObjectMeshManager.IncrementRefCount only bumps a counter — does NOT trigger mesh loading. Call PrepareMeshDataAsync explicitly. 2. ObjectRenderBatch.SurfaceId is unset — read batch.Key.SurfaceId. 3. Modern rendering (GL 4.3 + bindless = every modern GPU) packs every mesh into ONE global VAO/VBO/IBO. Use glDrawElementsInstancedBaseVertex(BaseInstance) with FirstIndex + BaseVertex from the batch, not naive DrawElementsInstanced. Plan doc flipped to Final state. Roadmap N.4 → Live ✓; N.5 rebranded from "Terrain rendering" to "Modern rendering path" (bindless + multi-draw indirect on top of N.4's foundation; terrain rendering moves to N.5b). CLAUDE.md "Currently in flight" pointer updated to N.5. New memory file project_phase_n4_state.md preserves the three WB gotchas for cross-session continuity. n4-verify*.log added to .gitignore. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Erik
parent
573526dae5
commit
c44536451d
5 changed files with 178 additions and 58 deletions
|
|
@ -1,6 +1,6 @@
|
|||
# acdream — strategic roadmap
|
||||
|
||||
**Status:** Living document. Updated 2026-05-08 for Phase N.3 shipping + N.4-N.9 strategy revision (rendering rebuild on shared WB infrastructure rather than independent substitutions).
|
||||
**Status:** Living document. Updated 2026-05-08 for Phase N.4 shipping (`WbMeshAdapter` + `WbDrawDispatcher` + `ACDREAM_USE_WB_FOUNDATION` default-on) + N.5 rebranded to "Modern rendering path" (bindless + multi-draw indirect on top of N.4's foundation).
|
||||
**Purpose:** One source of truth for where the project is and where it's going. Every observed defect or missing feature has a named phase that owns it; when something looks wrong in-game, look here to find the phase that'll address it. Implementation details live in per-phase specs under `docs/superpowers/specs/`, not in this file.
|
||||
|
||||
---
|
||||
|
|
@ -59,6 +59,7 @@
|
|||
| C.1 | PES particle system + sky-pass refinements — retail-faithful `ParticleEmitterInfo` unpack with all 13 motion integrators (`Particle::Init`/`Update` ports of `0x0051c290`/`0x0051c930`), `PhysicsScriptRunner` with `CallPES` self-loop semantics, `ParticleHookSink` with `EmitterDied` cleanup, instanced billboard `ParticleRenderer` with material-derived blend (DAT emitters never default additive — pulled from particle GfxObj surface), global back-to-front sort, BC clipmap alpha-keying, AttachLocal `is_parent_local=1` live-parent follow via `UpdateEmitterAnchor`. Sky pass: `Translucent+ClipMap` → alpha-blend cloud sheet (matches `D3DPolyRender::SetSurface` `0x0059c4d0`), raw-`Additive` fog-skip (matches `0x0059c882`), per-keyframe `SkyObjectReplace` Translucency/Luminosity/MaxBright divide-by-100, bit `0x01` pre/post-scene split (matches `GameSky::CreateDeletePhysicsObjects` `0x005073c0`), Setup-backed (`0x020xxxxx`) sky objects via `SetupMesh.Flatten`, persistent GL sampler objects (Wrap + ClampToEdge) replace per-frame wrap-mode mutation (ported from WorldBuilder's `OpenGLGraphicsDevice`), post-scene Z-offset gated on `(Properties & 4) != 0 && (Properties & 8) == 0` per `GameSky::UpdatePosition` `0x00506dd0`. Sky-PES playback disabled by default (named-retail proves `GameSky` drops `pes_id`); `ACDREAM_ENABLE_SKY_PES=1` opens the experimental path. 1325 → 1331 tests. | Live ✓ |
|
||||
| N.1 | WorldBuilder-backed scenery (Chorizite/WorldBuilder fork as submodule, SceneryHelpers + TerrainUtils replace our inline ports) | Live ✓ |
|
||||
| N.3 | WorldBuilder-backed texture decode — `SurfaceDecoder` delegates INDEX16 / P8 / A8R8G8B8 / R8G8B8 / A8(+Additive) to `TextureHelpers.Fill*`; `isAdditive` threaded through (terrain alpha → `FillA8Additive`, non-additive entity surfaces → `FillA8`). R5G6B5 + A4R4G4B4 newly handled (previously magenta). X8R8G8B8, DXT1/3/5, SolidColor remain ours (no WB equivalent). 9 conformance tests prove byte-identical equivalence per format. | Live ✓ |
|
||||
| N.4 | Rendering pipeline foundation — adopted WB's `ObjectMeshManager` as the production mesh pipeline behind `ACDREAM_USE_WB_FOUNDATION` (default-on). `WbMeshAdapter` is the single seam (owns `ObjectMeshManager`, drains the staged-upload queue per frame, populates `AcSurfaceMetadataTable` with per-batch translucency / luminosity / fog metadata). `WbDrawDispatcher` is the production draw path: groups all visible (entity, batch) pairs, single-uploads the matrix buffer, fires one `glDrawElementsInstancedBaseVertexBaseInstance` per group with `BaseInstance` slicing into the shared instance VBO. `LandblockSpawnAdapter` + `EntitySpawnAdapter` bridge spawn lifecycle to WB ref-counts (atlas tier vs per-instance). Perf wins shipped as part of N.4: per-entity frustum cull, opaque front-to-back sort, palette-hash memoization (compute once per entity, reuse across batches). Visual verification at Holtburg passed: scenery + connected characters with full close-detail geometry (Issue #47 regression resolved). Legacy `InstancedMeshRenderer` retained as `ACDREAM_USE_WB_FOUNDATION=0` escape hatch until N.6. | Live ✓ |
|
||||
|
||||
Plus polish that doesn't get its own phase number:
|
||||
- FlyCamera default speed lowered + Shift-to-boost
|
||||
|
|
@ -604,36 +605,54 @@ for our deletions/additions; merge upstream `master` periodically.
|
|||
byte-identical equivalence per format** before substitution; updated
|
||||
`SurfaceDecoderTests` to match the new A8 split semantics. Visual
|
||||
verification at Holtburg passed 2026-05-08 — no texture regressions.
|
||||
- **N.4 — Rendering pipeline foundation.** **Rebranded from "object
|
||||
meshing" 2026-05-08 after brainstorm.** WB's `ObjectMeshManager` is
|
||||
not a static helper — it's a 2070-line stateful asset pipeline that
|
||||
owns GPU resources (VAO/VBO/IBO), an LRU cache + memory budget,
|
||||
background staging, a shared texture atlas, and a bindless rendering
|
||||
path. Adopting it wholesale is the foundation that N.5 + N.6 + N.7
|
||||
build on. Concretely: (1) integrate `ObjectMeshManager` +
|
||||
`TextureAtlasManager` as the shared infrastructure; (2) build a
|
||||
per-instance customization layer that threads `CreaturePalette` /
|
||||
`GfxObjRemapping` / `HiddenParts` / `TextureChanges` / `SubPalettes` /
|
||||
`AnimPartChange` through WB's atlas keys; (3) extend WB's
|
||||
`MeshBatchData` to carry our surface metadata (`Translucency` /
|
||||
`Luminosity` / `Diffuse` / `SurfOpacity` / `NeedsUvRepeat` /
|
||||
`DisableFog`) — likely a fork patch on the `acdream` branch; (4)
|
||||
decide animation cache strategy (per-frame transform via uniform/SSBO
|
||||
vs. cache invalidation); (5) adapter from our streaming loader's
|
||||
Setup/Static spawn events to WB's `IncrementRefCount` lifecycle.
|
||||
**Estimate: 3-4 weeks.** No visible change yet — visual verification =
|
||||
"world looks identical to before." Foundation enables the next phases.
|
||||
- **N.5 — Terrain rendering.** Wire WB's `TerrainRenderManager` +
|
||||
`LandSurfaceManager` + `TerrainGeometryGenerator` onto the foundation
|
||||
N.4 builds. Closes N.2's deferred terrain math substitution: visual
|
||||
mesh and physics both switch to WB's `CalculateSplitDirection` +
|
||||
`GetHeight` + `GetNormal` in lockstep, resolving ISSUE #51. **Estimate:
|
||||
2-3 weeks** (was 3-4 — atlas + GPU pipeline already in place from N.4).
|
||||
- **✓ SHIPPED — N.4 — Rendering pipeline foundation.** Shipped 2026-05-08.
|
||||
WB's `ObjectMeshManager` is integrated as the production mesh pipeline
|
||||
behind `ACDREAM_USE_WB_FOUNDATION=1` (default-on). The integration is
|
||||
three pieces: `WbMeshAdapter` (single seam owning the WB pipeline,
|
||||
drains the staged-upload queue per frame, populates
|
||||
`AcSurfaceMetadataTable` for translucency / luminosity / fog),
|
||||
`WbDrawDispatcher` (production draw path — groups all visible
|
||||
(entity, batch) pairs, uploads matrices in a single `glBufferData`,
|
||||
fires one `glDrawElementsInstancedBaseVertexBaseInstance` per group
|
||||
with `BaseInstance` slicing the shared instance VBO), and the
|
||||
`LandblockSpawnAdapter` + `EntitySpawnAdapter` bridge that wires our
|
||||
streaming loader to WB's `IncrementRefCount` / `PrepareMeshDataAsync`
|
||||
lifecycle (atlas tier vs per-instance customized).
|
||||
Issue #47 (close-detail mesh) preserved; sky pass structurally
|
||||
independent of the WB foundation. Perf wins shipped as part of N.4:
|
||||
per-entity AABB frustum cull, opaque front-to-back sort, palette-hash
|
||||
memoization. Legacy `InstancedMeshRenderer` retained as flag-off
|
||||
fallback until N.6 fully retires it. Plan archived at
|
||||
`docs/superpowers/plans/2026-05-08-phase-n4-rendering-foundation.md`.
|
||||
- **N.5 — Modern rendering path.** **Rebranded from "Terrain rendering"
|
||||
2026-05-08 after N.4 perf review.** N.4 left two big remaining wins
|
||||
on the table that pair naturally: (1) bindless textures via
|
||||
`GL_ARB_bindless_texture` (WB already populates
|
||||
`ObjectRenderBatch.BindlessTextureHandle`; switch our shader to
|
||||
consume per-instance handles, eliminate 100% of `glBindTexture`
|
||||
calls), and (2) `glMultiDrawElementsIndirect` (one GL call per pass
|
||||
instead of one per group; build a `DrawElementsIndirectCommand`
|
||||
buffer, fire one indirect draw, the driver pulls everything). Both
|
||||
require shader changes (same shader, in fact — bindless + indirect
|
||||
are the same modern path WB uses internally). Together they target a
|
||||
2-5× CPU win on draw-heavy scenes (Holtburg courtyard, Foundry,
|
||||
dense dungeons). Also folds in: persistent-mapped instance VBO
|
||||
(`glBufferStorage` + `MAP_PERSISTENT_BIT | MAP_COHERENT_BIT` + ring
|
||||
buffer + sync) and texture pre-warm at landblock load (smooths
|
||||
streaming-boundary hitches). **Estimate: 2-3 weeks.**
|
||||
- **N.5b — Terrain rendering on N.5 path.** Wire WB's
|
||||
`TerrainRenderManager` + `LandSurfaceManager` + `TerrainGeometryGenerator`
|
||||
onto the modern rendering path. Closes N.2's deferred terrain math
|
||||
substitution: visual mesh and physics both switch to WB's
|
||||
`CalculateSplitDirection` + `GetHeight` + `GetNormal` in lockstep,
|
||||
resolving ISSUE #51. **Estimate: 1-2 weeks** (was 2-3 — modern path
|
||||
primitives already in place from N.5).
|
||||
- **N.6 — Static objects rendering.** Wire WB's
|
||||
`StaticObjectRenderManager` onto N.4's foundation; replace our
|
||||
`StaticMeshRenderer` + `InstancedMeshRenderer`. Mostly draw
|
||||
orchestration at this point — most of the substance landed in N.4.
|
||||
**Estimate: 1-2 weeks** (was 2-3).
|
||||
`StaticObjectRenderManager` onto the modern rendering path; **fully
|
||||
delete** legacy `StaticMeshRenderer` + `InstancedMeshRenderer` (they
|
||||
remain as `ACDREAM_USE_WB_FOUNDATION=0` escape hatches through N.5).
|
||||
Mostly draw orchestration at this point — most of the substance
|
||||
landed in N.4 + N.5. **Estimate: 1-2 weeks** (was 2-3).
|
||||
- **N.7 — EnvCells / dungeons.** Replace EnvCell rendering with WB's
|
||||
`EnvCellRenderManager` + `PortalRenderManager` on top of N.4's
|
||||
foundation. **Estimate: 1-2 weeks** (was 2-3 — naturally smaller now
|
||||
|
|
|
|||
|
|
@ -64,7 +64,12 @@ This plan is the **execution source of truth** for N.4. It is updated as tasks l
|
|||
- If a downstream task changes shape because of an earlier task's outcome, append the changes to the downstream task in-place rather than scattering deltas.
|
||||
- Final commit for the phase updates this header note from "Living document — work in progress" to "Final state at <date> — phase shipped (merge `<sha>`)."
|
||||
|
||||
Status: **Living document — work in progress, started 2026-05-08.**
|
||||
Status: **Final state at 2026-05-08 — phase shipped.** All tasks
|
||||
complete; `ACDREAM_USE_WB_FOUNDATION` flipped default-on. Visual
|
||||
verification at Holtburg passed. Three bugs surfaced + resolved during
|
||||
Task 26 are documented as Adjustments 7-9 below and as gotchas in
|
||||
CLAUDE.md. Followup work moves to N.5 (modern rendering path: bindless
|
||||
+ multi-draw indirect).
|
||||
|
||||
**Progress (2026-05-08):** Weeks 1 + 2 + 3 ✅ COMPLETE. WB pipeline running flag-on (constructed + ref-counted + per-frame Tick draining its queues). Per-instance tier wired (`EntitySpawnAdapter` routes server-spawned entities through existing `TextureCache.GetOrUploadWithPaletteOverride` path; per-entity `AnimatedEntityState` accumulates AnimPartChange + HiddenParts data, ready for the dispatcher). Five architectural adjustments documented: 1 (DefaultDatReaderWriter discovery), 2 (renderer is tier-blind), 3 (FPS regression = dual-pipeline cost; resolves at Task 22), 4 (WorldEntity missing HiddenPartsMask + AnimPartChanges fields, plumbing deferred), 5 (Task 20 is structural — same function called both paths). Build green, 947 tests pass, 8 pre-existing failures only.
|
||||
|
||||
|
|
@ -93,11 +98,14 @@ Status: **Living document — work in progress, started 2026-05-08.**
|
|||
| 20 — Per-instance decode conformance | ✅ structural (Adj. 5) | (no test file) |
|
||||
| 21 — Week 3 wrap-up | ✅ | (this commit) |
|
||||
| 22+23 — WbDrawDispatcher + side-table population | ✅ | `01cff41` |
|
||||
| 22+23 fixup — load triggers + SurfaceId source | ✅ | `943652d` |
|
||||
| 22+23 perf — FirstIndex/BaseVertex + #47 + grouped instanced | ✅ | `7b41efc` |
|
||||
| 22+23 perf 1-4 — drop dead lookup, sort, cull, hash memo | ✅ | `573526d` |
|
||||
| 24 — Sky-pass preservation check | ✅ structural (independent) | `5df9135` |
|
||||
| 25 — Component micro-tests round-out | ✅ all spec tests covered | — |
|
||||
| 26 — Visual verification + flag default-on | pending | — |
|
||||
| 27 — Delete legacy code paths | pending | — |
|
||||
| 28 — Update memory + ISSUES + finalize plan | pending | — |
|
||||
| 26 — Visual verification + flag default-on | ✅ | (this commit) |
|
||||
| 27 — Delete legacy code paths | ⚠️ deferred to N.6 (legacy retained as flag-off escape hatch) | — |
|
||||
| 28 — Update memory + ISSUES + finalize plan | ✅ | (this commit) |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -1016,6 +1024,52 @@ plumbing decision to Task 22. Two options:
|
|||
- GameWindow's CreateObject handler builds the `PartOverride[]` from the
|
||||
server-sent `AnimPartChanges` list.
|
||||
|
||||
### Adjustment 7 (2026-05-08, Task 26 visual verification): IncrementRefCount doesn't trigger mesh load
|
||||
|
||||
**Discovered when** Task 26's first launch showed only terrain — zero entities visible. Diagnostic counters (added the same launch via `ACDREAM_WB_DIAG=1`) showed `entitiesSeen=14M, entitiesDrawn=14M, drawsIssued=0` — every entity was visited but no draws were issued because `TryGetRenderData` returned null for everything.
|
||||
|
||||
**Root cause.** WB's `ObjectMeshManager.IncrementRefCount(id)` only bumps a usage counter — it does NOT trigger mesh loading. Loading is fired separately by `PrepareMeshDataAsync(id, isSetup)`, which dispatches to a background worker pool; the result auto-enqueues to `_stagedMeshData` (line 510 of `ObjectMeshManager.cs`) which our existing `WbMeshAdapter.Tick()` already drains.
|
||||
|
||||
The N.4 plan assumed `IncrementRefCount` was lifecycle-aware (it isn't). `LandblockSpawnAdapter` and the original `EntitySpawnAdapter` both called `IncrementRefCount` and stopped — meshes never loaded.
|
||||
|
||||
**Fix** (commit `943652d`):
|
||||
- `WbMeshAdapter.IncrementRefCount` now calls `_meshManager.PrepareMeshDataAsync(id, isSetup: false)` on first registration. `isSetup: false` is correct because acdream's MeshRefs already carry expanded per-part GfxObj ids (0x01XXXXXX) — WB's Setup-expansion path is unused.
|
||||
- `EntitySpawnAdapter` gained an optional `IWbMeshAdapter` constructor parameter. Per-instance entities (server-spawned characters / NPCs) had been entirely skipped by `LandblockSpawnAdapter` (which filters `ServerGuid != 0`); their GfxObjs now get registered + loaded at `OnCreate` and decremented at `OnRemove`. Includes both `MeshRefs.GfxObjId` AND `PartOverrides.GfxObjId` so weapon/clothing/helmet swaps load too.
|
||||
|
||||
**Lesson preserved.** Future cross-session work touching WB: **`IncrementRefCount` is not lifecycle-aware. Call `PrepareMeshDataAsync` to trigger loads.** Documented in CLAUDE.md "WB integration cribs" section.
|
||||
|
||||
### Adjustment 8 (2026-05-08, Task 26 visual verification): SurfaceId lives in batch.Key.SurfaceId
|
||||
|
||||
**Discovered when** the second Task 26 launch showed `drawsIssued=4.8M/5s` (draws ARE happening) but ZERO entities visible. Inspection of `ResolveTexture` showed it was returning early because `batch.SurfaceId == 0` for every batch.
|
||||
|
||||
**Root cause.** WB's `ObjectMeshManager.UploadGfxObjMeshData` (line 1746 of `ObjectMeshManager.cs`) constructs `ObjectRenderBatch` and sets `Key = batch.Key` (a `TextureAtlasManager.TextureKey` struct that contains a `SurfaceId` field) but does NOT populate the top-level `ObjectRenderBatch.SurfaceId` property. That property exists on the type but stays at its default 0.
|
||||
|
||||
**Fix** (commit `943652d`): `WbDrawDispatcher.ResolveTexture` reads `batch.Key.SurfaceId` instead of `batch.SurfaceId`. Also handles the dummy `0xFFFFFFFF` case used by WB's environment edge wireframes.
|
||||
|
||||
**Lesson preserved.** **`ObjectRenderBatch.SurfaceId` is not populated by WB. Read `batch.Key.SurfaceId`.** Documented in CLAUDE.md.
|
||||
|
||||
### Adjustment 9 (2026-05-08, Task 26 visual verification): Modern rendering uses one global VAO/VBO/IBO
|
||||
|
||||
**Discovered when** the third Task 26 launch finally showed real draws — but as "exploded" character body parts scattered around the world with no scenery. Visual was completely broken even though the GL pipeline was clearly issuing draws and binding textures correctly.
|
||||
|
||||
**Root cause.** WB's `ObjectMeshManager` has two rendering paths controlled by `_useModernRendering = HasOpenGL43 && HasBindless`. On any modern GPU (which is everything we target), modern is true and ALL meshes share a single `GlobalMeshBuffer` — one VAO, one VBO, one IBO. Each batch's `IBO` field points to that ONE global IBO; the batch's actual slice is identified by `FirstIndex` (offset into IBO, in indices) and `BaseVertex` (offset into VBO, in vertices). The dispatcher was issuing `glDrawElementsInstanced` with `indices=0` and no base vertex — so every entity drew the same first triangle of the global mesh starting at offset 0. That produced exactly the "exploded parts at scattered positions" symptom.
|
||||
|
||||
**Fix** (commit `7b41efc`): switch to `glDrawElementsInstancedBaseVertexBaseInstance`, pass `(void*)(batch.FirstIndex * sizeof(ushort))` as the indices argument, pass `(int)batch.BaseVertex` as base vertex. The grouped-instanced refactor in the same commit additionally uses `BaseInstance` to slice into the shared instance VBO per group.
|
||||
|
||||
**Bonus discovery:** because all meshes share one VAO under modern rendering, the dispatcher only needs to bind the VAO ONCE per frame (not per draw). Every draw goes to the same VAO. Significant CPU savings.
|
||||
|
||||
**Lesson preserved.** **WB's modern rendering path packs everything into one global VAO/VBO/IBO. Honor `FirstIndex` and `BaseVertex`.** Documented in CLAUDE.md.
|
||||
|
||||
### Adjustment 10 (2026-05-08, Task 26 visual verification): AnimatedEntityState overrides clobber Issue #47 close-detail mesh
|
||||
|
||||
**Discovered when** Task 26's fourth launch showed scenery + connected characters — but characters were "bulky and missing detail" compared to the legacy renderer. Recognized as a re-occurrence of Issue #47 (resolved 2026-05-06 via `GfxObjDegradeResolver`).
|
||||
|
||||
**Root cause.** Adjustment 6 stored AnimPartChanges on `WorldEntity.PartOverrides` using the raw `NewModelId` from the network packet — without applying `GfxObjDegradeResolver`. GameWindow's spawn path correctly resolves base GfxObjs (e.g., upper arm `0x01000055`, 14 verts/17 polys) to their close-detail equivalents (`0x01001795`, 32 verts/60 polys) and bakes the result into `MeshRefs`. But `WbDrawDispatcher` then called `animState.ResolvePartGfxObj(partIdx, meshRefGfxObjId)` which returned the raw (low-detail) override from `PartOverrides`, undoing the degrade.
|
||||
|
||||
**Fix** (commit `7b41efc`): the dispatcher trusts `MeshRefs` as the source of truth and does NOT re-apply `animState.ResolvePartGfxObj` at draw time. `AnimatedEntityState` overrides become relevant only for hot-swap appearance updates (0xF625 `ObjDescEvent`) which today rebuild MeshRefs anyway. `IsPartHidden` similarly skipped — `HiddenPartsMask` is never populated by spawn code (legacy renderer also doesn't check it).
|
||||
|
||||
**Lesson preserved.** **`MeshRefs` is the source of truth at draw time** — GameWindow's spawn path bakes overrides + degrades into it. Don't re-apply overrides downstream.
|
||||
|
||||
### Task 6 (original — kept for history)
|
||||
|
||||
**Files:**
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue