acdream/docs/research/2026-06-23-dense-town-fps-deepdive-handoff.md

Handoff — dense-town FPS deep-dive (push past 75 fps) — 2026-06-23

Branch: claude/thirsty-goldberg-51bb9b (continue in THIS worktree: C:\Users\erikn\source\repos\acdream\.claude\worktrees\thirsty-goldberg-51bb9b). References / named-retail / WorldBuilder live in the MAIN repo C:\Users\erikn\source\repos\acdream\ (read via absolute path; NOT in the worktree).

Read with fresh eyes. This session shipped a real, verified fix (EnvCell shell-draw batching: dense-town Arwic 29 → 75 fps, 2.6×) — KEEP IT. But the user finds 75 fps in a dense town on modern hardware unacceptable ("a modern GPU + OpenGL should chew through this") and wants a deep-dive to find the next root cause and push FPS much higher. This is not "ship it"; this is "keep hunting." The remaining cost is diffuse (no single big lever left) — so the deep-dive needs a better instrument than this session's glFinish hack. Strong recommendation: capture a frame with RenderDoc (the right tool for "where does the GPU time actually go, per draw").

---

0. TL;DR

1. SHIPPED + committed (do NOT revert): EnvCell shell batching. Root cause was EnvCellRenderer.Render called ~94×/frame (once per visible/look-in cell × opaque+transparent) = 24.75 ms = 75 % of the GPU frame at Arwic. Fixed by batching the opaque pass into ONE Render(Opaque, allCells) + skipping the transparent Render for opaque-only cells. cells: 94 calls/24.75 ms → 1 call/ 0.37 ms. Frame 34 ms/29 fps → ~13 ms/75 fps. Visually verified (no missing walls, transparency + look-ins correct).
2. STILL OPEN (the user's ask): 75 fps is unacceptable. The remaining ~12–13 ms is diffuse — particles ~3 ms, aperture punch/seal ~3 ms, ~5.5 ms scattered (sky/entities/clear/weather), plus periodic frame spikes to 30–40 ms. No second 24 ms sink. Deep-dive to find why a modern GPU needs ~12 ms for a dense AC town at 720p, and push FPS much higher.
3. The handoff's original theory (distance-degrade / LOD) is DEAD — refuted by measurement (entity-draw GPU is 0.22 ms; it was never triangle/detail-bound). Do not re-pursue distance-degrade.

---

1. What SHIPPED this session (commits on the branch, newest→oldest)

SHA	What
376f4e6	chore(diag): particle + punch/seal glFinish timers [throwaway]
8067d3b	perf(pview): batch EnvCell look-in shell opaque pass (interior roots)
3af7d00	perf(pview): batch EnvCell shell opaque + skip empty transparent (the Arwic win)
f72f7ce	feat(envcell): CellHasTransparent predicate (part of the fix, NOT apparatus)
e27923b	chore(diag): FPS profiling apparatus [throwaway]
fd9354f	docs: implementation plan
fcf60d8	docs: design spec

(Prior session, KEEP: 536f1c0 _datLock fix.)

The fix (KEEP): RetailPViewRenderer.DrawEnvCellShells + DrawBuildingLookIns no longer call EnvCellRenderer.Render per-cell. Opaque shells batch into one Render(Opaque, allCells) (z-buffer order; lighting is per-instance, CellId-keyed SSBO at EnvCellRenderer.cs RenderModernMDIInternal ~:1324 → safe to batch). Transparent stays per-cell far→near but skips opaque-only cells via EnvCellRenderer.CellHasTransparent. Spec/plan: docs/superpowers/specs|plans/ 2026-06-23-envcell-shell-batching-*.md. Plan Task 4 (final verify) + Task 5 (strip apparatus) are NOT done — apparatus kept for the deep-dive.

---

2. The apparatus (ACDREAM_FPS_PROF=1) — committed, throwaway, STRIP at the end

• src/AcDream.App/Rendering/FrameProfiler.cs — per-frame split, printed [FPS-PROF] every ~1 s: wall (real frame period, p50/p95/min/max), update (OnUpdate CPU), cpuRender (OnRender CPU), present(wait) (= wall − update − cpuRender), gpu (whole-frame TimeElapsed query — the reliable total). Plus [PASS-GPU] — per-renderer glFinish-bracketed GPU via AddRendererGpu(name, ms): cells, terrain, particles, punchseal, each with ms/frame + calls/frame.
• Hooks: GameWindow.OnRender BeginFrame/EndFrame + OnUpdate MarkUpdateStart
  • terrain glFinish (in DrawRetailPViewLandscapeSlice); EnvCellRenderer.Render glFinish try/finally; ParticleRenderer.Draw glFinish; PortalDepthMaskRenderer. DrawDepthFan glFinish. All gated on FrameProfiler.PassGpuEnabled.
• tests/AcDream.Core.Tests/Conformance/DegradeCoverageProbeTests.cs — throwaway dat probe (degrade-coverage; was for the dead distance-degrade theory).
• ⚠ glFinish CAVEAT: the per-renderer glFinish serializes CPU↔GPU and inflates the per-pass absolutes (and folds GPU wait into cpuRender). Use [PASS-GPU] for relative attribution only; trust the whole-frame gpu for the total. For accurate per-draw GPU, use RenderDoc (see §5).
• STRIP everything (plan Task 5) when the FPS work fully lands: FrameProfiler.cs, the GameWindow hooks/fields (_fpsProf/_frameProfiler/_msaaSamples), the terrain glFinish, the EnvCell/Particle/PortalDepthMask glFinish brackets (KEEP CellHasTransparent — it's the fix), DegradeCoverageProbeTests.cs, and the worktree-root fps-*.log / wbdiag.log / degrade-probe.log.

---

3. The investigation — what's RULED OUT (do not re-investigate)

Measured live in Arwic (the dense-town test bed). Frame was 29 fps / 34 ms pre-fix.

Suspect	Verdict + evidence
Distance-degrade / LOD / triangle count (the prior handoff's theory)	❌ DEAD — entity-draw GPU is 0.22 ms; never detail-bound. (Degrade-coverage data: 92–96 % of scenery HAS degrade tables, but trees hide at 200 m+ and degrade only saves triangles, which aren't the cost.)
MSAA / fill / overdraw / shading	❌ Resolution-independent — resizing the window tiny did NOT change FPS; MSAA-off (ACDREAM_MSAA_SAMPLES=0) didn't help. ⚠ the resize test was run on a FAR_RADIUS=4 build that had side-effects — RE-VERIFY resolution-dependence on a CLEAN build in the deep-dive before fully trusting "not fill".
Update thread (physics/anim/net/apply)	❌ update = 0.1 ms.
Far terrain draw distance	❌ ACDREAM_FAR_RADIUS=4 didn't help (and slowed things — streaming side-effects; that build's numbers are suspect).
Terrain draw	❌ 1 slice / 0.3 ms.
Entity leak	❌ modest (esg≈288), entity GPU 0.22 ms regardless.
EnvCell per-cell Render (94×)	✅ WAS the cause — FIXED this session.

---

4. The diffuse remainder — the deep-dive target

Arwic, after the cells fix, whole-frame gpu ≈ 12 ms (profiler on; ~13 ms / 75 fps profiler-off). glFinish-inflated per-pass (upper bounds):

Pass	GPU (inflated)	calls/frame	Notes
cells	0.23 ms	1	✅ fixed
terrain	0.31 ms	1	cheap
particles	~3.1 ms	22 batches	additive blend; overdraw?
punch/seal (DrawDepthFan)	~2.9 ms	31 fans	per-fan full GL-state setup (smells like the cells issue)
UNATTRIBUTED	~5.5 ms	—	sky, entities (0.22), depth clear, weather, MDI-dispatch overhead, + glFinish inflation

Plus frame SPIKES (separate from the steady cost): cpuRender p95 ~30 ms, present(wait) p95 ~18 ms → periodic dips to "30 low" fps (stutter). Likely GC or a periodic streaming/upload hitch. May matter more for perceived smoothness.

Open questions for the deep-dive (ranked):

1. Attribute the ~5.5 ms unattributed GPU. Instrument sky / weather / the depth clear / the entity MDI, OR (better) RenderDoc. Is one of them secretly big?
2. Separate CPU from GPU cleanly. The glFinish folds GPU into cpuRender. Run with the per-renderer glFinish disabled (whole-frame gpu query only) to get the honest cpuRender vs gpu vs present split. If cpuRender is high, suspect the CPU side: PortalVisibilityBuilder flooding ~47 nearby buildings in MergeNearbyBuildingFloods (the per-frame flood/walk for a dense town could be real CPU). That was never isolated.
3. particles (~3 ms, 22 batches): additive overdraw vs draw-count. Reconcile with "resolution-independent" (re-verify the resize test cleanly). If overdraw, reducing particle fill / count helps; if draw-count, batch the 22.
4. punch/seal (31 fans): are we punching/sealing too many apertures for an outdoor root? Each DrawDepthFan does a full GL-state set/restore (like the per-cell Render that we just fixed) — likely batchable (set state once, draw N fans). Check what retail does (decomp: DrawPortalPolyInternal, the seal/punch order).
5. Frame spikes: GC (allocations per frame — the pview path allocates new CellView()/new Vector4[] per aperture in DrawBuildingLookIns, new[] tuples per slice) or streaming. A GC/alloc audit of the per-frame render path is worthwhile.
6. The meta-question: even fully attributed, why ~12 ms GPU for this geometry on modern HW? Look for state thrashing / many small draws / the 2-pass translucency / redundant per-frame uploads. The user's "something is structurally off" thesis still stands for the remaining 12 ms.

---

5. RECOMMENDED next-session method

• RenderDoc frame capture in Arwic is the right tool (CLAUDE.md: rendering perf → RenderDoc). It gives exact per-draw GPU time, draw count, overdraw, and state changes WITHOUT the glFinish distortion. This will attribute the 12 ms far better than the apparatus. The user is on Windows with a modern GPU. Start here.
• Keep the [FPS-PROF]/[PASS-GPU] apparatus as a cheap secondary check, but trust RenderDoc for absolutes. Consider switching the per-pass timers from glFinish to non-nested per-pass TimeElapsed queries (no glFinish → no inflation; sequential passes don't nest) if you want accurate in-engine per-pass numbers without RenderDoc.
• Measurement discipline (this session's lessons): profiler-OFF for the true FPS; constant-view (stand still) for stable per-pass reads; re-verify the resize/ resolution test on a CLEAN build; the user drives the client lifecycle (graceful CloseMainWindow before any rebuild — the DLL is locked while it runs).

---

6. Repro / launch

User: notan / MittSnus81!. Test bed: Arwic surface town (the user can get there; dense buildings → the cost). Build Release; the user runs + watches FPS.

$env:ACDREAM_DAT_DIR="$env:USERPROFILE\Documents\Asheron's Call"; $env:ACDREAM_LIVE="1"
$env:ACDREAM_TEST_HOST="127.0.0.1"; $env:ACDREAM_TEST_PORT="9000"
$env:ACDREAM_TEST_USER="notan"; $env:ACDREAM_TEST_PASS="MittSnus81!"
$env:ACDREAM_FPS_PROF="1"   # [FPS-PROF]+[PASS-GPU]; OMIT for the true FPS the user sees
dotnet run --project src\AcDream.App\AcDream.App.csproj --no-build -c Release 2>&1 | Tee-Object -FilePath "fps.log"

Title bar shows fps | ms (vsync off by default). [FPS-PROF]/[PASS-GPU] flush ~1 s. Logs are UTF-16 (Tee-Object) — read via PowerShell Get-Content | Select-String.

---

7. Do-not-retry / lessons

• The cells fix is correct + verified — do NOT revert. Both phases shipped.
• Distance-degrade is NOT the cause — dead lead, don't re-pursue.
• glFinish per-call inflates per-pass absolutes — relative only; whole-frame gpu query is the reliable total; RenderDoc for true per-draw.
• The FAR_RADIUS=4 build had streaming side-effects — its numbers (incl. the resize test) are suspect; re-verify on a clean build.
• Don't declare FPS "fixed" on a sub-metric — only on the total frame time the user sees + user eyes (the prior session's mistake; honored this session).