Commit graph

37 commits

Author SHA1 Message Date
Erik
543bc79f8a Revert "feat(render): Campaign V slice V4c - move the world draw path onto the RHI"
This reverts commit f353fb53f8.
2026-07-27 22:38:02 +02:00
Erik
f353fb53f8 feat(render): Campaign V slice V4c - move the world draw path onto the RHI
The two renderers that draw everything in the world - WbDrawDispatcher for
entities and EnvCellRenderer for dungeon shells - now record through
IGpuPassEncoder instead of calling GL directly. They share mesh_modern and its
binding layout, which is why they had to move together.

What moved. Every per-frame upload became an IGpuFrame.AllocateRing slice:
instance transforms, batch metadata, clip slots, global lights, per-instance
light sets, indoor flags, opacity, selection lighting, and the indirect command
array. That retires both renderers' DynamicBufferSet pools outright. Those pools
existed so a second Draw within one frame could not overwrite an earlier draw's
still-pending data; the frame ring gives that structurally, because every
allocation within a frame is distinct memory that lives until the frame retires.
DynamicBufferSetCount now reports 0 for both, which is the truth rather than a
silent change - they own no such pool any more.

The imperative Enable/Disable/BlendFunc/DepthMask brackets around the two
multi-draw passes became pipeline variants: five for the dispatcher (opaque,
opaque+alpha-to-coverage, and the three retail blends) and three for the cell
shells. Cull mode and front face stay dynamic per MDI run, exactly where
ApplyCullMode and SetCullMode set them, because core Vulkan 1.3 makes those
dynamic and blend and alpha-to-coverage not. ApplyRetailBlend is gone: its three
cases are now three pipelines, including the inverse-alpha one that
GpuBlendMode.InverseAlpha was added for. uViewProjection, uDrawIDOffset,
uLightingMode, uRenderPass and uLightDebug became fields of the shared
GpuPushConstants block. Issue #52's per-pass batch offset is unchanged - the
draw index still resets per indirect call, and Vulkan's gl_DrawID resets
identically.

Depth compare is baked as GL_LESS, not the contract's LessOrEqual default. The
world frame runs under GL_LESS (RenderFrameGlStateController.RestoreFrameDefaults)
and neither renderer ever called glDepthFunc, so both inherited it; baking
LessOrEqual would have changed which of two coplanar retail surfaces wins.

Two uniform writes were dropped rather than ported, and both are no-ops today:
uFilterByCell and uHighlightColor are declared in neither mesh_modern stage, so
they resolved to location -1. Saying so here rather than letting them vanish.

GPU timing moved to IGpuPassEncoder.BeginTimerScope. The [WB-DIAG] median/p95
window is still fed and still measures opaque + transparent time for the
dispatch, but the sample now comes from IGpuTimerPool.TryResolve - the most
recent retired result - instead of a hand-rolled 3-deep query ring read at N-3.
A sample can therefore repeat when the GPU has not finished a newer query,
where the old code dropped it. The pool also owns the #125 "never read a query
that was never begun" guard now. Diagnostic-only, and flagged rather than left
to be discovered.

Three things deliberately did NOT move, per the campaign doc's section 5.3.
The interim GlBindlessHandleTable stays; both renderers still intern raw
bindless handles and now bind that table through the encoder as an ordinary
IGpuBuffer at binding 9. Retiring it is slice V4t, because the handles are
produced by the texture caches and carried through GroupKey and CachedBatch.
ClipFrame's region buffer (binding 2) and the SceneLighting UBO stay globally
bound by raw GL, because terrain and the viewport/portal renderers read the same
bindings and are raw GL until V4d/V4g. EnvCellRenderer's glMemoryBarrier stays a
raw call: it has no RHI verb, and it guards incoherent shader writes that
acdream does not make, so it was already a no-op against client-side uploads.

RetailAlphaQueue, the GroupKey bucketing, the front-to-back and translucent sort
orders, and every other piece of CPU fidelity logic are untouched. The deferred
alpha payload is still prepared exactly once per sorted alpha scope: a ring
allocation cannot outlive its frame as a ref struct, but its buffer, offset and
size can be stored, so DrawPreparedAlphaBatch binds the same bytes many times
without recopying them.

Two supporting changes outside the two renderers, both flagged.

GlGpuDevice.BeginPass no longer binds framebuffer 0 for a null colour target; it
leaves the binding alone and only binds an explicitly named target. A null target
means "whatever the spine bound", which is what GpuPassDescription's own remarks
describe when they say clears and framebuffer management stay with the spine
until V4h. Forcing 0 would have been fatal here and invisible to this gate:
PrivateEntityViewportRenderer binds its offscreen FBO and then calls
WbDrawDispatcher.Draw, as does PortalTunnelPresentation, so the paperdoll and
creature-appraisal viewports would have rendered to the backbuffer and left their
textures empty - and the offline gate does not cover those viewports. This is the
same class of fix as the ambient-capability save/restore in GlGpuPassEncoder.

GlGpuDevice.CreatePipeline now splices the slice-V2 shared preamble
(Shaders/common.glsl) into every pipeline, reusing Shader.InjectPreamble - widened
from private to internal - so a pipeline-compiled program and a Shader-compiled
one are built from byte-identical sources. mesh_modern requires it: the preamble
declares the binding-9 table and defines ACDREAM_TEXTURE_HANDLE, without which
the world shaders do not compile. Shaders that reference none of it gain an
unused SSBO declaration and two macros; every shader in the tree is #version 430
core, so that is always legal.

Both renderers keep their trailing raw-GL disable block after the pass closes.
The encoder's Dispose restores the capability state that was ambient on ENTRY,
which is not the state these renderers used to leave behind - terrain, sky and
particles are still raw GL and still inherit what the previous renderer left, so
the exit state is reasserted explicitly. It goes at V4h with the last raw-GL
renderer.

A defect caught in review and fixed before the gate: each IGpuPipeline owns its
own vertex array, and vertex attribute pointers plus the index binding are
vertex-array state, so switching blend variants mid-pass silently dropped the
mesh source while the storage bindings survived. Every pipeline switch now goes
through one helper that re-binds the arena.

Gates. Release build green with TreatWarningsAsErrors. App tests 3,844 passed /
3 skipped, stable over four consecutive runs, against a 3,843 baseline plus the
InverseAlpha contract test. Offline pixel gate against 111e7236: 20 differing
pixels of 563,200 compared (fraction 3.55e-05), against a same-commit control
captured immediately afterwards of 17 - indistinguishable from capture noise and
28x under the 0.001 threshold. The gate run's client log has zero exceptions and
an empty stderr.

Coverage gap, stated rather than assumed: the offline gate's scene is a fixed
outdoor view, so it exercises WbDrawDispatcher heavily and EnvCellRenderer not at
all. Dungeon interiors, the paperdoll and appraisal viewports, and portal transit
need a user visual check before this slice is considered proven.

No divergence-register row: this slice changes no retail-facing behaviour.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 20:55:25 +02:00
Erik
d365476ebb feat(render): Campaign V slice V2a - mesh path texture-index migration
Moves the mesh/EnvCell draw path's per-batch texture representation from a
64-bit ARB_bindless_texture handle to a small integer table index, entirely
on the still-shipping GL backend, with zero pixel change. This is the CPU-side
half of the eventual Vulkan descriptor-array indexing model: a table index is
the backend-neutral form (Vulkan indexes a descriptor array with it directly),
while a raw bindless handle is GL-only. Landing the data-model change now, on
GL, under a strict self-differential pixel gate, keeps it separate from V4c's
much larger RHI-plumbing change (see docs/plans/2026-07-27-vulkan-campaign.md
section 5.2 for why the table cannot be device-owned yet).

Mechanism: mesh_modern.vert's BatchData struct carries `textureIndex` (a slot)
instead of `textureHandle` (uvec2); the vertex shader looks the slot up in a
new binding=9 storage buffer (GpuBindingModel.StorageTextureTable) and passes
the reconstructed uvec2 handle to the fragment shader exactly as before, so
mesh_modern.frag needed no change at all beyond the UBO-set macro below. The
16-byte std430 stride is unchanged (GpuBindingModel.GpuBatchDataStrideBytes);
textureLayer/flags keep their offsets, so every existing CPU writer's layout
is untouched.

The handle->slot table (GlBindlessHandleTable, new, pure C#) is owned
separately by WbDrawDispatcher and EnvCellRenderer rather than shared through
a single TextureCache-owned instance: EnvCellRenderer never had a TextureCache
dependency, and nothing requires index agreement between renderers since each
rebinds its own binding=9 buffer immediately before its own draw call. This
avoided threading a new constructor parameter through EnvCellRenderer (and its
six test call sites) for no behavioral benefit. TextureCache and
CompositeTextureArrayCache turned out to need no changes at all: they only
ever produce raw ulong handles, and that production path is unaffected -
the new indirection is entirely a WbDrawDispatcher/EnvCellRenderer-side
concern, added exactly where each already assembles its per-batch GPU struct
(ToInput, the copy-back loop, PrepareDeferredAlphaDraws for the
RetailAlphaQueue path, and EnvCellRenderer's ModernBatchData construction).
The table itself is a single non-ring buffer (unlike the per-frame
triple-buffered SSBOs) because a genuinely new handle is rare - new dat
surfaces/composite overrides, not every frame - so it flushes only when
GlBindlessHandleTable.Dirty is set, mirroring how the existing texture caches
already upload infrequently.

Shader-side, introduced Rendering/Shaders/common.glsl as the shared preamble
GL has no #include for: Shader.cs gained an `includeCommonPreamble` overload
that splices the file's text in after the leading #version/#extension block
(GLSL requires #version first). It declares the binding=9 table plus the
ACDREAM_TEXTURE_HANDLE(idx) lookup macro, and a scaffolding ACDREAM_UBO_SET
macro (a no-op under GL today, redefined to `set = 1,` when the Vulkan
toolchain compiles this same source at V6+, per the campaign doc's set-1 UBO
note) applied to both SceneLighting UBO declarations now so no later slice
needs to touch them again.

Tests: WbDrawDispatcherIndirectBuilderTests updated for the renamed
IndirectGroupInput/BatchDataPublic fields; new ModernBatchDataLayoutTests
(mirrors ClipFrameLayoutTests' role, but for EnvCellRenderer's GPU struct) and
GlBindlessHandleTableTests (pure-CPU allocator behavior, including the
zero-handle case, which is registered like any other handle rather than
special-cased, since that's what reproduces the pre-V2 sampling result
bit-for-bit).

Gate: dotnet build -c Release green, dotnet test
tests/AcDream.App.Tests -c Release green (3843 passed / 3 skipped, +9 over
the 3834/3 baseline), and tools/run-offline-pixel-gate.ps1 passed with a
2.84e-05 differing-pixel fraction against the parent commit - within the
documented ~33x same-commit noise margin. No divergence-register row: this
introduces no retail behavior deviation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 15:51:26 +02:00
Erik
420e5eea70 refactor(app): key live projections by runtime identity
Move materialized live-object sidecars and presentation worksets to exact RuntimeEntityKey ownership. Runtime remains the only GUID/incarnation/local-ID authority while hydration, animation, effects, lights, equipped children, renderer resources, visibility, liveness, and teardown resolve exact projection identities. Preserve synchronous callbacks, local-ID allocation order, and current rendering behavior.
2026-07-25 21:50:58 +02:00
Erik
03b10183e9 perf(streaming): scope reveal warmup spatially
Source composite warmup from the canonical published destination neighborhood, including quiesced static and live projections, instead of rescanning the retained Far-tier world after every membership edge.

This lifecycle correction applies to both draw paths, preserving ef1d263337 as the sole G4 visual rollback.

Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-25 04:39:48 +02:00
Erik
129dd77ddd fix(rendering): decouple portal warmup discovery
Scan published destination candidates under a separate bounded budget from expensive mesh and composite preparation so large retained worlds cannot exhaust the reveal window before candidate discovery completes.

The correction is valid on both the retained and pre-cutover draw paths, so the G4 visual rollback remains the single commit ef1d263337.

Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-25 04:27:17 +02:00
Erik
54d17eb446 perf(rendering): retain packed classification
Retain appearance classification per projection while refreshing transforms, clip slots, lights, selection lighting, and group instance payloads each frame. Incomplete resources and active animation remain on the live classification path, and only an exact full packed comparison acknowledges scene dirtiness.
2026-07-25 02:46:40 +02:00
Erik
f9829d5f9e fix(rendering): normalize empty dispatcher evidence 2026-07-25 02:17:42 +02:00
Erik
06e7754619 fix(rendering): match empty packed submissions
The production dispatcher forces transparent deferral off when no top-level VAO was resolved, even if the frame alpha queue is active. Carry the same per-draw VAO sentinel into the compare-only packed classifier so its empty-submission state is exact rather than sampling only the later queue state.
2026-07-25 01:48:02 +02:00
Erik
29195fb255 feat(rendering): prove packed dispatcher output parity
Build a compare-only dispatcher classifier from RenderFrameView and compare complete opaque, alpha, selection, clip, light, texture, transform, and draw payloads against the accepted path. Preserve retail's stable equal-CYpt ordering with explicit draw-local submission ordinals so material-group history cannot affect alpha ties.
2026-07-25 01:31:56 +02:00
Erik
91e82c3c68 fix(streaming): prepare quiesced destination entities
Separate loaded spatial residency from the world presentation gate so destination live objects acquire mesh owners behind portal space while drawing and simulation stay quiesced. Prevent unowned CPU-cache hits from recreating stale GPU staging work.

Co-authored-by: Erik Nilsson <erikn@users.noreply.github.com>
2026-07-24 20:21:57 +02:00
Erik
e991eeca34 fix(render): make reveal warmup mutation-resumable
Retain composite candidate progress across live membership generations and run a stable follow-up pass after churn, preventing ACE object-stream updates from resetting portal readiness forever.

Co-authored-by: Erik Nilsson <erikn@users.noreply.github.com>
2026-07-24 19:54:40 +02:00
Erik
98f1ac8934 perf(streaming): cursor publication across frame budgets 2026-07-24 19:10:18 +02:00
Erik
230a7df454 perf(content): remove exception-driven setup probes 2026-07-24 15:12:21 +02:00
Erik
7b456b49d6 perf(diag): per-frame history export + checkpoint LOH/cache counters + soak capped mode (2026-07-24 audit review)
An adversarial performance review found our own instruments cannot
measure the project's own performance gates:

- FrameProfiler aggregated CPU/GPU/alloc/stage samples into ~5-second
  windows and reset the ring buffers after each report, so route-wide
  p50/p95/p99 distributions across a whole soak could not be
  reconstructed after the fact. ACDREAM_FRAME_HISTORY=<path> now opts
  into a separate per-frame history (one record per frame, ~72
  bytes/record, accumulated in memory with zero frame-thread I/O) that
  a shutdown-only Dispose() writes as CSV. The aggregated [frame-prof]
  report format and its existing metrics are unchanged.

- The canonical checkpoint JSON tracked cache residency (entry/byte
  counts) but never LOH size/fragmentation, process-wide allocated
  bytes, or cache hit/miss/eviction traffic — a committed audit JSON
  showed 65% LOH fragmentation that no tracked instrument recorded,
  and "does a revisit portal hit or miss the caches" was unanswerable
  from an artifact alone. WorldLifecycleResourceSnapshot now carries
  loh_size_bytes/loh_fragmentation_bytes (GCMemoryInfo.GenerationInfo
  index 3), process_total_allocated_bytes (GC.GetTotalAllocatedBytes),
  and Interlocked hit/miss/eviction counters for the CPU mesh cache,
  decoded-texture cache, and the four bounded DAT-object caches
  (portal/cell/highRes/language, aggregated).

- run-connected-r6-soak.ps1 unconditionally forced
  ACDREAM_UNCAPPED_RENDER=1 with no capped mode, while its sibling
  lifecycle-gate script correctly gated it behind a switch. Added
  -Uncapped (default capped, matching the sibling script's pattern),
  fixed the stationary dwell (12s -> 26s, past the 25s
  LiveEntityLivenessController deadline the adjacent comment already
  cited), and now write an env-disclosure.json into the automation
  artifact directory before every launch listing every ACDREAM_* var
  the script sets plus -Uncapped, since the prior audit could only see
  ACDREAM_DUMP_MOVE_TRUTH and nothing else was ever recorded anywhere.

Cache counters are wired via the existing composition path
(ObjectMeshManager already owns the CPU mesh cache and the mesh
extractor directly; content.Dats is threaded into
WorldLifecycleResourceSnapshotSource the same way every other
composition consumer receives it). The DAT-object cache lives behind
IDatReaderWriter, a third-party interface from the DatReaderWriter
package that cannot be extended; RuntimeDatCollection (the one
production implementation) exposes the aggregate stats directly and a
pattern match reads them, degrading to zero for any test double —
no new static registry was introduced (GpuMemoryTracker remains the
one precedented process-wide static).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 1da2c33c875b41fa383dd79694ee2765f0e21896)
2026-07-24 12:00:30 +02:00
Erik
c2cb83d11f perf(render): gate EnvCellRenderer visibility-snapshot rebuilds on real input changes (2026-07-24 audit review)
PrepareRenderBatches rebuilt the full EnvCell visibility snapshot every
frame any indoor/building root was resolved: a Parallel.ForEach dispatch
over every GpuReady EnvCell landblock (with per-landblock locks), a fresh
outer dictionary plus one fresh inner dictionary per visible cell, a new
snapshot object, and a complete transparency rescan - all while standing
perfectly still. The NeedsPrepare flag existed since Phase A8 as the
intended rebuild gate but was never read by production code.

This wires the gate on the snapshot's actual inputs:
- landblock commits/removals (the existing NeedsPrepare flag),
- the visible-cell filter (content-compared; the caller reuses one
  scratch HashSet across frames),
- the trim window (center/radius),
- mesh render-data availability - new
  ObjectMeshManager.RenderDataAvailabilityVersion, bumped at publish,
  pending-release hide, release completion, and teardown, because the
  snapshot bakes per-cell transparency from TryGetRenderData and a
  late-arriving transparent shell must reclassify its cell,
- the camera: eye position under a 1 mm ABSOLUTE epsilon (swallows the
  documented ~36 um rest jitter, dirties on any real movement; the VP
  translation row scales with AC's ~5e4 world coordinates where a
  relative tolerance would mask sub-meter motion) plus rows 1-3 of
  view*projection (position-independent rotation x projection) under
  relative 1e-5.

Skipping is pool-safe: RenderCore re-anchors _poolIndex to the active
snapshot's PostPreparePoolIndex on every call, so consecutive Renders
without an intervening Prepare reuse scratch lists past the snapshot's
owned region exactly as within-frame passes already do. The empty-filter
branch is now also idempotent instead of allocating a fresh empty
snapshot per frame. Render still receives the current frame's
view-projection every frame (the U.4 stale-matrix rule) - only the
snapshot rebuild is gated.

Pixels must be identical; needs the standard user visual pass (dungeon +
town-near-buildings) before the change is considered accepted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 11:49:40 +02:00
Erik
69a2ca0c6d refactor(world): extract live projection mechanics
Move appearance rebinding, collision construction, default-pose resolution, local shadow ownership, and exact leave-world presentation into focused owners. Preserve retail parent ordering with staged validation, committed recovery, recursive attached-subtree withdrawal, and retryable exact teardown across parent, pickup, position, unwield, and pose-loss edges.

Co-Authored-By: OpenAI Codex <codex@openai.com>
2026-07-21 16:17:03 +02:00
Erik
3718e341be fix #225: stabilize render pacing and frame CPU
Replace scheduler-quantized software sleeps with a reusable Windows high-resolution deadline timer, expose pacing in the frame profiler, and make shutdown wake every persistent mesh worker without losing the shared signal.

Preserve retail alpha order while using a stable radix, skip duplicate deferred-alpha SSBO packing, pack light sets, cache static selection descriptors, and retire historical material groups at the whole-frame boundary. The fixed dense-Caul sample improved from roughly 9-12 ms CPU to 5.3-6.2 ms without reducing visual quality.

Release build succeeds with zero warnings and all 6,300 tests pass with five intentional skips. Three independent retail, architecture, and adversarial reviews are clean; the post-review connected route remains pending because local ACE is offline.

Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-19 06:29:30 +02:00
Erik
749e8ceeb1 fix(rendering): bound portal resource lifetime
Separate logical ownership, render publication, and GPU retirement across live entities, landblocks, particles, textures, mesh arenas, portal/UI teardown, and per-frame scratch storage. Add bounded DAT/texture caches, upload budgets, three-frame fence retirement, exact-incarnation appearance reconciliation, frame pacing, and extensive lifetime conformance coverage.\n\nThe seven-destination connected route now cuts peak working/private memory roughly in half, returns Caul to 125-153 FPS locally, and produces no WER or AMD reset.\n\nCo-authored-by: OpenAI Codex <codex@openai.com>
2026-07-18 21:35:16 +02:00
Erik
3971997689 fix(streaming): retire stale portal-region entities
Port retail's 25-second leave-visibility lifetime over canonical live records, retaining spatially resident and owned entities while using the conservative ACE visibility envelope for nonresident records. Route expiry through the normal generation-safe F747 teardown so animations, effects, physics, and render owners unwind symmetrically.

Replace the append-only modern mesh buffer with coalescing vertex/index ranges and upload each mesh's vertices once instead of once per material. Released zero-reference meshes can now reuse GPU ranges after portal and cache churn.

A connected five-region round trip returned animation ownership to baseline, recreated the starting region on revisit, and held normal FPS. Release build succeeds and all 5,927 tests pass with five intentional skips.

Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-18 10:02:15 +02:00
Erik
6b0472ee32 fix(rendering): port retail shared alpha list
Queue translucent world GfxObj batches and scene particles in one stable far-to-near stream using transformed DAT sort centers, then drain it at retail's landscape and final-world boundaries. Preserve authored blend, cull, lighting, opacity, and adjacent-only batching so particles behind lifestones are composited through the crystal instead of overpainting it.

Release build succeeds and all 5,914 tests pass with five intentional skips.

Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-18 08:14:05 +02:00
Erik
ea4f52ec51 fix(selection): port SmartBox click lighting pulse
Port the retail high/low material-lighting cadence for successful world clicks, keep it instance-scoped in the modern renderer, and restore authored lighting after 0.8 seconds. Correct the selection oracle and pin timing plus per-frame buffer lifecycle with tests.

Co-authored-by: Codex <codex@openai.com>
2026-07-17 23:00:04 +02:00
Erik
e95f55f25b fix(portal): synchronize destination presentation state 2026-07-16 21:17:13 +02:00
Erik
8d63e5c28a test(vfx): harden live missile and effect lifetimes
Add deterministic 96-owner lifecycle and renderer-resource stress gates plus an exact twelve-cycle recall/portal ownership gate. Prove zero retained records, projectiles, spatial buckets, script queues, particle/light owners, shadows, pending effects, and mesh references after churn, GUID reuse, deletion, and session reset.

Make logical teardown incarnation-specific and reentrancy-safe with lifetime epochs, atomic resource registration, generation-aware effect/teleport cleanup, projection mutation tokens, and failure-isolated visibility fan-out. Finish canonical spatial transactions before reporting observer failures and never discard superseded cleanup failures.

Synchronize architecture, roadmap, milestones, retail research, divergence bookkeeping, and durable memory. All three independent review tracks are clean; Release build and the full 5,454-pass/5-skip suite are green.

Co-Authored-By: Codex <noreply@openai.com>
2026-07-14 15:47:00 +02:00
Erik
b0c175afc0 fix(streaming): commit EnvCell landblocks atomically (#214)
Carry one complete cell payload with each streaming completion and publish visibility, physics, and render state on the render thread. Remove the obsolete post-readiness login reload that triggered a duplicate dungeon build.

Co-Authored-By: Codex <noreply@openai.com>
2026-07-13 18:45:24 +02:00
Erik
119a2326be fix(#193): clear InstanceGroup.Opacities each frame — stop the ~1 GB/min LOH leak
Root cause (measured, not guessed): the OOM was a MANAGED Large-Object-Heap leak
(~5 GB in 6 min of roaming; working set 1.6 GB -> 7.6 GB), dominated by ~3.8 GB of
live System.Single[]. A heap-retention analysis of a captured gcdump traced 241 of
242 giant float arrays to a single List<float>: WbDrawDispatcher.InstanceGroup.Opacities.

#188 (2026-07-09, fading doors) added Opacities as a 5th per-instance parallel list
alongside Matrices/Slots/LightSets/IndoorFlags, but left it out of the per-frame
clear loop (WbDrawDispatcher.cs:959). So Opacities.Add(1.0f) fired once per drawn
instance per frame and the list never reset — growing forever; List<float> capacity
doubling produced the observed ~128 MB / ~512 MB power-of-two arrays and OOM after
~50 min. Hit both UI builds because it's core render, and started only recently
because Opacities did not exist before #188.

Fix: extract the per-frame reset into InstanceGroup.ClearPerInstanceData() (promoted
InstanceGroup private->internal) that clears ALL FIVE parallel lists in one place, so
a future 6th list can't silently drift out of the frame lifecycle again. Line 959 now
calls it. TDD: InstanceGroupClearTests asserts every parallel list resets (RED with
the old 4-list clear, GREEN after adding Opacities.Clear()).

Static-audit note: this is exactly why we measured before fixing — the audit ranked
GlobalMeshBuffer (GPU) #1 and the entity-dict leak #3; both were wrong about the
dominant cause. The dotnet-counters (managed vs native) + gcdump retention graph
named the real one-line culprit.

Secondary (NOT this commit): _groups dictionary is never pruned (bounded by distinct
group keys, empty groups are cheap) — filed as a follow-up, not the multi-GB driver.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 14:15:10 +02:00
Erik
ef5049fd58 fix(lighting): #142 — per-instance sun gate for windowed-building interiors
Standing inside (or looking into) a windowed building like the Agent of
Arcanum, interior objects (furniture, NPCs, the player) were lit by the
directional sun because acdream's sun gate was per-FRAME (keyed on the
player being in a sealed cell), not per-DRAW as retail does it.

Retail's PView::DrawCells (0x005a4840) runs two stages per frame:
  outdoor stage   → useSunlightSet(1) (0x005a485a): sun ON
  interior stage  → useSunlightSet(0) (0x005a49f3): sun OFF
DrawMeshInternal (0x0059f398) then calls minimize_object_lighting only
when useSunlight==0, so indoor objects ALWAYS skip the sun regardless of
whether the player's cell is windowed or sealed.

Fix: add a per-instance uint SSBO (binding=6 instanceIndoor[]) whose value
is IndoorObjectReceivesTorches(ParentCellId) — the same predicate AP-43
already uses for the torch gate. In mesh_modern.vert, nest the sun loop
inside an additional `if (instanceIndoor[instanceIndex] == 0u)` check
inside the existing `if (uLightingMode == 0)` block. Indoor objects get
torches (unchanged) but now skip the sun; outdoor objects keep the sun and
still get no torches. The ambient regime (UpdateSunFromSky: 0.2 sealed /
sky otherwise) is untouched — it was already correct.

Mechanically: _currentEntityIndoor set once per entity in
ComputeEntityLightSet; appended to InstanceGroup.IndoorFlags in
AppendCurrentLightSet; grown/packed/uploaded in the same cursor loop as
_clipSlotData and _lightSetData; deleted in Dispose. Mode-1 draws
(EnvCellRenderer) never read binding=6 — the sun loop is inside the
uLightingMode==0 uniform-control-flow branch.

AP-43 divergence register updated: the sun half is now per-draw (no
longer a residual). Residual narrowed to the unaudited ebp_2 test in
CellManager::ChangePosition (no observed impact).

Tests: WbDrawDispatcherIndoorFlagTests pins IndoorObjectReceivesTorches
for the spec §5 representative ids: 0xA9B40172 (Agent of Arcanum EnvCell)
→ 1; 0xA9B40031 (land sub-cell) → 0; 0xA9B4FFFF (landblock) → 0; null
(outdoor shell) → 0; plus the boundary cases 0x0100/0x00FF.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 12:55:56 +02:00
Erik
b7d655bce7 fix(lighting): A7 Fix D round 2 — outdoor objects get NO torches (retail useSunlight gate) (#140)
The Holtburg meeting-hall facade washed out warm/bright vs retail. The round-1
checkpoint blamed torch REACH (acdream Falloff 6×1.3=7.8m vs a supposed retail
Falloff 4). That theory is WRONG, and this commit fixes the real cause.

Empirical (HoltburgTorchFalloffProbeTests, headless dat dump via the production
LightInfoLoader): the orange entrance torch (setup 0x020005D8) is raw dat
Falloff 6 and acdream reads it FAITHFULLY — there is no Falloff-4 torch anywhere
in Holtburg. Both clients read the same dat float, so reach was never inflated.

Decomp (read verbatim + corroborated by an independent adversarial workflow):
retail's per-object torch binder minimize_object_lighting (0x0054d480) is gated
in RenderDeviceD3D::DrawMeshInternal (0x0059f398) by `if (Render::useSunlight == 0)`.
The outdoor landscape stage runs useSunlightSet(1) (PView::DrawCells 0x005a485a,
before LScape::draw), so the building EXTERIOR shell — drawn via
DrawBlock→DrawSortCell→DrawBuilding→CPhysicsPart::Draw→DrawMeshInternal — is lit
by SUN + ambient ONLY; torches are SKIPPED. The static bake
(SetStaticLightingVertexColors 0x0059cfe0) is EnvCell-only. So retail NEVER
torch-lights outdoor objects. This exactly explains the isolation test (object
point lights OFF → building matches retail).

Fix: WbDrawDispatcher.ComputeEntityLightSet gates per-object torch selection on
the object being INDOOR (ParentCellId is an EnvCell, (id&0xFFFF)>=0x0100) via the
pure predicate IndoorObjectReceivesTorches. Outdoor objects (building shells with
null ParentCellId, outdoor scenery, outdoor creatures) keep the all-(-1) light
set ⇒ sun + ambient only = retail. The indoor "no sun" half is already handled by
the global sun-kill when the player is inside a cell (UpdateSunFromSky). No
dungeon regression: EnvCell statics get ParentCellId set (keep torches).

Divergence register: AP-37 (residual: acdream keys sun/torch on the object's own
cell + a per-frame player-inside sun-kill, vs retail's per-draw-stage useSunlight;
only matters for through-doorway look-ins). The round-1 CHECKPOINT got a RESOLVED
banner correcting the reach theory.

Tests: WbDrawDispatcherTorchGateTests (7), HoltburgTorchFalloffProbeTests (dat
dump). App 280/1skip, Core 1486/2skip green. Held at the visual gate — not merged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 23:56:49 +02:00
Erik
1405dd8e90 feat(render): indoor render WORKS — terminating portal flood + every-cell seal + look-in FPS
Checkpoint of the unified retail-faithful indoor render. The two-week HANG/grey is fixed and the
interior seals (live-verified by the user). Commits the session render-rewrite foundation together
with the fixes that made it functional.

- HANG fix: PortalVisibilityBuilder.Build portal flood did not terminate (the faithful ProjectToClip
  near-side clip drifts per round, defeating the CellView dedup; the BFS had no bound after U.2a removed
  MaxReprocessPerCell). Fix = drift-tolerant snapped/canonical CellView.Add dedup (PortalView.cs) plus
  restored MaxReprocessPerCell=16 bounded re-enqueue (PortalVisibilityBuilder.cs). Re-enqueue is kept
  (load-bearing for late-slice propagation, Build_ViewGrowthAfterDoneCell_PropagatesNewSlicesToExit);
  only its count is capped. CellViewDedupTests added.
- Seal (DrawCells Task 2): RetailPViewRenderer.DrawEnvCellShells draws EVERY visible cell via
  IndoorDrawPlan.ShellPass (was gated on the ClipFrameAssembler slot filter, leaving slot-less cells grey).
- Look-in FPS: GameWindow exterior look-in candidates limited to the player landblock +-1 (was all ~81
  loaded LBs iterated every outdoor frame). No behaviour change (far cells were >48m, already culled).

Remaining dominant issue = the FLAP at transitions: viewer-cell metastability (render roots at the
camera-eye cell, which oscillates outdoor-indoor as the 3rd-person boom drifts across the doorway,
confirmed in render-sig). SEPARATE fix, NOT the DrawCells port. Full handoff + flap fix plan + tracked
follow-ups (#78 terrain, look-in-from-inside, look-in FPS, L-spotlight):
docs/research/2026-06-07-indoor-render-session-handoff.md.

Baselines: build 0 err; App.Tests 210/210; Core.Tests 1331 pass / 4 fail (pre-existing) / 1 skip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:14:43 +02:00
Erik
354ca746ad test(render): Phase U.4 — cover ResolveEntitySlot clip-slot resolution
Code review flagged the gate-critical per-instance slot resolution as untested.
Add RED→GREEN cases (live=unclipped slot 0, cell-static→cell slot, non-visible→cull,
outdoor-stab→OutsideView/cull, routing-inactive→all slot 0). Note the full-cell-id-space
invariant at ResolveEntitySlot; fix a stale RenderInsideOut comment in EnvCellRenderer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 18:16:21 +02:00
Erik
65781f5768 fix(render): Phase U.2b — resolve reciprocal portal by other_portal_id (retail 433557)
Code review caught a CRITICAL under-inclusion: ApplyReciprocalClip scanned for the
first OtherCellId match, so a cell with two portals to the same neighbour clipped both
near-side openings against the FIRST reciprocal polygon — hiding geometry through the
second opening (real on Holtburg cellar cells 0x148<->0x149). Plumb the dat's
OtherPortalId back-link through CellPortalInfo + BuildLoadedCell and index the reciprocal
directly (retail arg2->other_portal_id, 433557). Skip (degrade to over-include) when the
index is unresolvable — never clip against a guessed polygon. Adds a disjoint two-back-
portal regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 16:56:00 +02:00
Erik
5dc4140c11 feat(render): Phase A8 — indoor visibility + streaming fixes batch
Lands the working A8 indoor-rendering and streaming fixes accumulated this
session. User has verified these visually to some degree (e.g. lifestone /
translucent meshes confirmed fine under the FrontFace flip; bridge / wall /
collision regressions confirmed fixed after travel); not every path has been
exhaustively gated. The cellar-flap defect remains OPEN and will be solved
the retail-faithful way via a dedicated brainstorm (see handoff docs).

Rendering core (reviewed, high confidence):
- EnvCellRenderer SSBO stride fix: upload packed Matrix4x4[] (64B) instead of
  the 80B CPU InstanceData struct the shader never expected — fixes the
  transform/texture "explosion" for any draw with >1 instance (cells that
  dedupe to a shared cellGeomId). Real root cause.
- WB-style global FrontFace(CW) + per-batch CullMode carried through the MDI
  layout (GroupKey + BuildIndirectArrays + DrawIndirectRange split into
  same-cull runs with absolute uDrawIDOffset per run).
- EntitySet partitioning (IndoorPass / OutdoorScenery / LiveDynamic) +
  WorldEntity.BuildingShellAnchorCellId so building shells scope to their
  dat-derived building cell instead of rendering everywhere.
- RenderOutsideInAcdream (look into buildings from outside) +
  CollectVisiblePortalBuildings frustum cull of portal bounds.
- Sky-when-inside-building + per-cell audit probe + GL-state probe.

Streaming / perf (test-covered; not independently code-reviewed this session):
- Near/far priority queues so near work wins over far; PromoteToNear carries
  full landblock + mesh data; LandblockEntriesWithoutAnimatedIndex avoids
  rebuilding the animated-lookup dict in the hot draw path. Fixes the
  bridge-not-appearing / missing-walls / broken-collision-after-travel
  regressions and improves post-transition FPS.

Tooling + docs:
- tools/A8CellAudit: offline dat cell/portal/building dumper (portals +
  buildings modes) — reproduces the cellar-flap investigation with no launch.
- docs/research cellar-flap root-cause + option-2 handoff (the didInsideStencil
  double-duty finding + the WB-recursive design decision + brainstorm prompt),
  entity-taxonomy, replan, issue-78 visibility investigation.

Diagnostics retained on purpose: ACDREAM_A8_DIAG_* gates, portal_stencil.vert
provisional pos.w clamp, and the probe families are kept (env-var gated, zero
cost when off) because the pending option-2 cellar-flap brainstorm needs them.
Strip in the option-2 ship commit.

Indoor branch stays behind ACDREAM_A8_INDOOR_BRANCH=1 (default off = pre-A8
visual). Build green; App tests + Core (streaming/dispatcher/loader) tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 10:14:50 +02:00
Erik
9559726960 fix(render): Phase A8 — pool aliasing in EnvCellRenderer (visual chaos root cause)
The post-Wave-5 indoor branch chaos (flickering, missing walls, GPU 100%,
~10 FPS) is caused by two interconnected pool-management bugs in
EnvCellRenderer that line-by-line WB comparison surfaced in 30 minutes.
Neither was found by the five post-Wave-5 speculative fixes because none
of them inspected the pool path.

Bug #1 — GetPooledList missing list.Clear():
The reuse branch returned pool lists with prior-frame data still inside.
PrepareRenderBatches' merge phase pattern `gfxDict[k] = list; list.AddRange(...)`
assumes empty lists. Without Clear(), lists grow unbounded each frame, GPU
draws cumulative instance counts, and per-instance transforms become a stew
of past + present data. Mirrors WB ObjectRenderManagerBase.cs:1221-1233.

Bug #2 — Render uses snapshot.BatchedByCell.Count instead of PostPreparePoolIndex:
The snapshot author dropped WB's PostPreparePoolIndex field calling it
"scenery-only," then "compensated" in Render by setting _poolIndex to the
cell count. The cell count has no relation to the pool — Prepare may have
used 50+ pool lists for an 18-cell scene. Render's filter-path GetPooledList
then returns lists that ARE in snapshot.BatchedByCell, corrupting the snapshot
mid-Render. Restoring PostPreparePoolIndex (WB VisibilitySnapshot.cs:31)
correctly places Render's pool cursor past the snapshot's owned region.

Bug #3 (minor) — PopulateRecursive hardcoded isSetup:false for nested parts:
Setup IDs use high-byte 0x02 (per retail). WB ObjectRenderManagerBase.cs:813
checks `(partId >> 24) == 0x02` to detect nested Setups. Our port always
passed isSetup:false, silently dropping any nested Setup (its TryGetRenderData
returns IsSetup=true, Render's `!IsSetup` guard skips the draw). Probably
rare in EnvCells but fixed for completeness.

Regression coverage:
- GetPooledList_ReusedList_IsClearedBeforeReturn — would have failed pre-fix
- GetPooledList_FreshList_IsAlwaysEmpty — sanity check
- Snapshot_PostPreparePoolIndex_IsInitSettable — compile-time guarantee
- Snapshot_PostPreparePoolIndex_DefaultsToZero — defensive default

86/86 App tests pass. Build green. The fix is the audit's primary
deliverable; the GL state probe option-1 apparatus follows in a separate
commit as defense-in-depth for any unidentified residual issue.

Full audit + WB cross-reference in
docs/research/2026-05-28-a8-env-cell-renderer-audit-findings.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 19:08:49 +02:00
Erik
f16b8e9812 feat(render): Phase A8 Wave 2 — EnvCellRenderer (WB EnvCellRenderManager port)
The core port. 1013 LOC of WB-faithful rendering algorithm:

- GetEnvCellGeomId        : WB EnvCellRenderManager.cs:94-103 verbatim
- PrepareRenderBatches    : WB EnvCellRenderManager.cs:247-373 verbatim
                            (parallel frustum-cull, per-cell slow path,
                            ThreadLocal merge, atomic snapshot swap)
- Render(filter:)         : WB EnvCellRenderManager.cs:395-511 verbatim
                            (filter-driven gfxObj group + draw call build)
- RenderModernMDIInternal : WB BaseObjectRenderManager.cs:709-848
                            (single-slot variant; resize buffers,
                            group by cull mode + additive, MDI draw)
- PopulatePartGroups      : WB EnvCellRenderManager.cs:572-580 verbatim
                            (Setup part recursion via PopulateRecursive)
- RegisterCell / FinalizeLandblock / RemoveLandblock — streaming seam
  (no WB analog; bridges acdream's existing StreamingController +
  LandblockStreamer to the renderer's per-cell instance store)

Documented deviations from WB:
- Drop _useModernRendering branch (Phase N.5 mandatory modern path)
- Drop SelectedInstance/HoveredInstance highlights (no editor state)
- _activeSnapshotGlobalGroups/GfxObjIds as sibling fields on the class
  rather than on the snapshot (EnvCellVisibilitySnapshot per Task 4 spec
  only carries BatchedByCell + VisibleLandblocks; global groups only
  used in the unfiltered Render(pass) path which we don't take)
- ConcurrentDictionary<uint, EnvCellLandblock> keyed by full 32-bit
  landblock id (WB uses ushort packed key; acdream uses full id throughout)

10 unit tests (GetEnvCellGeomId determinism + bit-33 dedup flag +
NeedsPrepare + dispose semantics + RemoveLandblock idempotence). Build
green; 23/23 Wave 1+2 tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 14:55:15 +02:00
Erik
fc68d6d01f feat(render): Phase A8 Wave 1 — WB scaffolding extraction + stencil low-level method
Five tasks shipped together (interdependent at build time):

Task 1: WbRenderPass enum — verbatim port of WB RenderPass.cs:1-22
Task 2: WbFrustum + WbBoundingBox + FrustumTestResult — verbatim port
  of WB Frustum.cs (98 LOC) with namespace + BoundingBox-type adaptations.
  +7 unit tests.
Task 3: EnvCellSceneryInstance + EnvCellLandblock — verbatim port of WB
  SceneryInstance.cs:1-161, renamed scope-narrow. Dropped editor-only
  fields (DisqualificationReason, ParticleEmitters, IsQueuedForUpload,
  InstanceBufferOffset, InstanceCount, MdiCommands, IsTransformOnlyUpdate)
  + InstanceId narrowed uint (we don't use ObjectId's editor methods).
  +5 unit tests.
Task 4: EnvCellVisibilitySnapshot — direct port of WB VisibilitySnapshot
  narrowed to BatchedByCell + VisibleLandblocks only.
Task 7: IndoorCellStencilPipeline.RenderBuildingStencilMask — new
  low-level WB-faithful entry mirroring PortalRenderManager:471-484.
  No surrounding GL state setup (caller's responsibility). Probe fields
  LastStencilVertexCount / LastStencilWasFarPunch / LastStencilBuildingId
  for the [stencil] probe emitter in Task 9.

Build green, 18 tests pass (7 new Frustum + 5 new SceneryInstance + 6
existing stencil pipeline). Ready for Wave 2 (EnvCellRenderer port).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 14:46:07 +02:00
Erik
f8d0499d8b feat(render): Phase A8 RR4 — wire BuildingRegistry into landblock load
LoadedCell.BuildingId (init + internal setter) — set exactly once at
    landblock load time by BuildingLoader; null when the cell isn't
    part of any building (outdoor surface cells; dungeon cells not
    enumerated in LandBlockInfo.Buildings).

  GameWindow landblock-load path: builds BuildingRegistry from
    LandBlockInfo.Buildings; stamps each cell's BuildingId; stores the
    registry on _buildingRegistries[landblockId] (GameWindow-level dict)
    for render-frame lookups. Note: LoadedLandblock is AcDream.Core.World
    (a sealed record) — adding an App-type field there would violate
    Code Structure Rule #2, so the registry is stored in a new
    GameWindow-level dictionary instead. Cleanup wired in both
    removeTerrain lambdas (OnLoad + OnResize paths).

  drainedCells dict: the existing _pendingCells drain loop is extended
    to also build a local CellId→LoadedCell dict; BuildingLoader.Build
    uses this dict for the stamping pass so no second iteration is needed.

  New BuildingLoaderTest verifies the stamping path. 5 BuildingLoader
  tests total (4 from RR3 + 1 new).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 11:13:48 +02:00
Erik
f125fdb220 feat(render): Phase A8 RR3 — Building + BuildingRegistry + BuildingLoader
New per-landblock data model for WB-style per-building cell scoping:

  Building            — BuildingId, EnvCellIds, ExitPortalPolygons,
                        occlusion-query state (Step 5 lifecycle)
  BuildingRegistry    — two-way indexed (by cellId + by buildingId);
                        single source of truth per landblock
  BuildingLoader      — static factory from LandBlockInfo.Buildings;
                        walks interior portals to expand cell sets;
                        collects exit portal polygons in world space

10 new unit tests cover data invariants + registry indexing + loader
mapping per the algorithm resolved in RR2 findings.

LoadedCell.BuildingId stamping wired in RR4. Render-time consumption
arrives in RR7 (Steps 1-4) + RR9 (Step 5) + RR11 (RenderOutsideIn).

Design: docs/superpowers/specs/2026-05-26-phase-a8-wb-full-port-design.md
Spike: docs/research/2026-05-26-a8-buildings-data-shape.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 11:08:43 +02:00