Phase A.5's two-tier streaming spec promised that far-tier landblocks
ship terrain ONLY — no entities, no scenery, no interior cells. T13/T16
wired the controller side (RecenterTo emits ToLoadFar/ToLoadNear/ToPromote;
controller passes JobKind to the worker), but the worker's HandleJob
never branched on Kind: every load called BuildLandblockForStreaming
which runs the full hydration + scenery generation + interior cell path.
Result: at default radii (N₁=4 / N₂=12), 540 far-tier LBs each loaded
their full entity layer (~132 entities/LB → ~71K entities total) into
GpuWorldState. The dispatcher then walked all ~54K entities per frame
(post-frustum-cull), driving the entity dispatcher cpu_us from ~3.6ms
median (T24 baseline) to ~18-21ms (post-T22.5 horizon-test). User-
observed: 40 FPS / 25ms frame time at horizon-safe settings; system
crash at full High preset.
Minimum-diff fix: in LandblockStreamer.HandleJob, after
_loadLandblock returns, strip Entities to empty for LoadFar before
posting Loaded. Worker still does wasted hydration CPU (off the render
thread, harmless). Render-side dispatcher walk drops from ~54K to ~10K
entities/frame.
Math: post-fix entity dispatcher should drop to ~3-4ms median at N₁=4 /
N₂=12 (matches T24's 3.6ms at radius=5 single-tier, since N₁=4 has 33%
fewer near entities than N₁=5).
Future optimization (N.6 / A.6): plumb JobKind through
BuildLandblockForStreaming so the worker also skips the wasted CPU.
Out of A.5 scope.
Bug B (T17 WalkEntities allocation) is a smaller perf hit — defer if
post-Bug-A FPS is acceptable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per Phase A.5 spec §4.6 Change #1: when an LB is invisible AND
animatedEntityIds is non-empty, the inner loop walked every entity
in the LB just to find the few animated ones. At ~10.7K entities
(N1=4) that is wasted iteration cost per frame.
Extracted a pure-CPU internal static WalkEntities helper. When LB
is invisible: iterate animatedEntityIds directly and look each up
in a per-LB AnimatedById dictionary (typically <50 animated vs
~10K total). When LB is visible: walk all entities as before.
GpuWorldState.LandblockEntries now yields an AnimatedById map as a
5th tuple field alongside the AABB tuple. Dictionary is built on
each yield (cheap — ~132 entities/LB max). A caching layer is out
of A.5 scope.
WbDrawDispatcher.Draw signature updated to consume the 5-tuple.
GameWindow.cs call site passes _worldState.LandblockEntries which
now yields the 5-tuple — no change needed there.
8 new tests in WbDrawDispatcherBucketingTests cover T17 Change #1
(invisible LB / animated set / neverCull / null frustum) and
T18 Change #2 guard tests (cached AABB / dirty flag / animated bypass).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Code review on T13-T16 bundle (commits fb10c3f/aff35d2/b8d80fe/c4fd373/31d312a)
flagged 3 Important + 2 test-coverage gaps. Apply all 5:
Important #1: GpuWorldState.AddEntitiesToExistingLandblock didn't
canonicalize landblockId. Streaming callers always pass canonical
0xAAAA0xFFFF ids, but the public API silently key-missed for callers
that mirror AppendLiveEntity's cell-resolved-id pattern. Both new
methods now canonicalize the id on entry.
Important #2: RemoveEntitiesFromLandblock asymmetry with RemoveLandblock
re: persistent-entity rescue. Documented as intentional — demote-tier
entities are atlas-tier only (procedural scenery, dat-static stabs/
buildings; never ServerGuid != 0); the local player and live server
spawns live in their LB via RelocateEntity per frame and aren't
affected by atlas-layer demote.
Important #3: StreamingController.NearRadius / FarRadius were { get; set; }
but mutating them after the first Tick is a no-op (StreamingRegion
snapshots the values). Switched to { get; } only with XML doc warning.
Test gap #1: ToDemote routing through Tick — added test that walks
the player past hysteresis and asserts entities drop while terrain
stays.
Test gap #2: Promoted result routing through Tick — added test that
enqueues a Promoted and asserts AddEntitiesToExistingLandblock fires.
Deferred Minor: dead _streamingRadius write + style consistency on
fully-qualified IReadOnlyList — non-load-bearing, can roll into a later
cleanup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the single-radius Tick with a two-tier model that consumes
StreamingRegion's TwoTierDiff (5-list) and routes to the appropriate
JobKind:
- ToLoadFar -> _enqueueLoad(id, LoadFar)
- ToLoadNear -> _enqueueLoad(id, LoadNear)
- ToPromote -> _enqueueLoad(id, PromoteToNear)
- ToDemote -> _state.RemoveEntitiesFromLandblock(id) on render thread
- ToUnload -> _enqueueUnload(id)
Drain switch handles Loaded (terrain + entity layer), Promoted (entity
layer only -- terrain already loaded), Unloaded, Failed, WorkerCrashed.
Constructor signature: nearRadius/farRadius separate ints. Old single-
radius ctor removed; existing single-radius tests updated to pass
nearRadius=farRadius for backward-compat coverage.
GameWindow's enqueueLoad lambda updated from (id =>...) to (id, kind) =>
to match new Action<uint, LandblockStreamJobKind> signature; radius: arg
renamed to nearRadius:/farRadius: (both set to _streamingRadius until T16
wires the full two-tier env-var parsing).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new methods on GpuWorldState, used by two-tier streaming (T13):
- RemoveEntitiesFromLandblock(id): drop all entities from an LB while
keeping the terrain. Used for Near->Far demote (player walks past the
inner ring; LB stays loaded but entities leave).
- AddEntitiesToExistingLandblock(id, entities): merge new entities into
an already-loaded LB record. Used for Far->Near promote (terrain is
already on the GPU; just streaming the entity layer in).
Falls back to the pending bucket if the LB hasn't loaded yet.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Code review on T10-T12 bundle (commits 0cf86bb/00bb030/0405947 + audit
fix 76e1a64) found 3 Important issues:
1. LandblockStreamer.Start() had an idempotency race — the XML doc
claimed thread-safety but the implementation checked _worker != null
before assigning, allowing two callers to both pass the check and
spawn duplicate worker threads. Fixed via Interlocked.CompareExchange.
2. No test verified the worker emits Failed when buildMeshOrNull returns
null. Added Load_WhenBuildMeshReturnsNull_ReportsFailed.
3. StreamingControllerTests.cs:81 used MeshData: default! when
constructing a Loaded result. If a future test flows MeshData
through the apply callback, the null reference would NRE rather
than producing a meaningful assertion failure. Replaced with a real
empty LandblockMeshData instance.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the T7-temporary default! MeshData placeholder. Streamer
now takes Func<uint, LoadedLandblock?, LandblockMeshData?> at
construction; the worker calls it after _loadLandblock succeeds and
passes the pre-built mesh into LandblockStreamResult.Loaded.
GameWindow's buildMeshOrNull factory takes the already-loaded
LoadedLandblock (lb.Heightmap is the LandBlock dat object), so no
additional dat read is needed — _heightTable and _blendCtx are
read-only after init, _surfaceCache is ConcurrentDictionary (T9).
Zero dat lock needed inside the mesh-build closure.
StreamingController._applyTerrain delegate signature widened to
Action<LoadedLandblock, LandblockMeshData> so the pre-built mesh
flows render-thread-side via the Loaded result. ApplyLoadedTerrainLocked
now accepts meshData and calls _terrain.AddLandblock directly, skipping
the per-frame LandblockMesh.Build that previously ran on the render
thread (~5ms per LB at radius=12 first traversal).
StreamingControllerTests updated: all four applyTerrain lambdas
adapted to the two-arg Action signature.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase A.1 reverted to synchronous mode due to DatCollection thread-
safety; T10 documented the lock that makes concurrent reads safe. T11
activates the dedicated worker thread and switches enqueue methods
to non-blocking Channel.Writer.TryWrite.
EnqueueLoad now takes LandblockStreamJobKind (default: LoadNear from
all callers, matching previous full-load semantics). T13/T16 will
route by kind per TwoTierDiff.
Constructor gains optional buildMeshOrNull param (defaults to null-
returning stub); T12 wires the real LandblockMesh.Build factory.
GameWindow construction site updated: Action<uint> enqueueLoad
delegate now wraps a lambda (method group won't bind to Action<uint>
when the method has an optional second param).
LandblockStreamerTests updated: the synchronous-thread-pinning test
replaced by Load_ExecutesLoaderOnWorkerThread which asserts the
loader runs on a different thread; Load_FollowedByDrain now supplies
a stubMesh so the worker can produce Loaded (not Failed) results.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Code review on commits 295bce9/a0741bd/4be392b flagged 1 Important + 3
Minor issues. Apply the actionable two:
Important: 6 sites in GameWindow.cs (lines 3900, 4017-4024, 4138, 4270,
4315) wrote entity.Position = X directly, bypassing T8's SetPosition
mutator and therefore never marking AabbDirty. When T18 lands the
dispatcher's "if AabbDirty refresh" cull gate, these direct writes
would silently leave AABB stale (frustum culls dynamic entities at
their previous positions). Migrated all 6 sites to SetPosition().
Minor: Added a silent case LandblockStreamResult.Promoted arm in
StreamingController.Tick with a TODO(A.5 T13) marker. Today the
streamer never produces Promoted, so the arm is unreachable; the
explicit case prevents a future reader from wondering why the case
is missing.
Deferred Minor: surfaceCache thread-safety XML doc comment + style
consistency on System.Collections.Generic using directive — non-
load-bearing cosmetic.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends the Loaded result record with a LandblockStreamTier discriminator
and a LandblockMeshData payload (default! stub — T13 wires the real
off-thread mesh build). Adds the Promoted variant for Far→Near upgrades
that only need the entity layer, not a mesh rebuild.
LandblockStreamer.HandleJob passes Tier.Near + default! MeshData at the
existing synchronous load site; StreamingControllerTests updated to
match the new positional signature.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Code review on commits 7bcabab/fb6b61e/326b698 flagged 2 Important +
4 Minor issues. Apply all fixes:
Important:
- Two-tier RecenterTo + MarkResidentFromBootstrap now throw
InvalidOperationException on misuse — calling RecenterTo before the
bootstrap silently emitted the entire window as fresh loads (no
demotes/unloads since _tierResidence was empty), a correctness hazard
that produced no exception. Calling MarkResidentFromBootstrap twice
silently dropped accumulated tier state. Both now crash loudly via
a _bootstrapped flag.
- Dropped TierResidence.None from the enum — never assigned, never
checked; absence from the dictionary already encodes "not resident."
Minor:
- Renamed test: RecenterTo_FirstTick_* → ComputeFirstTickDiff_FirstTick_*
(the test calls ComputeFirstTickDiff, not RecenterTo).
- Strengthened RecenterTo_PlayerWalks_NullToFar_* with assertions for
ToPromote.Count==3 (the x=102 column promoting Far→Near) and
ToUnload.Empty (everything within hysteresis).
- Replaced System.Math.Abs with Math.Abs in new code to match the
file's existing `using System;` convention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds TierResidence enum (None/Far/Near), _tierResidence dictionary seeded
by MarkResidentFromBootstrap, and the canonical two-tier RecenterTo overload
returning TwoTierDiff. Pass 1 walks the new far window and emits ToLoadFar /
ToLoadNear / ToPromote; Pass 2 walks prior residents and emits ToDemote /
ToUnload using Chebyshev hysteresis thresholds (NearRadius+2 / FarRadius+2).
EncodeLandblockIdForTest exposes the encoding rule to test assemblies.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the first-tick bootstrap diff: ToLoadNear for the (2*near+1)^2 inner
window, ToLoadFar for the outer annulus up to FarRadius. Uses Chebyshev
distance, matching existing Recenter convention.
Also renames the single-tier RecenterTo → RecenterToSingleTier to free
the canonical name for the upcoming two-tier overload (T5). Updates
StreamingRegionTests and StreamingController to call the renamed method.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add NearRadius/FarRadius properties and a four-arg constructor
(centerX, centerY, nearRadius, farRadius). Radius is set to farRadius
so existing hysteresis math (unload threshold = Radius+2) uses the
outer ring as the bookkeeping boundary. Old three-arg constructor
becomes a thin wrapper: this(cx, cy, radius, radius) — no behaviour
change, 25 pre-existing streaming tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Code review on commit 90a2027 flagged that HandleJob silently ignores
load.Kind. Add a TODO(A.5 T11/T16) comment at the case arm so the
unused field reads as a planned stub, not a bug.
No semantic change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds TwoTierDiff — the five-list output of StreamingRegion.RecenterTo
(ToLoadFar/Near, ToPromote, ToDemote, ToUnload) per spec §4.2. Used by
T3–T6 (StreamingRegion) and T13 (StreamingController).
Extends LandblockStreamJob.Load with a LandblockStreamJobKind parameter
so the streaming worker can route far vs near vs promote jobs differently
(spec §4.3). Patches the one call site in LandblockStreamer.EnqueueLoad
with LoadNear as a placeholder that preserves today's full-load semantics
until T11 activates the worker thread and T16 routes by tier.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Final cross-cutting review of N.5 found that Task 15's deletion of
mesh_instanced.vert/.frag left InstancedMeshRenderer orphaned —
ACDREAM_USE_WB_FOUNDATION=0 silently rendered terrain+sky only with
no entities. The SHIP commit's "[x] ACDREAM_USE_WB_FOUNDATION=0 still
works" claim was inaccurate.
Resolution: formal retirement of the legacy renderer path within N.5
instead of deferring to N.6.
Deleted:
- src/AcDream.App/Rendering/InstancedMeshRenderer.cs
- src/AcDream.App/Rendering/StaticMeshRenderer.cs
- src/AcDream.App/Rendering/Wb/WbFoundationFlag.cs
GameWindow simplified — capability detection is unconditional, missing
bindless throws NotSupportedException with a clear message at startup.
WbDrawDispatcher + mesh_modern shader load are mandatory after init.
No escape hatch.
GpuWorldState simplified — WbFoundationFlag.IsEnabled guards on
AddLandblock/RemoveLandblock removed; adapter calls are unconditional
when the adapter is non-null.
PendingSpawnIntegrationTests updated — WbFoundationFlag.ForTestsOnly_ForceEnable
static ctor removed (flag is gone; adapter calls are unconditional).
The ApplyLoadedTerrain physics-data loop was also simplified: the
EnsureUploaded sub-loop that fed InstancedMeshRenderer is gone;
_pendingCellMeshes is now explicitly cleared to prevent unbounded
accumulation (the worker thread still populates it, but WB handles
EnvCell geometry through its own pipeline).
Spec §2 Decision 5 + §10 Out-of-Scope updated. Plan ship-amendment
section added. Roadmap updated (N.5 ships with retirement; N.6 scope
narrowed to perf-only). CLAUDE.md "WB integration cribs" updated.
Perf baseline doc updated. WbDrawDispatcher class summary docstring
corrected to describe the as-shipped SSBO + multi-draw-indirect path.
ISSUES.md #51 updated (terrain not in N.5 scope; deferred to N.7).
Bindless support is now a hard requirement. Modern desktop GPUs
universally expose GL_ARB_bindless_texture + GL_ARB_shader_draw_parameters;
if a user hits the NotSupportedException, that's a real bug report
worth investigating, not a silent fallback.
Build: 0 errors, 0 warnings. Tests: 71/71 (Wb+MatrixComposition+TextureCacheBindless filter).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Routes server-spawned (CreateObject) entities through the per-instance
rendering path. Filter: ServerGuid != 0. Atlas-tier entities (procedural,
ServerGuid == 0) flow through LandblockSpawnAdapter (Task 11) instead.
For entities with PaletteOverride set, walks each MeshRef.SurfaceOverrides
map and calls TextureCache.GetOrUploadWithPaletteOverride to pre-warm the
palette-composed GL texture before the first draw. Surfaces not in the
SurfaceOverrides map (i.e. whose ids are only known after opening the GfxObj
dat) are decoded lazily by the draw dispatcher on first use, consistent with
StaticMeshRenderer.
Builds AnimatedEntityState per server-guid via injected sequencer factory
(Func<WorldEntity, AnimationSequencer>). The factory decouples the adapter
from DatCollection so tests pass a stub lambda without a GL context.
OnRemove releases per-entity state. Unknown guids no-op.
Introduces ITextureCachePerInstance: thin seam interface over the palette
decode path so EntitySpawnAdapter tests can use a CapturingTextureCache
mock without constructing a GL context. TextureCache implements it.
Adjustment 4 documented in source comments: WorldEntity does not currently
expose HiddenPartsMask or AnimPartChanges (they are consumed upstream in the
network layer before the WorldEntity is built). HideParts / SetPartOverride
calls are placeholder TODO'd for when those fields are promoted.
Wired into GpuWorldState.AppendLiveEntity (OnCreate) and
RemoveEntityByServerGuid (OnRemove). Constructed in GameWindow under the
ACDREAM_USE_WB_FOUNDATION flag alongside LandblockSpawnAdapter. Sequencer
factory captures _dats + _animLoader at construction time; falls back to an
empty Setup + MotionTable via NullAnimLoader when dats are unavailable.
10 new tests: server-spawn routing, atlas-tier skip, palette decode pre-warm
(with and without surface overrides), OnRemove lifecycle, unknown-guid noop,
multi-entity isolation. All pass; 8 pre-existing failures unchanged.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GpuWorldState's constructor accepts an optional LandblockSpawnAdapter.
AddLandblock calls OnLandblockLoaded with the post-merge loaded record;
RemoveLandblock calls OnLandblockUnloaded with the landblock id at the
top of the method (before state mutation).
Both calls are gated behind WbFoundationFlag.IsEnabled — no behavioral
change with flag off (existing tests pass without modification).
GameWindow constructs the adapter under the flag and threads it into
GpuWorldState. With flag on, atlas-tier scenery now drives WB ref
counts; per-instance entities (ServerGuid != 0) are filtered out by
the adapter and don't reach WB.
Foundation for Task 13 (memory budget verification under stress).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Register dat-defined LightInfos as runtime LightSources when entities
stream in. Every Setup (0x02xxxxxx) with a non-empty Lights dictionary
gets its per-part lights pulled via LightInfoLoader, which converts
the local Frame + ColorARGB + Intensity + Falloff + ConeAngle fields
into world-space LightSource records owned by the entity id.
Wire the LightingHookSink into the animation-hook router so retail's
SetLightHook animations (ignite-torch, extinguish-lamp) flip the
matching LightSource.IsLit latches. One hook may own multiple lights
(lamp-posts with two LightInfo entries) — the sink maintains an
owner-indexed map so all get toggled together.
Unregister on landblock unload: the streaming controller's
removeTerrain callback grabs the loaded landblock's entity list (new
GpuWorldState.TryGetLandblock helper) and drops every owner from the
sink before the entities disappear — otherwise walking across
landblocks accumulates stale LightSources.
9 new tests (LightingHookSink routing + LightInfoLoader conversion).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three separate fixes landed today, each addressing a specific bug the
user observed during live play:
1. NPC clothing changes by camera angle (InstancedMeshRenderer)
- Group key was (GfxObjId) only, so every humanoid NPC using the
same body mesh piled into one instance group; only the first
instance's texture was used for the entire DrawInstanced batch,
so which NPC's palette "won" changed as frustum culling and
iteration order shuffled entries.
- Now keyed by (GfxObjId, PaletteHash ^ SurfaceOverridesHash)
so only compatible instances batch; each unique appearance gets
its own draw call. Perf hit is small (humanoid NPCs each emit
one more draw call); visually every NPC is now stable.
2. GpuWorldState dedup on respawn
- Server re-sends CreateObject for the same guid on visibility
refresh / landblock crossing / appearance update. AppendLiveEntity
was blindly appending each time, so GpuWorldState accumulated
multiple copies of the same entity, each with its own
PaletteOverride / MeshRefs. That alone wasn't the clothing bug
(that was #1) but it would have caused other overlap problems
downstream.
- Added RemoveEntityByServerGuid + WorldGameState.RemoveById;
OnLiveEntitySpawnedLocked calls both before creating the new
entity so respawns replace cleanly.
3. Motion wire format — run animation sync with retail observers
- ACE's MovementData constructor only computes interpState.ForwardSpeed
on the WalkForward/WalkBackwards branch; every other ForwardCommand
falls into `else` and passes through WITHOUT speed set, giving
observers speed=0. Sending RunForward directly meant retail
clients saw us "run in place" while position drifted forward.
- Wire: always WalkForward + HoldKey.Run for running. ACE
auto-upgrades to RunForward with creature.GetRunRate() for
broadcast — correct command + correct speed at observers.
- Added per-axis FORWARD_HOLD_KEY / SIDE_STEP_HOLD_KEY /
TURN_HOLD_KEY so every active axis carries HoldKey.Run when
running (matches holtburger's build_motion_state_raw_motion_state).
- Added LocalAnimationCommand to MovementResult so our own
client still plays the RunForward cycle locally while the wire
stays WalkForward. Wire vs. local animation command are now
decoupled.
- Walk-backward wire command changed from WalkForward@-0.65 to
WalkBackward@1.0 (holtburger pattern).
- Strafe speed changed from 0.5 to 1.0 on wire AND local physics
(matches retail sidestep pace).
4. Jump height default + env-var tuning
- Default jumpSkill bumped from 100 → 200 (jump ≈ 3m at full
charge, closer to retail feel for a mid-level character).
- ACDREAM_RUN_SKILL and ACDREAM_JUMP_SKILL env vars now override
the defaults so the user can tune per-character until we parse
PlayerDescription and plumb real skill values through.
5. JustLanded signal on MovementResult
- Tracks airborne→grounded transition so future animation code
can fire the landing cycle when we land. Just a bool flag for
now — no consumer yet (the proper action-queue path will use it).
Not in this commit: jump animation itself. An earlier attempt to
SetCycle(Jump=0x2500003b) fed an Action-type motion into the SubState
cycle resolver, which produced a "torso" mis-render. Reverted. The
proper fix is porting the retail motion action-queue semantics into
AnimationSequencer — see docs/research/deepdives/r03-motion-animation.md
for the spec. That's the next session's work.
470 tests pass, build clean.
TWO root causes for "character disappears when walking far":
1. MarkPersistent stored SERVER GUID but RemoveLandblock checked LOCAL
entity.Id — different namespaces, never matched. Fixed by adding
WorldEntity.ServerGuid field and checking it in RemoveLandblock.
2. Even with rescue working, the player entity stays in its SPAWN
landblock's entity list forever. When the player walks to a new
landblock and the spawn landblock gets frustum-culled, the entity
disappears because neverCullLandblockId is computed from the
player's current position (new landblock) but the entity is stored
in the old landblock.
Fixed by calling GpuWorldState.RelocateEntity every frame in the
player-mode update loop. This moves the entity from whatever
landblock it's currently in to the one matching its actual position.
The scan is O(entities) but only runs for one entity per frame.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ROOT CAUSE: MarkPersistent(chosen.Id) stored the SERVER GUID (e.g.
0x5000000A) but RemoveLandblock checked entity.Id which is the LOCAL
sequential counter (e.g. 42). Different number spaces → never matched
→ persistent rescue never triggered → player entity lost on unload.
Fix:
- Added WorldEntity.ServerGuid field (0 for dat-hydrated scenery)
- Live entity spawn sets ServerGuid = spawn.Guid
- RemoveLandblock checks entity.ServerGuid against _persistentGuids
- MarkPersistent still stores the server GUID (correct)
This bug has been reported across multiple sessions as "character
disappears when walking far." The neverCullLandblockId fix only
prevented frustum culling but didn't prevent the entity from being
removed from the render list entirely.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Four targeted fixes for user-reported movement/visual bugs:
1. Player entity disappearing: GpuWorldState now supports persistent
entities (MarkPersistent/DrainRescued). The player character survives
landblock unloads and gets re-injected into the streaming window at
the current center landblock.
2. Feet sinking into terrain: +0.15 Z bias in PlayerMovementController
keeps the character model above terrain z-fighting edge cases.
3. Camera after portal teleport: ChaseCamera.Update now called
immediately after teleport snap so the camera recenters on the new
position instead of lingering at the pre-teleport location.
4. Scenery on roads: SceneryGenerator now checks road status at the
final displaced position (not just the origin vertex), catching
objects that drift from non-road vertices onto road cells.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Populates the collision engine with TerrainSurface + CellSurface
entries when landblocks stream in, removes them when they stream
out. CellSurface vertices are transformed from cell-local to world
space using EnvCell.Position orientation + origin.
Phase B.2 (player movement mode) will call PhysicsEngine.Resolve()
to get collision-validated positions before sending them to the
server.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Per-landblock AABB culling against the view frustum. Each loaded
landblock has a 192×192 XY footprint + a Z range derived from the
terrain vertex min/max (padded +50 above / -10 below for entities
on top and basements). One AABB test per landblock per frame;
landblocks fully outside the frustum skip ALL their terrain draws
and entity draws (both opaque and translucent passes).
GpuWorldState gains SetLandblockAabb + LandblockEntries (per-landblock
iteration with AABB data). TerrainRenderer.Draw and
StaticMeshRenderer.Draw both accept an optional FrustumPlanes and
skip culled landblocks. GameWindow.OnRender extracts FrustumPlanes
from camera.View * camera.Projection and passes to both.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The previous fix in f792931 set MaxCompletionsPerFrame to int.MaxValue
on the theory that synchronous loading made the cap pointless. That
ignored the GPU upload cost: applying 25 landblocks in one Tick
allocates ~25 terrain VBOs + hundreds of entity GfxObj sub-mesh VBOs
+ all unique texture uploads in a single frame, which observably
crashes with OutOfMemoryException on the first frame after login.
The pending-spawn list (also added in f792931) is what actually
fixes the spawn-drop bug — it makes the cap safe by parking
late-arriving spawns until their landblock loads. With both fixes:
- Cap=4 spreads the 25-landblock first-frame load over ~7 frames
(~116ms at 60fps, below human perception)
- Spawns for the 21 not-yet-loaded landblocks land in pending and
back-fill as each one arrives over the next 6 frames
- No data lost, no OOM
219 tests still green.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fifth and final Phase A.1 hotfix. Replaces the previous "drop on
miss" semantics in GpuWorldState.AppendLiveEntity with a per-landblock
pending bucket that survives the race where a CreateObject arrives
before its landblock has been streamed in.
Root cause:
The post-login spawn flood (40+ NPCs/items) drains in a single
WorldSession.Tick() call. The synchronous streamer enqueues all 25
visible-window landblocks in one shot but StreamingController.Tick
was capped at MaxCompletionsPerFrame=4, so only 4 landblocks landed
in GpuWorldState on the first frame. The center landblock 0xA9B4FFFF
may or may not have been in those first 4 (HashSet iteration order
is undefined). Spawns whose target landblock wasn't yet loaded were
silently dropped by AppendLiveEntity. Re-ordering the OnUpdate
(streaming first, live second) didn't fix it because the cap still
limited to 4 per frame; spawns for landblocks #5+ kept dropping
until the queue drained, by which point the spawn flood was over.
The reordering was correct but insufficient. The cap was a relic of
the original async streamer design (limit GPU upload spikes per
frame). With the synchronous streamer there's no backlog to spread,
so the cap was pure latency for no benefit. Setting it to int.MaxValue
restores "drain everything you just enqueued" semantics.
The pending-spawn list is the *correct* architecture fix that makes
the system robust against any future ordering bug, not just the cap:
- AppendLiveEntity for an unloaded landblock parks the entity in a
per-landblock pending bucket instead of dropping it.
- AddLandblock drains pending entries for its landblock and merges
them into the loaded record before storing.
- RemoveLandblock drops pending entries for the same landblock —
if the player moved away, the spawns are no longer relevant; the
server resends them via CreateObject when the player returns.
Diagnostic counter PendingLiveEntityCount exposes the bucket size
so future regressions are visible without spelunking.
7 new GpuWorldStateTests pin the contract:
- AppendLiveEntity_LandblockAlreadyLoaded_AppendsImmediately
- AppendLiveEntity_LandblockNotLoaded_ParksInPending
- AddLandblock_DrainsPendingEntriesForThatLandblock
- AddLandblock_DoesNotDrainPendingForADifferentLandblock
- RemoveLandblock_DropsPendingForThatLandblock
- RemoveLandblock_LoadedThenRemoved_DropsItsEntities
- IsLoaded_ReturnsTrueForLoaded_FalseForPendingOnly
Also removes the diagnostic Console.WriteLine I added in the previous
debugging round and the old LiveAppendsResolved/Dropped counters that
were never read by anyone.
219 tests green (212 + 7 new).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After the 0xFFFF terminator fix in f83a8c1, terrain renders correctly
but live NPCs and weenies disappeared. Root cause: the server's
ServerPosition.LandblockId is in cell-resolved form (0xAAAA00CC where
the low 16 bits are the cell index within the landblock), but the
streaming system stores landblocks in GpuWorldState keyed by their
canonical 0xAAAAFFFF form. AppendLiveEntity was passing the raw
server id straight into the dict TryGetValue, missing every time, and
silently dropping the spawn.
Fix: canonicalize at the GpuWorldState boundary by masking with
(0xFFFF0000u | 0xFFFFu). The XML doc on the method explains the two
forms so future callers don't have to guess. Calling code stays
unchanged.
212 tests green.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ROOT CAUSE of the "giant ball with spikes" terrain corruption that
the previous two hotfix attempts (lock + synchronous loading) failed
to address. Threading was a red herring all along.
AC dat conventions:
0xAAAA0xFFFF — LandBlock dat (terrain heightmap)
0xAAAA0xFFFE — LandBlockInfo dat (static-object metadata)
WorldView.NeighborLandblockIds correctly uses 0xFFFF. My
StreamingRegion.EncodeLandblockId from Phase A.1 Task 1 used 0xFFFE
by mistake. Every streaming load was therefore calling
LandblockLoader.Load with the LandBlockInfo id, which makes
DatCollection ask DatBinReader to read a LandBlock from the
LandBlockInfo file. The reader's internal buffer position lands in
the middle of the wrong file's bytes, ReadBytesInternal asks for an
out-of-range slice, throws ArgumentOutOfRangeException, and the
landblocks that DON'T throw return half-populated LandBlock objects
whose Height[] arrays contain garbage. Garbage Z values render as
the spike pattern.
The kicker: my Task-1 review fix added a test
(Constructor_SmallRadius_IDsMatchEncodingRule) that asserted
Assert.Contains(0x1234FFFEu, region.Visible). The test was passing
because it pinned the wrong value. I literally codified the bug.
Fix: change EncodeLandblockId's terminator from 0xFFFEu to 0xFFFFu
and update the test to assert 0x1234FFFFu. The XML doc on Visible
now explicitly explains the 0xFFFF/0xFFFE distinction so this can't
recur.
The previous two hotfixes (_datLock in c991fb2, synchronous streamer
in 531c9f9) stay in place — _datLock is defensive belt-and-suspenders
that documents which entry points read dats, and synchronous loading
is correct-by-default until we decide whether to reintroduce
background loading (Phase A.3 may make it unnecessary anyway).
212 tests green. With this fix the streaming should actually work.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Second hotfix attempt for the "ball of spikes" terrain corruption.
The previous _datLock fix was insufficient because dat reads happen
from many render-thread code paths I didn't enumerate (animation
tick, OnLiveMotionUpdated, OnLivePositionUpdated, the live spawn
hydration, ApplyLoadedTerrain) and locking each is invasive and
fragile.
DatReaderWriter's DatCollection is fundamentally not thread-safe:
DatBinReader's internal buffer position is shared per-database, so
two concurrent .Get<T> calls corrupt each other's read state. The
ArgumentOutOfRangeException at DatBinReader.ReadBytesInternal in
the failure log is the smoking gun — one read started reading a
LandBlock, another moved the reader's position, the first one
asked for the wrong number of bytes.
Until Phase A.3 introduces a thread-safe dat wrapper (or until we
preload all dats into pure in-memory dictionaries), the streamer
runs synchronously: EnqueueLoad invokes the load delegate inline
on the calling thread and writes the result to the outbox in a
single call. The render-thread DrainCompletions loop picks it up
on the same frame.
API surface unchanged — Channel-based outbox, EnqueueLoad/Unload,
DrainCompletions, Start (now no-op), Dispose all preserved. Move
back to async loading is a single-class change once dat thread
safety lands.
Cost: visible frame hitch when crossing landblock boundaries
(rendering the new landblock is now on the render thread). For
default 5×5 the hitch is one landblock per cardinal step, ~50ms
worst case. Acceptable for the MVP — correctness over hitches.
Updated the off-thread test to assert the new synchronous contract
(loader runs on the calling thread). The other 4 tests still pass
unchanged because their spin-drain pattern works with synchronous
delivery too.
The previous _datLock from commit c991fb2 stays in place as
defensive belt-and-suspenders — it's free in synchronous mode and
keeps the contract documented at every dat-reading entry point.
212 tests green.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Called once per frame from OnUpdate. Owns a StreamingRegion and uses
delegates into LandblockStreamer + a terrain-apply callback so unit
tests can inject fakes. Handles first-tick bootstrap (whole window
loads), boundary recenter (diff against previous center), and
drain completions (up to N per frame to cap GPU upload spikes).
4 new tests, all green.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces GameWindow._entities flat list with a per-landblock dict
keyed by landblock id. The flat entity view is rebuilt on add/remove
so the renderer keeps its simple "iterate Entities" loop. Also
provides AppendLiveEntity for the CreateObject path that spawns
entities into an already-loaded landblock after hydration.
Not thread-safe — all mutation is render-thread only.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Code review follow-up to commit 0904372. Five Important fixes plus
three Minor polish items found by the reviewer before StreamingController
depends on this class under churn.
I1: Dispose is now thread-safe via Interlocked.Exchange on an int
guard. Two concurrent Dispose calls no longer double-dispose the
CancellationTokenSource.
I2: EnqueueLoad/EnqueueUnload now throw ObjectDisposedException when
called after Dispose instead of silently dropping the job. Jobs
vanishing into a completed channel was a debugging hazard.
I3: Start throws ObjectDisposedException when called after Dispose
instead of silently doing nothing (the old guard only checked
whether the thread was non-null, not whether the streamer was
still usable).
I4: New test Load_ExecutesLoaderOnBackgroundThread captures the
loader delegate's ManagedThreadId and asserts it differs from
the test thread's id, proving the whole reason this class
exists (off-thread execution) is actually happening.
I5: New LandblockStreamResult.WorkerCrashed record type for the
outer catch in WorkerLoop. Previously the crash path wrote
Failed(0, ex.ToString()) which collided with landblock (0, 0)
in the north ocean, making "worker crashed" indistinguishable
from "landblock 0 failed to load".
Minor polish:
- M1: Test spin constants (SpinTimeoutMs, SpinStepMs,
SpinMaxIterations) extracted so the 200 x 10ms pattern has one
source of truth.
- M2: DefaultDrainBatchSize public const on LandblockStreamer so
the batch cap has a name and a comment explaining why 4.
- M3: Safety-argument comment on the sync-over-async
WaitToReadAsync call explaining why it cannot deadlock (dedicated
thread, no SyncContext).
- M6: XML remarks on the class and on DrainCompletions documenting
threading contract (Enqueue = any thread, Drain = single consumer
thread).
112 Core + 96 Core.Net tests green.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Background thread pulls load/unload jobs from an inbox channel, invokes
a caller-supplied Func<uint, LoadedLandblock?> (production wraps
LandblockLoader.Load, tests inject a fake), and posts results to an
outbox channel the render thread drains. Graceful shutdown via
CancellationToken; failed loads reported rather than retried.
4 new tests, all green.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
LandblockStreamJob (Load/Unload) and LandblockStreamResult
(Loaded/Failed/Unloaded) are the channel payload types the next
task's LandblockStreamer will use. Separate file because they're
shared between the worker thread and the render thread and deserve
a focused home.
Folds in two carryover nits from the Task 1 fix review:
- Stale "radius + 1" comments in StreamingRegionTests updated to
match the real radius+2 threshold (no numeric-assertion changes).
- Single-step recenter test now asserts Visible.Count == 25 and
Resident.Count == 30, locking in the Visible/Resident semantic
split behaviorally.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Review follow-up from commit 11df793. Three fixes:
1. Visible semantics: StreamingRegion.Visible now strictly describes the
current (2r+1)×(2r+1) window, not window + hysteresis retainees.
Added a parallel Resident property exposing the actual loaded set
(window + hysteresis buffer). This matters because StreamingController
(next task) reads these to decide what to render vs what to unload;
conflating them in one set would have forced awkward post-processing
downstream.
2. Doc/code disagreement: updated the RecenterTo and RegionDiff doc
comments from "radius + 1" to "radius + 2" to match the actual
implementation (which is what the tests require). Also updated the
plan doc so future readers don't hit the same contradiction.
3. Edge-clamping test coverage: added a single-axis edge test
(cx=0, cy=50 → 15 entries) and an ID-encoding test (radius=0 at
0x12,0x34 → 0x1234FFFE) so a swapped-shift bug in EncodeLandblockId
or an asymmetric off-by-one would fail a test instead of passing
silently.
9 tests green, full suite regressions-free.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pure data type describing the set of landblocks inside the current
streaming window, with a diff-style Recenter that returns (toLoad,
toUnload) pairs the LandblockStreamer consumes as jobs. Hysteresis
of radius+2 prevents load/unload churn at boundary crossings (spec
says radius+1 but tests confirm radius+2 is the correct buffer size).
First piece of Phase A.1 per docs/superpowers/plans/2026-04-11-foundation-a1-streaming.md.
7 new tests, all green. Total suite: 105/105.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
