Four small wins on top of the grouped-instanced refactor.
1. Drop unused animState lookup. Was a side-effect-free
_entitySpawnAdapter.GetState call per per-instance entity, made
redundant by the Issue #47 fix that trusts MeshRefs.
2. Front-to-back sort opaque groups. Squared distance from camera to
each group's first-instance translation; ascending sort. Lets the
GPU's depth test reject fragments behind closer geometry — real
win on dense scenes (Holtburg courtyard, Foundry interior).
3. Per-entity AABB frustum cull. 5m-radius AABB check per entity
before walking parts. Skips work for distant entities even when
their landblock is partially visible. Animated entities (other
characters, NPCs, monsters) bypass — they always need per-frame
work for animation regardless. Conservative radius covers typical
entity bounds; large outliers stay landblock-culled.
4. Memoize palette hash per entity. TextureCache.HashPaletteOverride
is now internal; new GetOrUploadWithPaletteOverride overload takes
a precomputed hash. The dispatcher computes it ONCE per entity and
reuses across every (part, batch) lookup, avoiding the per-batch
FNV-1a fold over SubPalettes. Trees / scenery without palette
overrides skip entirely (palHash stays 0).
Visual output unchanged; FPS up further, especially in dense scenes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Routes server-spawned (CreateObject) entities through the per-instance
rendering path. Filter: ServerGuid != 0. Atlas-tier entities (procedural,
ServerGuid == 0) flow through LandblockSpawnAdapter (Task 11) instead.
For entities with PaletteOverride set, walks each MeshRef.SurfaceOverrides
map and calls TextureCache.GetOrUploadWithPaletteOverride to pre-warm the
palette-composed GL texture before the first draw. Surfaces not in the
SurfaceOverrides map (i.e. whose ids are only known after opening the GfxObj
dat) are decoded lazily by the draw dispatcher on first use, consistent with
StaticMeshRenderer.
Builds AnimatedEntityState per server-guid via injected sequencer factory
(Func<WorldEntity, AnimationSequencer>). The factory decouples the adapter
from DatCollection so tests pass a stub lambda without a GL context.
OnRemove releases per-entity state. Unknown guids no-op.
Introduces ITextureCachePerInstance: thin seam interface over the palette
decode path so EntitySpawnAdapter tests can use a CapturingTextureCache
mock without constructing a GL context. TextureCache implements it.
Adjustment 4 documented in source comments: WorldEntity does not currently
expose HiddenPartsMask or AnimPartChanges (they are consumed upstream in the
network layer before the WorldEntity is built). HideParts / SetPartOverride
calls are placeholder TODO'd for when those fields are promoted.
Wired into GpuWorldState.AppendLiveEntity (OnCreate) and
RemoveEntityByServerGuid (OnRemove). Constructed in GameWindow under the
ACDREAM_USE_WB_FOUNDATION flag alongside LandblockSpawnAdapter. Sequencer
factory captures _dats + _animLoader at construction time; falls back to an
empty Setup + MotionTable via NullAnimLoader when dats are unavailable.
10 new tests: server-spawn routing, atlas-tier skip, palette decode pre-warm
(with and without surface overrides), OnRemove lifecycle, unknown-guid noop,
multi-entity isolation. All pass; 8 pre-existing failures unchanged.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ports retail's ParticleEmitterInfo / Particle::Init / Particle::Update
(0x005170d0..0x0051d400) and PhysicsScript runtime to a C# data-layer
plus a Silk.NET billboard renderer. Sky-PES path is debug-only behind
ACDREAM_ENABLE_SKY_PES because named-retail decomp confirms GameSky
copies SkyObject.pes_id but never reads it (CreateDeletePhysicsObjects
0x005073c0, MakeObject 0x00506ee0, UseTime 0x005075b0).
Post-review fixes folded into this commit:
H1: AttachLocal (is_parent_local=1) follows live parent each frame.
ParticleSystem.UpdateEmitterAnchor + ParticleHookSink.UpdateEntityAnchor
let the owning subsystem refresh AnchorPos every tick — matches
ParticleEmitter::UpdateParticles 0x0051d2d4 which re-reads the live
parent frame when is_parent_local != 0. Drops the renderer-side
cameraOffset hack that only worked when the parent was the camera.
H3: Strip the long stale comment in GfxObjMesh.cs that contradicted the
retail-faithful (1 - translucency) opacity formula. The code was
right; the comment was a leftover from an earlier hypothesis and
would have invited a wrong "fix".
M1: SkyRenderer tracks textures whose wrap mode it set to ClampToEdge
and restores them to Repeat at end-of-pass, so non-sky renderers
that share the GL handle can't silently inherit clamped wrap state.
M2: Post-scene Z-offset (-120m) only fires when the SkyObject is
weather-flagged AND bit 0x08 is clear, matching retail
GameSky::UpdatePosition 0x00506dd0. The old code applied it to
every post-scene object — a no-op today (every Dereth post-scene
entry happens to be weather-flagged) but a future post-scene-only
sun rim would have been pushed below the camera.
M4: ParticleSystem.EmitterDied event lets ParticleHookSink prune dead
handles from the per-entity tracking dictionaries, fixing a slow
leak where naturally-expired emitters' handles stayed in the
ConcurrentBag forever during long sessions.
M5: SkyPesEntityId moves the post-scene flag bit to 0x08000000 so it
can't ever overlap the object-index range. Synthetic IDs stay in
the reserved 0xFxxxxxxx space.
New tests (ParticleSystemTests + ParticleHookSinkTests):
- UpdateEmitterAnchor_AttachLocal_ParticlePositionFollowsLiveAnchor
- UpdateEmitterAnchor_AttachLocalCleared_ParticleFrozenAtSpawnOrigin
- EmitterDied_FiresOncePerHandle_AfterAllParticlesExpire
- Birthrate_PerSec_EmitsOnePerTickWhenIntervalElapsed (retail-faithful
single-emit-per-frame behavior)
- UpdateEntityAnchor_WithAttachLocal_MovesParticleToLiveAnchor
- EmitterDied_PrunesPerEntityHandleTracking
dotnet build green, dotnet test green: 695 / 393 / 243 = 1331 passed
(up from 1325).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Iteration on the sky rendering pipeline to restore stars/moon visibility
at night and fix washed-out grey daytime clouds. Key fixes:
* sky.frag: disable fog-mix on sky meshes. Retail's keyframe FogEnd
(0..400m at midnight, up to 2400m during day) is calibrated for
terrain; sky meshes are authored at radii 1050-14271m which sits
past FogEnd universally, causing every sky pixel to saturate to
fogColor (dark navy). Stars, moon, dome texture all got
obliterated. The horizon-glow trade-off is noted in the shader
comment; research item to find retail's sky-specific fog range
later.
* SkyRenderer + sky.frag: promote rep.Luminosity into uEmissive so the
vertex lighting saturates properly for bright keyframes. Retail's
FUN_0059da60 non-luminous path writes rep.Luminosity into
material.Emissive via the cache +0x3c slot; we were instead using
it as a post-fragment multiply which could only dim, never brighten.
Net effect: daytime clouds now render saturated white, dome dims
correctly at night (rep.Luminosity=0.11 → Emissive=0.11), stars
and moon unchanged.
* terrain.vert: MIN_FACTOR 0.08 -> 0.0 per retail FUN_00532440 decompile
(DAT_00796344 ambient-floor = 0.0). Back-lit terrain now falls to
pure ambient rather than getting an 8% sun floor.
New research / tooling (no runtime impact):
* docs/research/2026-04-24-lambert-brightness-split.md — retail's
ambient-brightness formula pinned from PE .rdata read + live
RetailTimeProbe capture: effAmbBright = AmbBright + |sunDir| * 0.2
where scale constant 0x0079a1e8 = 0.2f exactly.
* docs/research/2026-04-23-lightning-real.md — research note on the
dat-baked PhysicsScript-driven lightning path (Rainy DayGroup has
explicit PES-triggered flash SkyObjects with 5ms time windows).
* Corrections stapled to sky-decompile-hunt-{B,C}.md: DAT_00842778 is
DirColor, DAT_0084277c is AmbColor (the hunt docs had the swap
backwards).
* tools/RetailTimeProbe/Program.cs: extended with pid=NNNN selector,
sky global probe (DirColor/AmbColor/AmbBright/sunDir/cache.amb),
and the 0x0079a1e8 scale-factor readout.
* tools/SkyObjectInspect/: throwaway dat-inspector built by the Opus
deep-dive agent. Identified GfxObj 0x010015EF as the stars layer
(A8R8G8B8 128x128 texture, 4% bright-pixel ratio).
* src/AcDream.App/Rendering/TextureCache.cs: per-texture alpha
histogram dump under ACDREAM_DUMP_SKY=1 for diagnosing "are the
clouds decoded with proper alpha" type questions.
README: rewrite to reflect current state (playable pre-alpha rendering
Dereth with animated characters, day-night cycle, weather, etc.)
instead of the stale "Phase 0 dat inventory only" description.
All 742 tests green.
Implements the other half of ObjDesc: SubPalettes (palette-range
overlays) that repaint palette-indexed textures with per-entity color
schemes. Ported algorithm from ACViewer Render/TextureCache.IndexToColor
after the user pointed out I was prematurely implementing from scratch
instead of checking all the reference repos.
The Nullified Statue of a Drudge sends (setup=0x020007DD with a drudge
GfxObj animPart replacing part 1, plus 2 texChanges targeted at part 1,
plus 1 subpalette id=0x04001351 offset=0 length=0). The TextureChanges
swap fine detail surfaces; the SubPalette with length=0 ("entire palette"
per Chorizite docs) remaps the drudge's flesh-tone palette to stone.
Without this commit, the statue looked like a normal flesh drudge
because palette-indexed textures decoded with the base flesh palette.
Added:
- Core/World/PaletteOverride.cs: per-entity record carrying
BasePaletteId + a list of (SubPaletteId, Offset, Length) range
overlays. Documents the "offset/length are wire-scaled by 8"
convention and the "length=0 means whole palette" sentinel.
- WorldEntity.PaletteOverride nullable field. Per-entity (same across
all parts), in contrast to MeshRef.SurfaceOverrides which is per-part.
- TextureCache.GetOrUploadWithPaletteOverride: new entry point that
composes the effective palette at decode time. Composite cache key
is (surfaceId, origTexOverride, paletteHash) so entities with
equivalent palette setups share the GL texture.
- ComposePalette: ports ACViewer's IndexToColor overlay loop:
for each subpalette sp:
startIdx = sp.Offset * 8 // multiply back from wire
count = sp.Length == 0 ? 2048 : sp.Length * 8 // sentinel
for j in [0, count):
composed[j + startIdx] = subPal.Colors[j + startIdx]
Critical detail: copies from the SAME offset in the sub palette, not
from [0]. Both base and sub are treated as full palettes sharing an
index space.
- StaticMeshRenderer.Draw: three-way switch on (entity.PaletteOverride,
meshRef.SurfaceOverrides) picks the right TextureCache path:
- Both → palette override (it handles origTex override internally)
- Only tex override → GetOrUploadWithOrigTextureOverride
- Neither → plain GetOrUpload
- GameWindow.OnLiveEntitySpawned: builds PaletteOverride from
spawn.BasePaletteId + spawn.SubPalettes when the server sent any.
Reference note: the user asked "but I mean THIS MUST BE IN WORLDBUILDER"
which was the right push. WorldBuilder is actually a dat VIEWER and its
ClothingTableBrowserViewModel is a 10-line stub — it doesn't apply
palette overlays because it doesn't need to. The actual algorithm lives
in ACViewer (a MonoGame character viewer), which I should have checked
earlier. CLAUDE.md updated with a standing rule: always cross-reference
all four of references/ACE, ACViewer, WorldBuilder, Chorizite.ACProtocol,
plus holtburger. A single reference can be misleading; the intersection
is usually the truth.
Tests: 77 core + 83 net = 160, all green.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Finishes the TextureChange half of ObjDesc. Characters' clothing now
renders with correct per-part textures (user-verified "looks good"
after previous "partial coverage" / "wrong clothes"). The Nullified
Statue still looks like a flesh-colored drudge because the statue's
color comes from SubPalettes (palette-indexed texture recoloring),
which is the remaining major Phase 5 piece.
The first attempt at TextureChange application was silently broken by
an ID-type mismatch: the server encodes OldTexture/NewTexture as
SurfaceTexture (0x05XXXXXX) ids, but my sub-meshes are keyed by
Surface (0x08XXXXXX) ids. The override dict was keyed by one type
and looked up by the other, so TryGetValue never hit and no override
actually applied.
Diagnosed via Phase 1 systematic debugging with resolve-level logging:
live: spawn +Acdream texChanges=20
live: texChange part=0 old=0x05000BB0 new=0x0500025D
...
live: resolve part=0 surface=0x08000519 origTex=0x05000BB0 [MATCH]
live: resolve part=0 surface=0x0800051C origTex=0x05000CBE [MATCH]
... 10/10 lines [MATCH]
The [MATCH] lines proved the server's OldTexture IS reachable via a
Surface→OrigTextureId lookup, just needed keying by the right value.
Fix:
- TextureCache.GetOrUploadWithOrigTextureOverride(surfaceId, origTexOverride):
loads the base Surface dat for its color/flags/palette, but
substitutes the override SurfaceTexture id in the decode chain.
Caches under a (surfaceId, origTexOverride) composite key.
- MeshRef.SurfaceOverrides is now Dictionary<uint, uint> keyed by
Surface id, value = replacement OrigTextureId. Null means no
overrides.
- GameWindow.OnLiveEntitySpawned now does TWO passes when texture
changes are present:
1. Group the raw server changes by PartIndex into (oldOrigTex →
newOrigTex) dicts
2. For each affected part's post-animPartChange GfxObj, iterate
its Surfaces list, resolve each Surface → OrigTextureId, and
if that matches a raw change's oldOrigTex, write an entry
Surface id → newOrigTex into the final override map
- StaticMeshRenderer.Draw: when sub-mesh surface id has an override,
call GetOrUploadWithOrigTextureOverride instead of GetOrUpload.
Verified live: +Acdream's clothing renders correctly, NPCs are
"much better" (characters previously naked are now dressed). Statue
has the full mechanical pipeline working (resolve diagnostic shows
2/2 Surfaces [MATCH] for the statue's override dict) but its visible
color comes from the separate SubPalette overlay that isn't wired yet.
Also added a statue-targeted diagnostic block that dumps its full
ObjDesc contents (texChanges + subPalettes + animPartChanges) by
name match, which is how I traced the Nullified Statue of a Drudge's
specific ObjDesc. Lives under `if (isStatue && ...)` so normal logins
aren't spammed.
Cross-referenced against two new references this session:
* references/Chorizite.ACProtocol (cloned from github.com/Chorizite/
Chorizite.ACProtocol.git on user's suggestion) — confirms the
ObjDesc field order and PackedDword-of-known-type convention.
* references/WorldBuilder/... (already in repo) — confirms the
Surface→OrigTexture→SurfaceTexture→RenderSurface chain and the
P8/INDEX16 palette decode path.
Tests: 77 core + 83 net = 160, all green.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three root causes found via systematic debugging after the user reported
that the dc60405 texture fix and 4763b97 height table fix had no visible
effect on Holtburg.
## Heightmap transpose (LandblockMesh.Build)
Phase 1's LandblockMesh.Build indexed block.Height as y*9+x but AC packs
per-vertex heights in x-major order (x*9+y, matching ACViewer's
LandblockStruct: Height[x * VertexDim + y]). The bug was invisible on
flat landblocks (Phase 1 smoke test) but left buildings buried by 10-13
world-Z units on Holtburg, because building Frame.Origin positions
reference the un-transposed ground truth.
Diagnostic evidence (before fix, Holtburg 0xA9B4FFFF):
entity 0x020000A5 at ( 84.6,126.0) entityZ= 66.03 terrainZ= 78.15 delta=-12.13
entity 0x02000118 at ( 74.2,139.9) entityZ= 66.03 terrainZ= 78.92 delta=-12.89
After fix: deltas are 0.03 to 2.18 — buildings now sit on the ground
with small positive offsets for foundations.
Regression test added: Build_HeightmapPackedAsXMajor_NotYMajor asserts
asymmetric heights land at the correct world positions.
## Solid-color surfaces with Translucency=1.0 (SurfaceDecoder.DecodeSolidColor)
The "bright pink doors and windows" the user saw were 11 Holtburg
surfaces with OrigTextureId==0 — these carry a ColorValue instead of
a texture chain. Phase 2a's TextureCache dropped them into the magenta
fallback. All 11 turned out to be Base1Solid|Translucent with
Translucency=1.00, meaning "fully transparent placeholder surface"
(debug ColorValue is gray/green/red/blue/black, never displayed).
DecodeSolidColor now takes a translucency parameter and multiplies
alpha by (1 - translucency), so Translucency=1.0 → alpha=0, and the
mesh shader's existing alpha discard (< 0.5) makes the pixel invisible.
TextureCache honors Surface.Type.HasFlag(Base1Solid) and passes
surface.Translucency through.
Regression tests added: DecodeSolidColor_Opaque_PreservesAlpha and
DecodeSolidColor_FullyTranslucent_AlphaGoesToZero.
## Clipmap alpha-key (DecodeIndex16)
AC convention (per ACViewer TextureCache.IndexToColor): on surfaces
marked Base1ClipMap, palette indices 0..7 are treated as fully
transparent regardless of their actual palette color. Without this,
low-index pixels on clipmap surfaces (typically doorway cutouts and
foliage) render as opaque using whatever sentinel color is at those
palette slots.
DecodeRenderSurface now takes an isClipMap parameter. TextureCache
passes Surface.Type.HasFlag(Base1ClipMap). DecodeIndex16 forces
rgba=(0,0,0,0) when isClipMap && idx < 8.
Regression test added: DecodeIndex16_ClipMap_ZerosAlphaForLowIndices.
## Notes
- dc60405's PFID_INDEX16 palette decoder remains correct — no change.
- 4763b97's LandHeightTable wiring remains correct — real-table lookup
still runs, it just happens to be linear at Holtburg's height range.
The fix is forward-compatible with mountains elsewhere.
- All three bugs were invisible to the original unit tests. The new
regression tests pin them down.
## State
- dotnet build: 0 warnings, 0 errors
- dotnet test: 42 passing (was 38 + 4 new)
- Runtime: 126 entities hydrated on Holtburg, no exceptions, no
magenta fallback (counter was 11, now 0 via diagnostic confirmation)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Addresses the 'doors, windows, and alpha-keyed parts render bright
pink' issue the user observed after the Phase 2a visual checkpoint.
SurfaceDecoder gains a second overload taking an optional Palette
parameter. When the render surface format is PFID_INDEX16 and a
palette is supplied, each 16-bit value in SourceData is treated as
an index into Palette.Colors (a List<ColorARGB>) and the corresponding
ARGB color's channels are written to the output buffer. The original
no-palette overload is preserved so the Task 3 unit tests that
confirm INDEX16 -> magenta fallback still describe their behavior
correctly (INDEX16 without a palette still returns magenta).
TextureCache now resolves the RenderSurface's DefaultPaletteId via
the dats and passes the resulting Palette (or null) to the decoder.
mesh.frag adds an alpha cutout: fragments with sampled alpha < 0.5
are discarded. Without this, transparent regions of alpha-keyed
textures (doors, windows, foliage cutouts) would render as opaque
rectangles using the texture's background color. This is the
standard alpha-tested approach, simpler than full alpha blending
and matches how AC's original client rendered these surfaces.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Erik