The directional-shadow topology rebuilt on every streaming-churn frame
and was the measured body of the run-hitch stalls (701 of 708 baseline
stalls alloc-correlated):
- The draw sort comparer's enum-vs-enum CompareTo bound to
Enum.CompareTo(object) and boxed BOTH operands on every comparison —
a constant ~38.9 MB of garbage per topology rebuild (~4M boxes),
handing the GC a forced gen0 collection mid-frame. The full ~100k-draw
sort is replaced outright: draws hash-group by exact DrawKey in one
O(n) pass over retained chained-index arrays, and only the
few-thousand DISTINCT group keys sort (order-preserving packed
material|cull|firstIndex|baseVertex + count|slot|layer|foliage keys,
first-appearance tie-break) — bit-identical emission order to the old
stable sort, near-zero allocation, and no per-draw comparisons at all.
- The caster frame sorts 4-byte indices keyed on SortKey.Value instead
of shuffling multi-hundred-byte records through a boxing comparer.
- Owner-approved pipelining: on a frame whose shadow inputs just changed
(the same frame already paying frame-view/landscape rebuilds), the
caster-frame and prepared-draws topology rebuilds defer to the next
quieter frame, capped at two consecutive deferrals — inside the GPU
fence depth, so retained draws never reference a released arena range.
First build, generation change, caster BuildSequence change, and
journal overflow force the immediate path; deferred refreshes skip
identity-mismatched journal rows.
Owner-accepted in both presentation modes: stall frames 5.8/s -> ~0.45/s
uncapped (0.49/s capped), median stall 20.3 -> 13.7 ms, >25 ms frames
near zero, 275 fps uncapped baseline restored. Allocation gate: a warmed
topology rebuild must allocate <2 KiB (DirectionalShadowPreparedDrawTests).
docs/ISSUES.md carries the full evidence trail; the residual
content-proportional rebuild milliseconds are filed as the
incremental-topology successor, and the pre-existing town-view scaling
latch is filed as #432.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Round-5 narrow re-review of 754d59d9: APPROVE (verified to the SPIR-V
disassembly; shadows-on path proven opcode-identical). Four
non-blocking nits.
N1 (test robustness): T3 (ReceiverVertexShadersFallBackToThePlainLightDirectionWhenTheShadowFlagIsClear)
was satisfied by comment prose -- round 5's own explanatory comments
quote the exact plain-pipeline direction substring
("-uLights[i].dirAndRange.xyz, ... matched bit-for-bit"), so the
substring-only assertion passed even with the code mutated. Fixed:
comment lines are now stripped (new StripLineComments helper) before
any assertion, and a new ordered regex per shader asserts the exact
branch SHAPE -- shadowGatedOff ? -uLights[...] : normalize(uShadow...)
-- not just substring presence. Mutation-tested locally against the
rewritten test: (1) swapping the ternary's true/false operands --
FAILED (previously passed); (2) deleting the fallback entirely,
collapsing to the pre-round-5 buggy expression -- FAILED (previously
passed). Original file restored and reverified passing after each
mutation.
N2 (doc accuracy): "numerically the plain pipeline" overstated the
round-5 fix in three places (plan doc, mesh_atmospheric.vert,
terrain_atmospheric.vert). The direction expression is bit-for-bit;
the SUM is not, because the atmospheric shaders' split ambient+point
vs directional accumulation (and terrain's two varyings vs the plain
pipeline's one) reassociates float summation order by ~1 ulp -- which
is exactly the measured mean |Delta| 0.007 the coordinator's own
pixel-proof evidence already recorded (well under the 65 px noise
floor). All three rewritten to say the receiver "matches the plain
pipeline to within float summation-order rounding (measured mean
|Delta| 0.007 on the offline scene)."
N3 (coverage): RenderPrepared's own cascadeCount == 0 exit (the F2
fix) had no direct test even though RenderPrepared already has 8
direct call sites in this file. Added one: ResidentMaximumReachMeters
at/below CameraNearMeters, passed straight to RenderPrepared with an
otherwise-fully-valid environment (so the fitter, not the environment
gate, is what returns zero cascades), asserting IsBindableFor true /
IsValidFor false.
N4 (latent): RenderPrepared's OWN "if (!environment.ShouldRender)"
exit is a fourth bufferless-disabled path -- unreachable via Render
(whose own gate already validates ShouldRender first) but the same
shape, and RenderPrepared is called directly by tests and any future
caller. Took the preferred fix: publishes the disabled binding there
too, via the same helper, so every exit on a frame that draws the
world publishes when the pack's AtmosphericFrame is bound. Also made
EvaluateGateAndPublishDisabledBinding self-contained: it now resets
_currentFrameBinding to Disabled on its own entry instead of relying
on Render having done so first (idempotent with Render's own reset).
Regenerated SPIR-V: mesh_atmospheric.vert and terrain_atmospheric.vert
are comment-only changes (N2), so only the manifest's source hashes
changed -- compiled .spv bytes are unchanged, consistent with round 3's
precedent for comment-only shader edits.
Verify: Release build 0 warnings/0 errors. App hermetic-lane filter
6,055/0 failed. Full hermetic-filtered solution: 15,283/0 failed
across 15 projects.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Round 4 (eec95535) fixed wind but introduced a new lighting bug: the
receiver VERTEX shaders source the sun direction from the shadow block,
not only the shadow visibility term. mesh_atmospheric.vert's
accumulateLights read uShadowLightDirectionAndSource unconditionally for
every directional light; terrain_atmospheric.vert did the same for its
single sun term. Round 4's PublishDisabledReceiverBinding writes
direction (0,0,1) into that block on every shadow-gated-off frame (user
sun-shadow-strength 0 in daylight, indoor/portal cover, night), so every
such frame was lighting outdoor terrain and objects from straight
overhead instead of the authored sun. Publishing the environment's real
direction would not have restored parity either -- the celestial shadow
source direction (sun/moon disc) is not the authored light direction.
F1 (BLOCKER): fixed in the shaders themselves, exact parity with the
plain pipeline. Both receiver verts now branch on the same flag bit
acdreamDirectionalShadowVisibility already reads
((uShadowTextureAndFlags.w & 1u) == 0u) and, when clear, use the EXACT
plain-pipeline expression instead of the shadow block's direction:
-uLights[i].dirAndRange.xyz in mesh_atmospheric.vert (matching
mesh_modern.vert, hoisted out of the light loop as a uniform branch);
-uLights[0].dirAndRange.xyz in terrain_atmospheric.vert (matching
terrain_modern.vert's sunDir/-sunDir form). The (0,0,1) word in the
disabled block stays as the documented normalize()-cannot-NaN guard; its
comment now says so explicitly since it is no longer read as a light
direction when the flag is clear.
F2: RenderPrepared's cascadeCount == 0 return is a third bufferless-
disabled path reachable from a frame that already passed Render's own
two gates (the cascade fitter can still find zero usable cascades) --
publishes the same disabled binding now, via the same
PublishDisabledReceiverBinding helper (re-signatured to take a bare
AtmosphericFrameBufferBinding so all three call sites -- Render's two
early-outs plus this one -- share it).
F3: removed a stray duplicated " -- Closeout and merge" fragment under
the plan's VM7 heading.
F4: corrected the false "the flag bit makes it numerically the plain
lighting sum" claim in the plan's round-4 paragraph and in
WbDrawDispatcher.DirectionalShadowReceivers.cs -- the flag bit alone
only fixed the shadow VISIBILITY term (already correct before round 4);
it took both that AND round 5's light-DIRECTION fallback to actually
match the plain pipeline.
T1: extracted Render's gate prologue (environment evaluate -> two
early-outs -> PublishDisabledReceiverBinding) into internal
EvaluateGateAndPublishDisabledBinding(frame, in input, out environment,
out environmentGateTicks), behaviour-preserving, called by Render before
it touches world/terrain -- the ArgumentNullException.ThrowIfNull(world)/
ThrowIfNull(terrain) calls keep their exact position relative to the
gate. No test in this suite constructs a real WbDrawDispatcher +
TerrainModernRenderer pair (still true), so this extraction is what
makes the gate itself testable; two new tests drive it directly with
PlayerInsideCell: true and with ResidentMaximumReachMeters <=
CameraNearMeters, asserting TryGetCurrentFrameBinding true / IsValidFor
false for both.
T2: proves the actual composition WbDrawDispatcher.PipelinesFor and
TerrainModernRenderer both use -- TryGetCurrentFrameBinding feeding
ShouldSelectReceiverPipeline -- selects the receiver pipeline for the
atmospheric world pass once a disabled binding is published, and still
refuses a non-atmospheric pass name.
T3: shader-source guard (same style as AtmosphericPostProcessGraphTests'
existing shader-text tests) pinning that both receiver verts contain the
flag-gated fallback and reference the same uLights expression the plain
verts use, so a future edit that drops the fallback fails this test
instead of only showing up in a pixel capture.
T4: the (0,0,1) test's doc comment and an inline assertion comment now
say the value is a NaN guard, not a light direction.
Regenerated SPIR-V: mesh_atmospheric.vert and terrain_atmospheric.vert
recompiled to different bytes this time (a real code change, not a
comment); manifest updated to match.
Verify: Release build 0 warnings/0 errors. App hermetic-lane filter
6,054/0 failed. Core.Tests 4,695/0 failed. RenderPackValidator 30/30.
Full hermetic-filtered solution: 15,282/0 failed across 15 projects.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The reviewer's offline pixel apparatus found a real design defect, not a
test artefact: foliage wind was welded to "directional shadows rendered
this frame." Evidence: offline High preset, sun-shadow-strength=0,
wind-strength 2 + lean/branch 1 m — wind-on vs wind-off at the same
pinned clock differed by only 49-65 px, inside the apparatus's own 22 px
run-to-run noise floor (no measurable motion). A CPU probe independently
confirmed ResolveFoliageWind was correct (first advance snaps to Clear
0.25/0.15, gate 1, one graph) — the correct uniform never reached the
world pass.
Root cause: DirectionalSunShadowRenderer.Render's two early-out paths
(!environment.ShouldRender, ResidentWindowUnavailable) left
_currentFrameBinding at its pure Disabled (no-buffer) default.
WbDrawDispatcher.PipelinesFor and TerrainModernRenderer's matching
selection logic only choose the atmospheric receiver pipeline
(mesh_atmospheric, the only pipeline that #includes foliage_wind.glsl)
when TryGetCurrentFrameBinding returns true; with no buffer it always
returned false, so the world pass silently fell back to the plain
mesh_modern pipeline, which has no wind code at all. Because the shadow
gate is ActiveDayGroupMultiplier = dayGroupPolicy x elevationResponse x
strength, this killed wind every night (elevation response -> 0), at
user sun-shadow-strength 0, and under the portal/login cover.
Fix (decouple, not patch): DirectionalShadowFrameBinding gained
IsBindableFor ("a real current-frame allocation exists") separate from
IsValidFor ("...and it is Enabled with real shadow content" -- kept
exactly as VolumetricShaftRenderer's own gate needs it).
TryGetCurrentFrameBinding now returns IsBindableFor. When the built-in
pack supplies an AtmosphericFrame binding (declared packs never do, so
their receiver shaders -- which never declare set 3 binding 5 -- are
unaffected), Render's two early-out paths call a new
PublishDisabledReceiverBinding: it allocates one real ring slice and
writes a DISABLED DirectionalShadowUniforms block -- every matrix
Identity, every control/bias term zero, TextureAndFlags all zero (bit 0
clear is exactly what directional_shadow_receiver.glsl's
acdreamDirectionalShadowVisibility already reads as "no shadow, full
visibility" via its existing early return 1.0), and a unit light
direction (0,0,1) so a fragment shader's normalize() can never produce
NaN. BindDirectionalShadowReceiver and TerrainModernRenderer's
shadow-buffer bind now check Buffer is not null instead of Enabled, so
the disabled block actually gets bound once it is selected.
PublishDisabledReceiverBinding is internal (not private) specifically so
it is testable without standing up a real WbDrawDispatcher/
TerrainModernRenderer pair -- no test in this suite constructs either.
New tests: (a)/(b) PublishDisabledReceiverBinding is bindable-not-valid
with a bound AtmosphericFrame and a genuine no-op with an unbound one;
(c) BindDirectionalShadowReceiver emits both UniformDirectionalShadow and
UniformAtmosphericFrame binds for a disabled binding; (d)
VolumetricShaftRenderer's gate still reports NoCurrentDirectionalShadow
for a disabled binding. ShouldSelectReceiverPipeline itself is untouched
and its existing tests (parametrized directly on bindingValid) remain
valid; no existing test asserted the old "disabled shadows -> plain
pipeline / no binding" behaviour in a way this fix invalidates -- every
existing caller either bypasses Render (calls RenderPrepared directly)
or uses a stale-serial binding IsBindableFor still correctly rejects.
Verify: Release build 0 warnings/0 errors. App hermetic-lane filter
6,050/0 failed. Core.Tests 4,695/0 failed. Full hermetic-filtered
solution: 15,278/0 failed across 15 projects.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Opus dual-lens review of the three VM6 commits (0930c35d, 39e8408c,
6cc5e183) found two blockers and two should-fix issues; all landed here
along with the review's nits and documentation corrections.
Blockers:
- A1: the procedural-scenery classifier tested bit 31 alone instead of
the full top nibble (0xF000_0000 == 0x8000_0000), so it also matched
LandblockStaticEntityIdAllocator's 0xC... namespace (fences/gates/
building shells with a cutout subset), the 0xDA11_D0xx paperdoll id,
and the 0xFFFF_FF01 portal-tunnel id as procedural scenery — all
three would have swayed. ProceduralSceneryIdAllocator.IsInNamespace
now does the exact top-nibble test; FoliageWindClassification
delegates to it.
- A2: GroupKey (the receiver's instance-batching key) did not carry
FoliageFlags while the caster's dedup key already did, so a scenery
instance and a non-scenery instance sharing a mesh subset coalesced
into one receiver InstanceGroup whose flags were last-writer-wins —
disagreeing with the correctly-keyed caster. GroupKey now carries
FoliageFlags, computed before key construction and set exactly once
at group creation; the imperative re-stamp is gone, and CachedBatch's
now-redundant FoliageFlags field is removed.
Should-fix:
- A3: the world receiver pass bound UniformAtmosphericFrame only by
accident (leftover from the caster pass, which runs first each
frame, since Vulkan binding state isn't reset between passes).
DirectionalShadowFrameBinding now carries the caster's exact
AtmosphericFrameBufferBinding and BindDirectionalShadowReceiver binds
it explicitly.
- A4: a Setup-composed tree's opaque trunk part never got the trunk
flag because HasCutoutSubset is cached per GfxObj part, not per
entity. FoliageWindClassification.ComputeEntityHasCutoutSubset now
ORs HasCutoutSubset across an entity's resolved sibling parts once
per entity, threaded into ClassifyBatches/AddDirectionalShadowBatches
via a new optional override parameter.
Nits: A5 hashes the per-vertex flutter seed relative to the instance
origin instead of absolute world XY (fp32 sin() precision loss at far
landblock corners), mirrored in both foliage_wind.glsl and
FoliageWindModel; A7 documents the max(maxHeight, 0.5) divide-guard as
a deliberate pseudocode divergence; A8 switches FoliageWindExclusions'
construction to ToFrozenSet() and softens the "never stale" doc
comment to "no slower than one frame behind."
Tests added: top-nibble classification (0xFFFFFFFFu now correctly
false), GroupKey inequality across entity-driven scenery/landblock-
static classification, a caster-batch test proving the same pairing
never coalesces, ComputeEntityHasCutoutSubset unit + end-to-end
two-part-Setup tests, the caster→receiver AtmosphericFrame binding
carry-through, flutter-hash translation invariance relative to
instance origin, and a Storm-wind mid-height displacement floor
guarding against a "no motion" regression.
Docs: plan VM6 body corrected to the five-row WeatherKind table, "bits
1 and 2", "all four" caster shaders, and top-nibble wording throughout;
the owner gate checklist's Rain/Storm step; the stale v1-only shader-
interface compatibility entry; semantic-bindings-v1.md's v2 members
folded into the main 192-byte block; the IA-25 register row's top-
nibble wording; AtmosphericFrameInputs.cs's ABI size reference.
foliage_wind.glsl's A5 change recompiled exactly the five shaders that
include it (mesh_atmospheric.vert, the four directional_shadow_world_*
casters) plus the manifest; no other .spv changed.
Verify: Release build 0 warnings/0 errors. App hermetic-lane filter
6,041/0 failed (no environment-specific failures this run).
RenderPackValidator 30/30. Full hermetic-filtered solution: 15,269/0
failed across 15 projects.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Procedural-scenery foliage (trees/bushes — entity ids in the
ProceduralSceneryIdAllocator's 0x8XXYYIII namespace) sways with weather in
mesh_atmospheric.vert and all four directional_shadow_world_* caster vertex
shaders, both calling the identical new foliage_wind.glsl include so the
shadow moves with the leaf by construction.
Classification (FoliageWindClassification, AcDream.App.Rendering.Wb): two
new BatchData.flags bits, computed once per (entity, subset) from four
inputs — entity id (bit 31 for procedural scenery), the pack's declared
FoliageExclusions membership, the subset's TranslucencyKind, and
ObjectRenderData.HasCutoutSubset (computed once per mesh at build time, not
per frame). Bit 1 marks an alpha-cutout leaf subset; bit 2 marks an opaque
trunk subset (only when its own mesh also owns a cutout subset, so rocks
stay still). WbDrawDispatcher.ClassifyBatches (world receiver) and
AddDirectionalShadowBatches (caster) call this with the same four inputs, so
casters and receivers classify identically without needing to share state.
Retail's mesh_modern/terrain_modern/mesh_detail pipelines never read these
bits, so pack-off output is unaffected.
Motion model (foliage_wind.glsl, mirrored bit-for-bit in the new
FoliageWindModel for hermetic CPU tests): height-squared-scaled slow lean
for every foliage subset, plus branch swing and per-vertex-hash-decorrelated
flutter for cutout subsets only. AtmosphericPostProcessGraph.ResolveFoliageWind
resolves the wind block once per frame.Serial — advanced by whichever of
RenderDirectionalShadows (which runs first) or RenderPostProcess is called
first that frame, with the second reading the already-advanced state, which
is what keeps the caster and receiver reading byte-identical clock/strength
values. The per-day-group mean/gust target (AtmospherePolicyDeclaration.
FoliageWindByDayGroup, keyed by the same day-group index convention
ActiveDayGroupMultipliers already established: Clear/Cloudy/Overcast/Rainy)
eases toward its target over WeatherSystem.TransitionSeconds (10s) so a
weather change never snaps; wind-enabled off or indoor instead gates the
OUTPUT to an exact zero (not an asymptotic approach) so a settings toggle or
cell transition is immediate. The wind clock is a Stopwatch started at graph
construction (monotonic, session-relative magnitude for GPU sin() accuracy),
overridable by the same ACDREAM_SKY_PHASE_SECONDS pin SkyRenderer already
uses, for deterministic offline gates.
New settings: wind-enabled, wind-strength, wind-direction-degrees (225°
default — no authored retail wind direction exists to read),
wind-lean-metres, wind-branch-metres, wind-flutter-metres (0 on Low),
wind-canopy-height-metres.
Register row IA-25 files this as an intentional, strictly opt-in divergence:
retail applies no per-vertex wind displacement to any geometry. Known,
accepted limitation: classification is per mesh-subset (one BatchData.flags
word per indirect-draw batch), not per entity instance, so the rare case of
one mesh subset being reachable from both a procedural-scenery and a
non-scenery placement would classify all of that subset's instances alike.
Tests: FoliageWindClassificationTests (the full classification matrix),
FoliageWindModelTests (identity on non-foliage/calm-wind/base-vertex,
canopy-top displacement bound, z-never-increases, trunk has no flutter
term), RenderPackAtmospherePolicyEvaluationTests (exact day-group lookup,
no interpolation across day-group ids, easing convergence without overshoot
or discontinuity), AtmosphericShaderAbiTests (each of the five shaders calls
acdreamFoliageDisplace exactly once; mesh_modern/terrain/mesh_detail call it
never), and four AtmosphericPostProcessGraphTests additions (indoor/disabled
exact-zero gating, settings-to-UBO wiring, same-frame-Serial idempotency —
the last proxies the caster/receiver agreement invariant without needing
this hermetic harness's WbDrawDispatcher/TerrainModernRenderer dependency
chain to exercise RenderDirectionalShadows directly).
App hermetic filter: 6015/6017 (the same 2 pre-existing failures as VM6a,
confirmed unrelated). Core.Tests hermetic: 4697/4697. RenderPackValidator.Tests:
30/30. Full solution Debug and Release builds green. Shader recompile
touched exactly the 5 edited files' .spv (plus manifest); the retail oracle
set and every other pack shader are byte-identical.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
AtmosphericFrame (set 3/binding 5) grows additively from 160 to 192 bytes:
two appended vec4 members, uAtmosphereClockWind and uAtmosphereWindAmplitude,
carry the foliage-wind clock/weather and amplitude inputs VM6b's shader
displacement will read. RenderPackShaderAbi renames the old constant to
AtmosphericFrameSizeBytesV1 (160), adds AtmosphericFrameSizeBytesV2 (192),
keeps AtmosphericFrameSizeBytes pointing at the current (v2) size, and adds
ShaderAbiVersion = 2. RenderPackSpirvValidator.ValidateAtmosphericFrame
accepts either the v1 (seven-member, 160-byte) or v2 (nine-member, 192-byte)
shape and rejects anything else naming both — this is why the frozen
external sample packs under samples/*/Shaders/*.spv, whose GLSL sources are
not in this tree, need no rebuild: a v1 shader bound to the 192-byte buffer
still reads correctly, since a bound range only needs to be >= the block's
own declared size.
DirectionalSunShadowRenderer's caster pass now binds AtmosphericFrame too
(both the multiview and per-cascade sites), through a new
AtmosphericFrameBufferBinding the graph owns and supplies via
DirectionalSunShadowRenderInput. AtmosphericPostProcessGraph.RenderDirectionalShadows
builds its own 192-byte ring allocation for this, separate from the world
receiver's frame block, because the caster pass runs before RenderPostProcess
constructs that block within the same frame. The four world caster pipeline
variants (opaque/cutout, base/multiview) are now allowed to declare binding
5 in the validator; terrain casters are untouched.
This commit is plumbing only: the two new members are always written but
never read by any shader yet (zero placeholders), so pack-on and pack-off
output are both pixel-identical to before. VM6b wires the real weather-driven
values and the shader-side displacement.
App hermetic filter: 5972/5974 (2 pre-existing failures unrelated to this
change, confirmed against the unmodified baseline). Core.Tests hermetic:
4697/4697. RenderPackValidator.Tests: 30/30. VulkanShaderManifestTests
(retail oracle set): 7/7, byte-identical.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Erik