Iteration on the sky rendering pipeline to restore stars/moon visibility
at night and fix washed-out grey daytime clouds. Key fixes:
* sky.frag: disable fog-mix on sky meshes. Retail's keyframe FogEnd
(0..400m at midnight, up to 2400m during day) is calibrated for
terrain; sky meshes are authored at radii 1050-14271m which sits
past FogEnd universally, causing every sky pixel to saturate to
fogColor (dark navy). Stars, moon, dome texture all got
obliterated. The horizon-glow trade-off is noted in the shader
comment; research item to find retail's sky-specific fog range
later.
* SkyRenderer + sky.frag: promote rep.Luminosity into uEmissive so the
vertex lighting saturates properly for bright keyframes. Retail's
FUN_0059da60 non-luminous path writes rep.Luminosity into
material.Emissive via the cache +0x3c slot; we were instead using
it as a post-fragment multiply which could only dim, never brighten.
Net effect: daytime clouds now render saturated white, dome dims
correctly at night (rep.Luminosity=0.11 → Emissive=0.11), stars
and moon unchanged.
* terrain.vert: MIN_FACTOR 0.08 -> 0.0 per retail FUN_00532440 decompile
(DAT_00796344 ambient-floor = 0.0). Back-lit terrain now falls to
pure ambient rather than getting an 8% sun floor.
New research / tooling (no runtime impact):
* docs/research/2026-04-24-lambert-brightness-split.md — retail's
ambient-brightness formula pinned from PE .rdata read + live
RetailTimeProbe capture: effAmbBright = AmbBright + |sunDir| * 0.2
where scale constant 0x0079a1e8 = 0.2f exactly.
* docs/research/2026-04-23-lightning-real.md — research note on the
dat-baked PhysicsScript-driven lightning path (Rainy DayGroup has
explicit PES-triggered flash SkyObjects with 5ms time windows).
* Corrections stapled to sky-decompile-hunt-{B,C}.md: DAT_00842778 is
DirColor, DAT_0084277c is AmbColor (the hunt docs had the swap
backwards).
* tools/RetailTimeProbe/Program.cs: extended with pid=NNNN selector,
sky global probe (DirColor/AmbColor/AmbBright/sunDir/cache.amb),
and the 0x0079a1e8 scale-factor readout.
* tools/SkyObjectInspect/: throwaway dat-inspector built by the Opus
deep-dive agent. Identified GfxObj 0x010015EF as the stars layer
(A8R8G8B8 128x128 texture, 4% bright-pixel ratio).
* src/AcDream.App/Rendering/TextureCache.cs: per-texture alpha
histogram dump under ACDREAM_DUMP_SKY=1 for diagnosing "are the
clouds decoded with proper alpha" type questions.
README: rewrite to reflect current state (playable pre-alpha rendering
Dereth with animated characters, day-night cycle, weather, etc.)
instead of the stale "Phase 0 dat inventory only" description.
All 742 tests green.
Phase 2 of the sky port. Empirically confirmed from the Phase 1 dump
(ACDREAM_DUMP_SKY=1 on the live Dereth region): retail distinguishes
self-illuminated sky meshes from lit ones by the `Surface.Luminosity`
FLOAT field (0..1), NOT by the `SurfaceType.Luminous` flag bit (none of
Dereth's sky meshes have the flag set).
Observed values on the 4 currently-visible sky GfxObjs:
GfxObj 0x010015EE (dome, 4 surfaces) Luminosity = 1.0
GfxObj 0x010015EF (upper cloud) Luminosity = 0.0
GfxObj 0x01004C36 (lower drift cloud) Luminosity = 0.0
GfxObj 0x01001348 (sun/moon additive) Luminosity = 1.0
Retail uses this as an emissive coefficient in the per-vertex lighting
formula (decompiled chunk_00500000.c:7535 FUN_00508010 + chunk_00530000.c
AdjustPlanes per-vertex math):
tint = clamp(vec3(Luminosity) + AmbColor*AmbBright
+ max(dot(N, -sunDir), 0) * DirColor*DirBright,
0.0, 1.0)
fragment = texture * tint
When Luminosity=1.0 the clamp saturates → full texture brightness
regardless of time of day (dome gradient preserved; sun/moon always
bright). When Luminosity=0.0 only the ambient + diffuse term drives the
tint, so clouds pick up the time-of-day ambient (purple at midnight
per AmbColor=(200,100,255)×AmbBright=0.4 ≈ (0.31,0.16,0.40); warm tan
at dusk; pale-cool at noon).
Also empirically confirmed: raw SkyObjectReplace Transparent/Luminosity
/MaxBright are in 0..100 percent range (observed 11, 15, 22, 66, 100,
and -1 sentinel). The `/100` divide in SkyDescLoader (eeae83a) is
retail-correct; `_DAT_007a1870` in the decompile must be 0.01f.
Code changes:
- src/AcDream.Core/Meshing/GfxObjSubMesh.cs: new `Luminosity` field on
the per-submesh record (0..1, defaults to 0 for non-sky meshes).
- src/AcDream.Core/Meshing/GfxObjMesh.cs: pull Surface.Luminosity when
building submeshes (alongside existing Translucency capture).
- src/AcDream.App/Rendering/Sky/SkyRenderer.cs:
- SubMeshGpu gains SurfLuminosity, propagated from GfxObjSubMesh.
- Render() pushes uAmbientColor/uSunColor/uSunDir once per frame from
the interpolated keyframe; uEmissive once per submesh.
- uTint uniform removed (replaced by the vTint varying computed in
the vertex shader).
- src/AcDream.App/Rendering/Shaders/sky.vert: computes vTint per-vertex
using the retail AdjustPlanes formula.
- src/AcDream.App/Rendering/Shaders/sky.frag: consumes vTint, drops
uTint uniform. uLuminosity (the per-keyframe SkyObjectReplace
override) still applied as a final scalar multiply.
Expected visual difference from Phase 1 baseline:
- Dome gradient: IDENTICAL (Luminosity=1 saturates).
- Sun / moon: IDENTICAL (Luminosity=1 saturates, additive blend).
- Clouds: now tinted by time of day. Midnight → purple haze. Noon →
pale cool. Dusk → warm tan.
Open questions (unchanged from Phase 1 doc):
- Does the 15s LightTickSize throttling need porting? Phase 3.
- Does FUN_00532440 (AdjustPlanes per-cell terrain relight) need
porting for non-sky geometry to follow the sky? Phase 3.
Build + 717 tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The uncommitted uTint=AmbientColor-for-alpha-submeshes experiment (from
the 2026-04-22 inference) dimmed the sky dome's baked gradient — a
user-verified visual regression. Reverting to the eeae83a baseline
(uTint=Vector4.One for every submesh) while we execute the proper
retail-verbatim port.
Research: three parallel decompile-hunt agents landed verifying
retail's ground-truth sky pipeline for the first time (prior audits
searched for stripped symbol names; the trail opened via the Region
dat-type-index 0x1c registration at chunk_00410000.c:12952). Key
retail functions now mapped in chunk_00500000.c:1097-7535:
- FUN_00501530: keyframe bracket-picker (with 1.0f wrap denominator)
- FUN_00501600: sun+ambient interpolator (sunVec = DirBright ×
(sin yaw·cos pit, cos yaw·cos pit, sin pit))
- FUN_00501860: fog interpolator
- FUN_00502820: SkyDesc::Unpack (2 doubles + DayGroup list)
- FUN_00502a10: build per-frame sky-object table
- FUN_00505f30: apply light state + per-cell AdjustPlanes relight
- FUN_005062e0: per-frame sky tick (throttled by LightTickSize)
- FUN_00508010: sky-object render loop (enqueues through the NORMAL
mesh pipeline via FUN_00514b90 — not a bespoke path)
Surprise findings:
- D3DRS_AMBIENT is set to 0 once at init and NEVER changes per-frame
(chunk_005A0000.c). The r12-inferred "clouds = texture × D3DRS_
AMBIENT" formula is falsified. Retail instead routes keyframe
AmbColor through per-vertex lighting on non-Luminous sky meshes
via _DAT_008682bc/c0/c4.
- Retail does NOT anchor the sky to the camera or use a separate
sky projection. Sky meshes live in world space and follow the
camera via scene-graph parent.
- FUN_00532440 (AdjustPlanes) re-lights every terrain cell on every
keyframe tick — the "terrain follows the sky" effect we don't yet
reproduce.
Phase 1 code change (this commit):
- src/AcDream.App/Rendering/Sky/SkyRenderer.cs: revert uTint to white
for all submeshes (the per-submesh blend split stays — sun gets
additive, clouds get alpha). Keep the `keyframe` parameter in the
signature for Phase 2 readiness. Comments now cite the retail
functions and reference docs instead of the (disproven) r12 formula.
- src/AcDream.Core/World/SkyDescLoader.cs: ACDREAM_DUMP_SKY=1 logs
the entire Region SkyDesc on load — DayGroups, SkyObjects, every
SkyTimeOfDay keyframe, and every SkyObjectReplace with RAW pre-/100
Transparent/Luminosity/MaxBright values so we can settle the unit
question empirically.
- src/AcDream.App/Rendering/Sky/SkyRenderer.cs: ACDREAM_DUMP_SKY=1
additionally logs each sky GfxObj's Surfaces and their SurfaceType
flags on first load, so we can identify which meshes carry the
Luminous bit (dome? sun? moon? stars?) vs which are lit.
- src/AcDream.App/Rendering/GameWindow.cs: passes the interpolated
keyframe to the sky renderer (kept — needed for Phase 2).
Research docs (pushed as part of this commit):
- docs/research/2026-04-23-sky-retail-verbatim.md: full synthesis
with retail function map, struct layouts, globals, pseudocode, and
a 4-phase port plan.
- docs/research/2026-04-23-sky-decompile-hunt-{A,B,C}.md: raw hunt
outputs.
- docs/research/2026-04-23-sky-references-crossref.md: WorldBuilder/
ACE/ACViewer/holtburger/Chorizite coverage.
- docs/research/2026-04-23-sky-dat-schema.md: full dat schema + unit
analysis.
- docs/research/2026-04-22-sky-lighting-decompile.md: prior agent's
(superseded) inference — kept for provenance.
Phase 2 will port Surface.Luminous-flag-aware per-vertex lighting for
sky submeshes once the dump resolves the open questions (Luminous-flag
distribution per Dereth sky mesh; _DAT_007a1870 scale constant value).
Build + 717 tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Retail's Region dat stores SkyObjectReplace.Luminosity / Transparent /
MaxBright as percentages in the 0..100 range. Our shader expects
fractions in 0..1. We were passing raw values (luminosity up to 100)
straight into the sky fragment shader's rgb-multiplier:
rgb = sampled.rgb * uTint.rgb * uLuminosity;
At the "Sunny" DayGroup's noon keyframes (verified via live diag),
Luminosity = 100 → shader multiplied the cloud texture RGB by 100 →
min(rgb, vec3(1.2)) clamped all channels to 1.2 → pure white sky.
Also gave the dawn/dusk purple sky effect on top of the pale texture.
Fix: SkyDescLoader.ConvertTimeOfDay divides Luminosity, Transparent
and MaxBright by 100 when loading each SkyObjectReplace. The Rotate
field stays as degrees (values like 270° are genuine headings, not
percentages).
Transparent was accidentally surviving via a 0..1 clamp downstream,
but we fix it for consistency and so brightness-attenuating values
in the 0..99 range (partial fade during dawn/dusk) work correctly
instead of rounding to full-transparent.
WorldBuilder's SkyboxRenderManager does NOT apply these fields at
all — that's why they never hit this bug. Our port applies them for
per-keyframe day-night fades, so we needed the unit conversion.
Also picked up in this commit (incidental, already running):
- Sky render: per-submesh blend mode from TranslucencyKind.Additive
for sun/moon-style self-bright objects (Additive bit 0x10000).
Luminous flag 0x40 intentionally NOT mapped to additive — that
flag is on the sky dome + cloud sheets and making them additive
produced the previous "fully white" iteration of this bug.
- ToD default seed: DayTicks/16 (Midsong = hour 9 = true noon)
instead of DayTicks*0.5 which landed on Gloaming-and-Half (sunset)
due to DerethDateTime's +7/16 day-fraction offset. Pre-TimeSync
view now correctly starts at noon.
- Lightning flash: brighter white-blue (vec3(1.5,1.5,1.8)) instead
of dim grey; ceiling relaxed during flash so the strobe actually
blows out. Cadence (strike intervals, decay) unchanged.
- Saved docs/research/2026-04-21-sky-deep-audit.md with the
decompile+ACE+ACME+WorldBuilder research done to corner this bug.
Open follow-up (not fixed here): sky clouds are white at noon /
don't get the dusk/night purple tint. Our sky shader is fully unlit
— doesn't apply sun/ambient directional light like the terrain
shader does. AmbientColor in the keyframe data carries the right
tint (purple at midnight, magenta at dusk) but we pass
uTint = Vector4.One instead of the keyframe value. Next commit will
wire directional-sun + ambient into sky.frag so cloud meshes pick
up the time-of-day color.
All 717 tests green. User-confirmed: sky colors are now "much
better" after this change (previously fully white).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two runtime blockers discovered after merging the sky/weather/lighting
branch:
1. GLSL reserved word: mesh.frag + mesh_instanced.frag used \`int active\`
as a local. On GLSL ES / some drivers \`active\` is a reserved identifier
and compile fails hard (\"ERROR: 0:38: 'active' : Reserved word\").
Renamed to \`activeLights\`.
2. SkyRenderer.EnsureMeshUploaded called DatCollection.Get<GfxObj>
without the _datLock that wraps the streaming pipeline's dat reads.
DatBinReader has shared buffer state; concurrent reads race and
throw ArgumentOutOfRangeException from Vec2Duv.Unpack deep in the
mesh parse. Wrapped both Get<GfxObj> and GfxObjMesh.Build in
try/catch and cache a null entry on failure so we don't retry every
frame and crash the render loop. Full fix would plumb _datLock into
the sky renderer, left as a TODO.
Client now stable end-to-end — in-world, spawn stream flowing,
animation + audio + sky + light UBO all live.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wire the existing LightManager + WorldTimeService state into visible
rendering. Every draw call (terrain, static mesh, instanced mesh, sky)
now shares one SceneLighting UBO at binding=1 carrying:
- 8 Light slots (Directional / Point / Spot, retail hard-cutoff)
- Ambient RGB + active light count
- Fog start/end/mode + color + lightning flash scalar
- Camera world position + day fraction
The CPU side (SceneLightingUbo in Core.Lighting) is a POD struct that
gets BufferSubData'd once per frame from GameWindow.OnRender. Shaders
read the block via `layout(std140, binding = 1) uniform SceneLighting`
— no per-program uniform uploads.
Shader changes:
- mesh.frag + mesh_instanced.frag accumulate 8 dynamic lights per
fragment using the retail no-attenuation hard-cutoff model
(r13 §10.2 / §13.1). Sun reads slot 0; spots use hard cos-cone test.
Additive lightning flash + linear fog layered on top. Saturate
clamps per-channel to 1.0.
- terrain.vert bakes AdjustPlanes sun+ambient per vertex using the
retail MIN_FACTOR = 0.08 ambient floor (r13 §7). terrain.frag adds
fog + flash on top of the baked vertex color.
- mesh.vert + mesh_instanced.vert emit vWorldPos so the fragment
stage can do per-pixel lighting against world-space positions.
- New sky.vert / sky.frag pair — unlit, scroll-UV, camera-centered,
with its own 0.1..1e6 far plane. Ports WorldBuilder's skybox.
SkyRenderer (new file in App/Rendering/Sky/) ports WorldBuilder's
SkyboxRenderManager verbatim for the C# idiom: zeroed view translation,
dedicated projection, depth mask off, iterate each visible SkyObject
in the day group, apply arc transform (Z rot for heading + Y rot for
arc sweep), feed TexVelocityX/Y as a scrolling UV offset, apply
per-keyframe SkyObjectReplace overrides (mesh swap + transparency +
luminosity) for overcast / dusk cloud variants.
GameWindow integration:
- OnLoad parses Region (0x13000000) into LoadedSkyDesc and hot-swaps
WorldTime's provider to the dat-accurate keyframes. Seeds to noon
for offline rendering. Creates the SceneLightingUboBinding and the
SkyRenderer.
- OnRender: set clear color from atmosphere fog, tick WeatherSystem,
spawn/stop rain/snow camera-local emitters on kind change, feed
sun to LightManager (zero intensity indoors — r13 §13.7), tick
LightManager against viewer pos, build + upload the UBO, draw
sky before terrain, draw terrain + static + instanced using the
shared UBO.
5 new UBO packing tests (struct sizes, slot population, 8-light cap,
directional slot 0).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Erik