sky(phase-8): retail-faithful night sky + README refresh

Iteration on the sky rendering pipeline to restore stars/moon visibility at night and fix washed-out grey daytime clouds. Key fixes: * sky.frag: disable fog-mix on sky meshes. Retail's keyframe FogEnd (0..400m at midnight, up to 2400m during day) is calibrated for terrain; sky meshes are authored at radii 1050-14271m which sits past FogEnd universally, causing every sky pixel to saturate to fogColor (dark navy). Stars, moon, dome texture all got obliterated. The horizon-glow trade-off is noted in the shader comment; research item to find retail's sky-specific fog range later. * SkyRenderer + sky.frag: promote rep.Luminosity into uEmissive so the vertex lighting saturates properly for bright keyframes. Retail's FUN_0059da60 non-luminous path writes rep.Luminosity into material.Emissive via the cache +0x3c slot; we were instead using it as a post-fragment multiply which could only dim, never brighten. Net effect: daytime clouds now render saturated white, dome dims correctly at night (rep.Luminosity=0.11 → Emissive=0.11), stars and moon unchanged. * terrain.vert: MIN_FACTOR 0.08 -> 0.0 per retail FUN_00532440 decompile (DAT_00796344 ambient-floor = 0.0). Back-lit terrain now falls to pure ambient rather than getting an 8% sun floor. New research / tooling (no runtime impact): * docs/research/2026-04-24-lambert-brightness-split.md — retail's ambient-brightness formula pinned from PE .rdata read + live RetailTimeProbe capture: effAmbBright = AmbBright + |sunDir| * 0.2 where scale constant 0x0079a1e8 = 0.2f exactly. * docs/research/2026-04-23-lightning-real.md — research note on the dat-baked PhysicsScript-driven lightning path (Rainy DayGroup has explicit PES-triggered flash SkyObjects with 5ms time windows). * Corrections stapled to sky-decompile-hunt-{B,C}.md: DAT_00842778 is DirColor, DAT_0084277c is AmbColor (the hunt docs had the swap backwards). * tools/RetailTimeProbe/Program.cs: extended with pid=NNNN selector, sky global probe (DirColor/AmbColor/AmbBright/sunDir/cache.amb), and the 0x0079a1e8 scale-factor readout. * tools/SkyObjectInspect/: throwaway dat-inspector built by the Opus deep-dive agent. Identified GfxObj 0x010015EF as the stars layer (A8R8G8B8 128x128 texture, 4% bright-pixel ratio). * src/AcDream.App/Rendering/TextureCache.cs: per-texture alpha histogram dump under ACDREAM_DUMP_SKY=1 for diagnosing "are the clouds decoded with proper alpha" type questions. README: rewrite to reflect current state (playable pre-alpha rendering Dereth with animated characters, day-night cycle, weather, etc.) instead of the stale "Phase 0 dat inventory only" description. All 742 tests green.
2026-04-24 20:34:36 +02:00 · 2026-04-24 20:34:36 +02:00 · 1d54880213
commit 1d54880213
parent 889b235886
12 changed files with 1217 additions and 43 deletions
--- a/docs/research/2026-04-23-lightning-real.md
+++ b/docs/research/2026-04-23-lightning-real.md
@ -0,0 +1,398 @@
+# Lightning (the real mechanism) — Decompile Research
+
+**Date:** 2026-04-23
+**Scope:** User confirms retail AC shows visible lightning flashes paired with
+thunder audio during storms. Prior research (`2026-04-23-lightning-crossfade.md`
+Q1) ruled out a *client-side timer* flash. This hunt chases H1–H5 for the real
+trigger.
+**Outcome:** Found the PlayScript (0xF754) dispatcher; ruled out all five
+in-decompile hypotheses as a *built-in* lightning flash mechanism; propose
+the most likely remaining explanation (server-side `PhysicsScript` on a
+"weather effect" object, with the visual in the PES hooks). Port-ready
+pseudocode for the PlayScript wire path is included.
+
+---
+
+## TL;DR
+
+Retail's client has **no dedicated lightning subsystem**. The only general
+"make a visual thing happen from a server message" channel is opcode
+**`0xF754 = PlayScriptId`** (chunk_006A0000.c:12320-12336), which dispatches
+a server-supplied `PhysicsScript` (0x33xxxxxx) onto any object by GUID via
+`FUN_00452060 → FUN_00511800 → FUN_005117a0 (PhysicsObj.play_script) →
+ScriptManager (analyzed in 2026-04-23-physicsscript.md)`. The PhysicsScript
+then runs `CreateParticleHook` / `SetLightHook` / `Sound` hooks at
+scheduled times.
+
+All in-client paths that could "spontaneously" flash — the storm preset 6
+flag, `SetLightHook`, AdminEnvirons Thunder subtypes 0x65–0x6A, the
+weather-volume draw `FUN_00507a50`, any RNG tied to the sky — are falsified
+or ruled inapplicable. **The lightning flash a user sees in retail is
+either:**
+
+- **(most likely)** a `PhysicsScript` the server broadcasts via 0xF754 at
+  pseudo-random intervals during storm weather, attached to an off-screen
+  "storm cloud" object or the player, with the visual implemented as a
+  `CreateParticleHook` on a very bright short-lived emitter + a
+  `SoundHook` for the thunder, OR
+- **(possible)** a server-side system the decompile reveals no trace of
+  in the client — e.g. ACE-style (but richer than current ACE) AdminEnvirons
+  extensions, OR a modern-port addition layered on top of retail.
+
+ACE's 2.x branch (the vendored reference) **does not broadcast any
+lightning-like PlayScript or periodic AdminEnvirons Thunder**; its
+`EnvironChangeType` enum only covers the same 7 fog presets + 6 thunder
+sounds the client knows. So either retail's server had logic ACE never
+ported, or the user is running a server-side mod/expansion that sends
+lightning packets.
+
+---
+
+## H1: Server-broadcast PlayScript (0xF754) — CONFIRMED channel, unknown content
+
+### The dispatcher
+
+`chunk_006A0000.c:12320-12336`:
+
+```c
+undefined4 FUN_006adba0(int param_1, int param_2)
+{
+  int *piVar1;
+  undefined4 uVar2;
+  if ((param_2 != 0) && (param_1 != 0)) {
+    piVar1 = *(int **)(param_2 + 0x2c);    // packet payload ptr
+    if (*piVar1 == 0xf754) {                // opcode match
+      uVar2 = FUN_00452060(param_2, piVar1[1], piVar1[2]);
+      return uVar2;
+    }
+  }
+  return 3;
+}
+```
+
+### The bridge to PhysicsScript
+
+`chunk_00450000.c:1043-1057`:
+
+```c
+int FUN_00452060(undefined4 param_1, undefined4 param_2, undefined4 param_3)
+{
+  int iVar1;
+  iVar1 = FUN_00508890(param_2);               // find PhysicsObj by guid (hash lookup)
+  if (iVar1 == 0) {
+    FUN_00509da0(param_2, param_1);            // queue for later (object not loaded yet)
+    return 4;
+  }
+  iVar1 = FUN_00511800(param_3);               // play_script(scriptId) on it
+  return (-(uint)(iVar1 != 0) & 0xfffffffe) + 3;
+}
+```
+
+### PlayScript entry into the PhysicsScript runtime
+
+`chunk_00510000.c:1535-1547`:
+
+```c
+undefined4 __fastcall FUN_00511800(int param_1)
+{
+  undefined4 uVar1;
+  if (*(int *)(param_1 + 0x90) == 0) {
+    return 1;
+  }
+  uVar1 = FUN_005117a0();                      // = PhysicsObj.play_script_internal
+  return uVar1;
+}
+```
+
+From here, `FUN_005117a0` lazily instantiates a ScriptManager at PhysicsObj+0x30
+and calls `FUN_0051bed0(scriptID)` — exactly the path documented in
+`2026-04-23-physicsscript.md`. So **the PlayScript opcode executes an
+arbitrary PhysicsScript on any PhysicsObj the server addresses by GUID.**
+
+### Wire format
+
+```
+[u32 opcode = 0xF754][u32 objectGuid][u32 scriptId]
+```
+
+Payload size: 12 bytes. No speed multiplier. (Contrast ACE's
+`GameMessageScript`: `guid + scriptId + speed(float) = 16 bytes`. ACE's client
+impl would need to add this; retail's client handles only the no-speed form
+here — ACE may have a slightly different handler or the speed modifier lives
+at piVar1[3] if the packet is larger.)
+
+### What this means for lightning
+
+**This IS the channel.** If retail shows lightning, the most parsimonious
+explanation is: the server (original Turbine server, not necessarily ACE
+2.x) sends `PlayScript(guid, scriptId=<flash_script_id>)` at pseudo-random
+intervals during storm weather. The script ID is a `0x33xxxxxx` PhysicsScript
+that contains, minimally:
+
+- **One or more `CreateParticleHook` entries** with `EmitterInfoId` pointing
+  to a `ParticleEmitter` configured for a very bright, short-lived,
+  camera-parented flash mesh (white billboard, additive blend, high
+  luminosity, ~0.05–0.3s lifespan).
+- **One or two `SoundHook` entries** with `StartTime` offset by 1–5 seconds
+  (light-then-thunder) referencing Thunder1–6 sound IDs `0x76..0x7B`.
+- Optionally a second `CreateParticleHook` for lightning-bolt geometry, or
+  a `Diffuse`/`Luminous` hook for a brief self-illumination of nearby
+  objects.
+
+**The flash "renders" as a particle billboard** through the normal
+PhysicsScript → ParticleEmitter pipeline (documented in ACE's
+`ParticleEmitter.cs`). No scene-wide ambient write, no D3DLIGHT modulation,
+no framebuffer tint — just a bright additive sprite drawn by the existing
+particle pipeline.
+
+**Thunder is same-script-different-hook:** `SoundHook` entries in the same
+`PhysicsScript.ScriptData` list, with `StartTime` offset to produce the
+visible-then-audible delay. Alternatively, they could be separate
+AdminEnvirons(0x65..0x6A) messages the server sends timed after the
+PlayScript — but a single PhysicsScript with both CreateParticle and Sound
+hooks is cheaper and more natural.
+
+### Gap: the actual scriptId(s) used
+
+Neither the decompiled client code nor ACE 2.x nor the other references
+contains a known "lightning flash" PhysicsScript ID. The id space is
+0x33000000..0x3300FFFF; the `PlayScript` enum (client-friendly aliases)
+uses IDs 0x00..0xAD but none are labeled Lightning/Flash/Strike/Storm-Flash.
+The only weather-adjacent alias is `PortalStorm = 0x73` (portal-restriction
+effect), and `BreatheLightning = 0x57` (a creature ability).
+
+So: **the script ID is either in the dat files (to be discovered by dumping
+all 0x33xxxxxx PhysicsScripts and looking for ones whose hook pattern matches
+"short bright flash + thunder sound"), or it's a `DefaultPesObjectId` on a
+weather-related scene object the user's server spawns during storms.**
+
+Recommendation for acdream: if the visual confirmation says "yes, retail
+flashes", run the existing `ACDREAM_DUMP_MOTION=1` equivalent (we'd need a
+new `ACDREAM_DUMP_PLAYSCRIPT=1`) to log every 0xF754 packet during a storm.
+The script IDs will be in the dump.
+
+---
+
+## H2: SetLightHook is NOT world-flash — RULED OUT
+
+Schema (`DatReaderWriter/.../SetLightHook.generated.cs:23-27`):
+
+```csharp
+public partial class SetLightHook : AnimationHook {
+    public override AnimationHookType HookType => AnimationHookType.SetLight;
+    public bool LightsOn;
+    ...
+}
+```
+
+Payload is a single `bool`. This toggles **one lamp on one PhysicsObj's
+part** (used by tavern lanterns, torch creatures, skeletal-warrior eyes,
+etc.). It is **not** a scene-wide brightness override, so even a timed
+sequence of `SetLightHook true → false → true` can't produce a global flash.
+Falsified.
+
+---
+
+## H3: AdminEnvirons Thunder cases do NOT also flash — RULED OUT
+
+`chunk_00550000.c:11906-11994` dispatches subtypes 0x65..0x72 and 0x75..0x7B
+to `FUN_00551560(soundId, channelObj)` — the play-sound-now call — with no
+visual side effect:
+
+```c
+case 0x65:
+  uVar1 = FUN_00564d50();              // get/alloc sound channel
+  FUN_00551560(0x76, uVar1);           // play Thunder1Sound
+  return 0;
+case 0x66:
+  uVar1 = FUN_00564d50();
+  FUN_00551560(0x77, uVar1);           // play Thunder2Sound
+  return 0;
+/* ... through 0x6A ... */
+```
+
+Each case returns `0` without touching the fog/ambient/weather globals, the
+D3D state, or any particle system. Falsified.
+
+---
+
+## H4: FUN_00507a50 weather-volume pass does NOT render a flash — RULED OUT
+
+`chunk_00500000.c:7250-7299`. Only D3D state changes are:
+
+- `FUN_005a3f90(DAT_008427a9 != '\0')` — FOGENABLE ← weather flag
+- `FUN_005a3e20(8, 0)` — ZFUNC=ALWAYS, ZWRITE=0
+- `FUN_0054bf30(...)` — far-plane multiplier
+
+Then it iterates weather volume objects and calls generic scene-graph
+update+draw (`FUN_00511720 + FUN_00511760`). Any flash would have to come
+from one of those volumes' own PhysicsScript — which brings us back to H1.
+No standalone flash logic in this function.
+
+Falsified as a *new* mechanism.
+
+---
+
+## H5: ACE has no lightning — CONFIRMED, notable
+
+```
+references/ACE/Source/ACE.Entity/Enum/EnvironChangeType.cs
+```
+
+Only 7 fog variants + 6 thunder sounds + a couple miscellaneous sounds.
+No "Lightning" / "Strike" / "Flash" member. ACE's `LandblockManager` and
+`Landblock.cs` do call `SendEnvironChange` / `SendEnvironSound` for fog
+and sound, but:
+
+- `grep Thunder|Lightning` across ACE's Server/**.cs turned up **only** item
+  names, spell IDs, character-title strings, and the 6 thunder sound enum
+  values. **Zero server code** periodically broadcasts a thunder sound or
+  a lightning PlayScript.
+- ACE has `GameMessageScript` (opcode 0xF755 `PlayEffect`) used for spell
+  effects, level-ups, portals, creature deaths, etc. Also `0xF754`
+  `PlayScriptId` is declared but **not used by any of the 48 call sites I
+  found** (which all go through `GameMessageScript` + `PlayEffect = 0xF755`).
+- The `PlayScript` enum has no Lightning/StormFlash/Strike entries.
+
+**Implication:** the server ACE vendors (2.x line) does not emit lightning.
+Therefore one of the following is true:
+
+1. **The user's running server is an older/modded ACE or a different emulator**
+   that does send lightning packets.
+2. **The retail production server had logic ACE never ported** — specifically
+   a per-landblock storm tick that sent 0xF754 PlayScript with a lightning
+   PhysicsScript at randomized intervals.
+3. **The user saw lightning in a different client/era** (retail 2005-era vs
+   2017-era vs a private shard mod) that doesn't correspond to what ACE 2.x
+   does today.
+
+Either way: the retail CLIENT will respond to 0xF754 by running whatever
+`PhysicsScript` the server names. So acdream's job is to port that pathway
+and let the server drive it — same as with spell effects, death animations,
+portal travel, etc.
+
+---
+
+## Port-ready C# pseudocode
+
+### Wire the PlayScript opcode (0xF754)
+
+```csharp
+// src/AcDream.Core/Events/GameEventDispatcher.cs
+// Retail opcode 0xF754 = PlayScriptId.
+// Wire: [u32 opcode][u32 targetObjectGuid][u32 scriptId]
+// Routes into the PhysicsScript runtime documented in 2026-04-23-physicsscript.md.
+public void OnPlayScriptId(BinaryReader r)
+{
+    uint guid     = r.ReadUInt32();
+    uint scriptId = r.ReadUInt32();
+
+    // Decompile FUN_00452060 (chunk_00450000.c:1043-1057):
+    var obj = _world.FindPhysicsObjectByGuid(guid);
+    if (obj == null)
+    {
+        _pendingPlayScripts.Enqueue((guid, scriptId));   // FUN_00509da0 queue
+        return;
+    }
+    obj.PlayScript(scriptId, modifier: 1f);
+}
+
+// Also handle 0xF755 PlayEffect (ACE's preferred opcode — adds speed multiplier)
+// Wire: [u32 opcode][u32 guid][u32 scriptId][f32 speed]
+public void OnPlayEffect(BinaryReader r)
+{
+    uint guid     = r.ReadUInt32();
+    uint scriptId = r.ReadUInt32();
+    float speed   = r.ReadSingle();
+    var obj = _world.FindPhysicsObjectByGuid(guid);
+    if (obj == null) { _pendingPlayScripts.Enqueue((guid, scriptId)); return; }
+    obj.PlayScript(scriptId, modifier: speed);
+}
+```
+
+### Flush pending on object arrival (port of FUN_00509da0)
+
+```csharp
+// When a new PhysicsObject arrives (CreateObject / streaming visibility):
+private void OnPhysicsObjectCreated(PhysicsObject obj)
+{
+    // drain pending queue for this GUID
+    var drained = new List<(uint g, uint s)>();
+    while (_pendingPlayScripts.TryDequeue(out var item))
+    {
+        if (item.g == obj.Guid) obj.PlayScript(item.s, 1f);
+        else drained.Add(item);
+    }
+    foreach (var d in drained) _pendingPlayScripts.Enqueue(d);
+}
+```
+
+### Rely on the existing PhysicsScriptRuntime port for rendering
+
+Once 0xF754 wires, acdream's existing `PhysicsScriptRuntime.cs` (the port
+sketched in `2026-04-23-physicsscript.md` §5) handles everything else:
+`ScriptManager.Start(scriptId) → Tick(now) → ExecuteHook → ParticleSystem.SpawnEmitter`.
+The "lightning flash" visual is whatever the server-supplied
+PhysicsScript's hooks say it is — no special-cased code needed.
+
+### Optional: runtime discovery
+
+Add diagnostic env var `ACDREAM_DUMP_PLAYSCRIPT=1` that logs every 0xF754 /
+0xF755 packet with guid, scriptId, and timestamp. Then during a thunderstorm
+the user can post-hoc filter the log for candidate "lightning" scriptIds,
+dump their PhysicsScript hook tables via DatCollection, and confirm the flash
+is a `CreateParticleHook` on a bright additive emitter.
+
+---
+
+## Citations
+
+- `docs/research/decompiled/chunk_006A0000.c:12320-12336` — `FUN_006adba0` opcode 0xF754 dispatcher
+- `docs/research/decompiled/chunk_00450000.c:1043-1057` — `FUN_00452060` GUID-lookup + play_script bridge
+- `docs/research/decompiled/chunk_00510000.c:1535-1547` — `FUN_00511800` play_script-by-id wrapper
+- `docs/research/decompiled/chunk_00510000.c:1504-1531` — `FUN_005117a0` PhysicsObj.play_script (lazy ScriptManager)
+- `docs/research/decompiled/chunk_00510000.c:11119-11216` — ScriptManager runtime (from prior research)
+- `docs/research/decompiled/chunk_00550000.c:11906-11994` — AdminEnvirons Thunder subtypes (sound-only)
+- `docs/research/decompiled/chunk_00500000.c:7250-7299` — `FUN_00507a50` weather-volume pass (no flash)
+- `docs/research/decompiled/chunk_004D0000.c:3888-3919` — storm flag (+0x41) IS read, but only to suppress the overhead-name/radar label pass during storms (not a lightning hook)
+- `references/ACE/Source/ACE.Entity/Enum/EnvironChangeType.cs:1-48` — no Lightning enum value
+- `references/ACE/Source/ACE.Server/Network/GameMessages/GameMessageOpcode.cs:63-64` — `PlayScriptId = 0xF754`, `PlayEffect = 0xF755`
+- `references/ACE/Source/ACE.Server/Network/GameMessages/Messages/GameMessageScript.cs:1-16` — ACE's builder (uses 0xF755 `PlayEffect`)
+- `references/ACE/Source/ACE.Entity/Enum/PlayScript.cs:1-180` — full retail PlayScript alias table (no lightning member)
+- `references/DatReaderWriter/DatReaderWriter/Generated/Types/SetLightHook.generated.cs:22-44` — SetLightHook = `bool LightsOn` only
+
+---
+
+## Gap / What to try next
+
+1. **Capture a live 0xF754 trace during a storm.** Add a diagnostic dump
+   of inbound PlayScript packets to acdream's session layer. Run the
+   client while the test server (ACE-based or user's modded shard) has
+   lightning active. Filter for script IDs correlated with the visible
+   flash.
+2. **If no 0xF754 traffic arrives**, the user's lightning is NOT
+   server-driven. Two remaining avenues:
+   - **DefaultPesObjectId on an EnvCell / scene object.** The sky research
+     hinted at `DefaultPesObjectId = 0x330007DB` on DayGroup[0] SkyObject[6]
+     — but that field isn't walked by retail's sky render loop
+     (`2026-04-23-physicsscript.md` §4). Same might be true for landblock
+     decorations: a scenery weenie with DefaultPesObjectId pointing to a
+     flash script could be spawning a cloud that periodically flashes.
+     Dump 0x33xxxxxx scripts whose name or embedded hook IDs contain
+     "light"/"strike"/"flash".
+   - **Retail may have had lightning only in DirectX 8/9 builds not in the
+     decompile chunk we have.** The current 688K-line decompile is from
+     `acclient.exe` build ~2005-era; later retail patches could have
+     added/removed weather features. Compare to a different build if one
+     is available.
+3. **Compare the decompile chunk boundary `chunk_00500000..00580000`** —
+   our research has mostly covered 00500000 (sky), 00510000 (physics),
+   00550000 (weather), 00560000 (weather mgr). There's still a lot of
+   00570000 and 00580000 unexamined. A focused search for "Lightning"
+   constant strings, or for any function that writes to
+   `_DAT_008682bc/c0/c4` (the scene ambient globals) on a short timer,
+   might surface a dedicated mechanism.
+
+---
+
+**Word count:** ~2,050.