acdream/docs/research/2026-04-26-sky-investigation-handoff.md
Erik 8db7a9ec28 docs(research): sky/weather investigation handoff + diagnostic tools
Captures everything learned from a long worktree iteration on the
foreground-rain bug (ISSUES.md #1 / #26) plus a new star-rendering
bug observed in the same area. The code work from that worktree
(WeatherDispatcher, EmitterDescLoader.LoadFromDat, WeatherCellRenderer,
GameWindow integration) was reverted because it didn't visibly fix
the rain bug — but the research findings + diagnostic tools are
durable and should not have to be rediscovered.

What's added:
- docs/research/2026-04-26-sky-investigation-handoff.md
  Comprehensive seed prompt for the next session. Covers:
  * Bug A: foreground rain (#26) — what's open, what's confirmed,
    what's been tried
  * Bug B: stars rendering as square in corner (NEW, user-observed)
  * 40-agent decomp scan findings — retail rain is NOT camera-
    particles, NOT server-driven, NOT screen-space; the mesh IS
    a hollow octagonal tube; only 5 weather GfxObjs in Dereth
  * Things ruled out by trial (envelope, scaling, unlit, depth-
    always alone, Setup loading)
  * Things to try next (depth+zfar combined, full render-state
    audit, frame ordering, star UV bug as easier first target)
  * Acceptance criteria for "done"

- docs/research/2026-04-26-chorizite-pr-draft.md
  Upstream PR draft for Chorizite/DatReaderWriter. Five generated
  DBObj source files reference nonexistent enum values and are
  silently excluded from the NuGet build:
  ParticleEmitterInfo, Clothing, PaletteSet, DataIdMapper,
  DualDataIdMapper. Fix: delete the duplicates. Independent of
  the rain work — benefits the AC modding ecosystem broadly.

- docs/research/2026-04-26-datreaderwriter-reference.md
  Developer reference for our DatReaderWriter usage. Version,
  types we consume, known broken types, thread-safety caveats,
  upgrade procedure, NuGet-vs-vendored decision matrix.

- tools/PesChainAudit/
  Recursive PES walker — given a 0x33xxxxxx script id, walks all
  CallPES references and dumps every hook + every referenced
  ParticleEmitter's parameters. Used to prove no weather PES
  emits rain particles.

- tools/TextureDump/
  Dumps texture pixel statistics (alpha histogram, brightness,
  max) and saves as PNG for visual inspection.

- tools/WeatherEnumerator/
  Enumerates every DayGroup in a Region, lists weather SkyObjects
  (Properties & 0x04), dumps GfxObj bounding boxes.

- tools/WeatherSetupProbe/
  Loads a Setup id, dumps each part's GfxObj + frame + scale +
  surface. Used to prove weather Setups are 5cm dummy carriers.

Worktree feature/sky-fixes is being deleted in a follow-up step.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 21:40:34 +02:00

164 lines
12 KiB
Markdown

# Sky / Weather Investigation — Handoff for Next Session
**Date:** 2026-04-26
**Author:** Claude (Opus 4.7) closing out a long worktree
**Purpose:** seed prompt for a fresh session that resumes investigation of two sky-related visual bugs without re-deriving everything we learned the hard way.
---
## What you're picking up
Two open visual bugs in the sky/weather pipeline, both observed against a live ACE server in Holtburg:
### Bug A — Foreground rain missing (ISSUES.md #26)
In retail, when the active DayGroup is "Rainy", you see rain everywhere — in the air between the camera and your character, on the horizon, overhead. Volumetric appearance.
In acdream as of `main` (post-revert), you see no falling rain at all. Even the previous (rolled-back) work that rendered the rain mesh in world space produced only a "discrete cylinder of rain" around the player, not the volumetric look retail has.
### Bug B — Stars appear as a square in one corner instead of stretched across the sky
User-observed during a recent night-time launch. The star sky-object's texture appears as a small square in one corner of the geometry instead of being mapped across the full sky dome / star plane. Suspected: UV-mapping bug, or wrong texture wrap mode (CLAMP vs REPEAT), or wrong geometry for the star object.
**Both bugs may share a root cause** in how we handle sky GfxObj UVs / texture sampling. Worth investigating B first because it might be simpler AND might reveal something that explains A too.
---
## What's on `main` right now
- The original `SkyRenderer` (camera-anchored sky pass) — unchanged.
- The original hardcoded camera-attached rain emitter (`BuildRainDesc`/`BuildSnowDesc` + `UpdateWeatherParticles`) in `GameWindow.cs` — still uses the broken `StartAlpha = 0.3 → EndAlpha = 0` linear fade. This is the alpha-fade bug from the original Issue #1 (rain fades to invisible exactly at ground level).
- ISSUES.md #1 is OPEN and #26 hasn't been filed yet on main (it lived only on the deleted `feature/sky-fixes` branch).
**This handoff doc captures everything #26 contained**, so don't worry about losing it.
---
## What we already learned (40-agent decomp scan, 2026-04-26)
**Confirmed truths about retail rain:**
1. **Retail rain is NOT camera-attached particles.** Across the entire 1.4M-line decomp at `docs/research/named-retail/acclient_2013_pseudo_c.txt`, only TWO callers of `ParticleManager::CreateParticleEmitter` exist — both are animation-hook executors (`CreateParticleHook::Execute`, `CreateBlockingParticleHook::Execute`). No camera-attached rain spawner exists in retail.
2. **Retail rain is NOT server-driven.** No inbound network message triggers rain. ACE doesn't broadcast it. Weather is entirely client-side, dat-driven from `Region.SkyInfo.DayGroups[i].SkyObjects[]`. `EnvironChangeType` (fog colors + 6 thunder sound enums) is the only weather-related broadcast and it doesn't trigger visuals.
3. **Retail rain is NOT screen-space.** No fullscreen quad, no orthographic projection, no render-target swap, no post-process. Verified via grep for `D3DXMatrixOrtho`, `SetRenderTarget`, `FullscreenQuad`, etc.
4. **The mesh IS exactly a hollow octagonal tube.** Vertex dump of `0x01004C42` and `0x01004C44`:
- 24 verts (some duplicated for UV seams)
- 16 triangles forming 8 vertical wall quads
- Octagonal corners at radius 113m around center
- 815m tall (z=0.11 to z=814.90)
- **No top/bottom caps** — purely walls
- Surface 0x080000C5 = `Base1Image | Translucent | Alpha | Additive`, SurfTranslucency=0.5
5. **Only 5 unique weather GfxObjs exist in all of Dereth's 20 DayGroups:**
- `0x01004C44` — hollow octagonal tube, TexVel=(0.02, -1.7) — direct GfxObj
- `0x01004C42` — hollow octagonal tube, TexVel=(0.02, -2.0) — direct GfxObj
- `0x02000BA6`, `0x02000588`, `0x02000589` — Setups, each containing a SHARED single 5cm dummy GfxObj (`0x010001EC`, 3 verts, 1 polygon) carrying a flash PES via DefaultScript
6. **The PES referenced by weather Setups produces ONLY flash + sound effects.** Recursive walk via `tools/PesChainAudit/` confirmed: across all 5 weather PESes (`0x33000453`, `0x33000428`, `0x3300042C`, `0x33000429`, `0x3300042D`), there is exactly ONE CreateParticleHook anywhere in the chain, and it spawns `ParticleEmitter 0x320002C2` — a 10ms-lifetime, max-2-particle, zero-extent flash. The other 4 PESes are pure SoundTweaked + CallPES recursion loops (rolling thunder cadence). Zero rain particles.
7. **The rain streak texture is `0x050016A6`** — 256x256, R8G8B8 (no alpha), 99.4% black, sparse bright streaks. Engineered for additive blending. Not the texture itself broken — works as designed for an additive sparse-streak overlay.
8. **Retail uses `DEPTHTEST_ALWAYS` and `zfar*4`** for the weather pass, set at decomp `0x00507063` and `0x00507055` in `GameSky::Draw(arg2=1)`. We tried `DEPTHTEST_ALWAYS` alone — it made the visual WORSE (rain mesh now covers everything as a wall over scene geometry). Whatever combination retail uses, it's not just depth-always.
**Things we ruled out by trying:**
- Camera-attached envelope spawn (made up r12-deepdive params; appeared as small dots near camera, not retail-like)
- Scaling the rain mesh down (turns it into an obvious cylinder around player)
- Unlit additive shader (no perceptible change)
- DEPTHTEST_ALWAYS alone (made it look worse)
- Setup loading for weather Setups (Setup parts are 5cm dummy quads, not visual)
**Things we did NOT try and might be the answer:**
- **Retail's frame ordering** — maybe the weather pass renders BEFORE the scene, not after. Worth checking the main render loop in retail (probably around `LScape::draw` or `SmartBox::Draw`).
- **Per-surface render state on the additive surface** — maybe retail enables D3DRS_ALPHATESTENABLE + D3DRS_ALPHAREF for the rain Surface specifically, discarding pure-black texels.
- **Texture sampling differences** — mipmap settings, bilinear vs nearest, anisotropy. Retail's view of the cylinder wall at 113m goes through mipmap selection that might dramatically affect appearance.
- **The DEPTHTEST_ALWAYS + zfar*4 combination together** — we only tried depth alone. Maybe the zfar change is what makes it work.
- **Multiple instances of the same mesh** — retail might spawn the rain CPhysicsObj at SEVERAL anchor positions (not just the player), creating overlapping coverage. Worth re-checking `GameSky::CreateDeletePhysicsObjects` for loop bounds.
- **The actual texture pixel content displayed in retail vs ours** — `tools/TextureDump/` already dumps the texture as PNG. Worth visually comparing what retail's rain looks like to verify the texture is correct.
---
## Probe tools available (already on main, in `tools/`)
These are one-shot diagnostic console apps. Each builds standalone with `Chorizite.DatReaderWriter` NuGet + an env var pointing at the dat dir.
- **`tools/PesChainAudit/`** — recursive PES walker. Given a starting `0x33xxxxxx` script id, walks all CallPES references and dumps every hook + every referenced ParticleEmitter's params. Used to prove no weather PES has rain particles.
- **`tools/TextureDump/`** — dumps texture pixel stats (alpha histogram, brightness mean/stddev, max) and saves as PNG for visual inspection.
- **`tools/WeatherEnumerator/`** — enumerates all DayGroups in a Region, lists every weather SkyObject (Properties & 0x04), dumps GfxObj bounding boxes.
- **`tools/WeatherSetupProbe/`** — loads a Setup id, dumps every part's GfxObj, frame, scale, and surface info. Used to prove weather Setups are 5cm dummy quads.
- **`tools/SetupProbe/`** — already in main (similar to WeatherSetupProbe, slightly different focus).
Run from `tools/<name>/` with `dotnet run -c Release`. Set `ACDREAM_DAT_DIR` if needed (defaults to `%USERPROFILE%\Documents\Asheron's Call`).
---
## Other useful research artifacts on main
- **`docs/research/2026-04-26-chorizite-pr-draft.md`** — upstream PR draft for the Chorizite/DatReaderWriter project. Five DBObj source files in their `Generated/` directory are stale duplicates that reference nonexistent enum values, causing them to be silently excluded from the published NuGet (`ParticleEmitterInfo`, `Clothing`, `PaletteSet`, `DataIdMapper`, `DualDataIdMapper`). The fix is to delete those files. **You can file this PR independent of the rain investigation — it benefits the whole AC modding ecosystem.**
- **`docs/research/2026-04-26-datreaderwriter-reference.md`** — DRW dev reference. Versioning, types we use, broken types catalog, upgrade procedure, NuGet vs vendored decision matrix.
- **`docs/research/named-retail/`** — the PDB-named retail decomp. Always grep here FIRST for any AC behavior question. Particularly relevant for sky/weather:
- `GameSky::Draw` at `0x00506ff0`
- `GameSky::CreateDeletePhysicsObjects` at `0x005073c0`
- `GameSky::MakeObject` at `0x00506ee0`
- `GameSky::UpdatePosition` at `0x00506dd0`
- `LScape::weather_enabled` at `0x0081cbe9`
- `SmartBox::EnableWeather` at `0x00451dd0`
- **`docs/research/deepdives/r12-weather-daynight.md`** — pre-decomp deep dive on weather mechanism. **Several claims in r12 turned out to be incorrect** (specifically the "rain is a camera-attached ParticleEmitter" claim was disproven by the 40-agent scan). Read it for context but don't trust the rain-mechanism details.
- **`docs/research/2026-04-23-lightning-real.md`** — corrected understanding of lightning (server-driven via opcode `0xF754 PlayScript`, not sky-walked). Relevant if you also work on Issue #2.
---
## Suggested next investigation paths (priority-ranked)
1. **Investigate Bug B first (the star bug)** — much more concrete (UV mapping, texture wrap, geometry — all easily inspectable). May reveal the same kind of issue that affects rain. Specifically:
- Identify which SkyObject in DayGroup[0] (Sunny / clear night) renders the stars
- Dump that GfxObj's vertices + UVs via the existing `tools/SetupProbe/` or extend `WeatherEnumerator/`
- Compare authored UV range to expected sky-dome coverage
- Check texture wrap mode in `SkyRenderer.cs` (we set REPEAT for scrolling, CLAMP_TO_EDGE otherwise — stars probably need REPEAT or full UV remapping)
2. **Re-grep retail decomp for the FULL render-state setup around the weather pass.** The agents found DEPTHTEST_ALWAYS but maybe missed alpha test, blend mode, fog state, sample state, etc. Specifically dump the full `GameSky::Draw(arg2=1)` body and verify EVERY D3D state call we should be replicating.
3. **Try DEPTHTEST_ALWAYS + zfar*4 TOGETHER.** We only tried depth alone. The zfar change may matter for how the cylinder's far wall projects.
4. **Investigate retail's frame ordering.** Where in retail's main render loop does `GameSky::Draw(arg2=1)` actually fire? Before or after scene meshes? If it's BEFORE, the depth test gives a totally different result than rendering after.
5. **File the Chorizite PR.** Independent of rain. Easy contribution.
---
## What "done" looks like
For Bug A (rain): standing in Holtburg during a Rainy DayGroup with `ACDREAM_DAY_GROUP=7`, rain visible IN the air between camera and character, not just on horizon. Looking up still matches retail. Walking forward, rain follows naturally.
For Bug B (stars): standing in Holtburg at night (`ACDREAM_DAY_GROUP=<night-group-index>`), stars visible as a full sky covering, NOT a small square in one corner.
---
## Suggested first prompt for the next session
```
Read docs/research/2026-04-26-sky-investigation-handoff.md.
Tackle Bug B (stars rendering as a square in one corner) first
since it's more concrete. Use tools/WeatherEnumerator/ as a model
for a new probe that dumps the star SkyObject's geometry and UVs.
Iterate to a fix, verify visually, commit.
Once stars work, return to Bug A (rain) using the priority-ranked
investigation paths in §"Suggested next investigation paths" of
the handoff doc. The 40-agent findings have ruled out enough that
the next likely answer is in retail's full render-state setup
around the weather pass — specifically things the agents may have
missed beyond DEPTHTEST_ALWAYS.
Use a worktree per CLAUDE.md, and don't ask "should I continue"
between sub-steps.
```