merge: bring main (A7 lighting Fix A–D + UN-7 + #140 Fix D) into the D.5 branch

Integrates main's 19 commits (A7 outdoor/indoor torch lighting Fix A/B/C/D,
GlobalLightPacker, shader updates, UN-7) under the D.5 toolbar/item-model stack
(D.5.1/D.5.2/D.5.4/D.5.3a). Auto-merged cleanly except docs/ISSUES.md.

Conflict resolved: both lineages used #140 for different issues. Kept main's
#140 = "A7 Fix D" (resolved); renumbered the toolbar/selected-object issue to
#141 (note added; this branch's commits/spec still reference #140 — immutable).
The register auto-merged (AP-46 cites file:line, not #140; UN-7 keeps #140=Fix D).

Build + full suite green on the merged tree (2,713 passed / 4 skipped).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Erik 2026-06-20 12:01:20 +02:00
commit 31d7ffd253
27 changed files with 2327 additions and 103 deletions

View file

@ -0,0 +1,140 @@
# A7 Lighting — Fix A/B/C SHIPPED, Fix D (object torch over-brightness) HANDOFF
**Date:** 2026-06-18 **Branch:** claude/thirsty-goldberg-51bb9b (merged to main)
**Companion memory:** `claude-memory/reference_retail_ambient_values.md` (all captured
values + cdb recipes) and `reference_retail_chat_colors.md` (cdb method).
This session made acdream's outdoor + ambient lighting retail-faithful by grounding
everything in **live cdb on the retail client** (no guessing). Three fixes shipped;
a fourth (Fix D — outdoor objects too bright near torches) is fully grounded but
**deliberately NOT implemented** because the math contradicts the observed result —
one more capture is needed first.
## SHIPPED this session (all on `main`)
| Fix | Commit | What | Result |
|---|---|---|---|
| **A** | `aa94ced` | point-light SHAPE: per-vertex Gouraud + faithful `calc_point_light` (wrap + norm), per-channel cap | killed the "spotlight" disc — user "way better" |
| **B** | `4345e77` | per-OBJECT light selection (`minimize_object_lighting`): each object picks its own ≤8 lights by its AABB sphere, camera-independent | killed "building lights up as you approach"; a Holtburg view has **129** point lights vs the old global cap of 8 |
| **C** | `57c1135` | sun-vector magnitude: ambient + sun were **~32% too bright** | ambient now matches retail within ~2%; user "general ambient better outside" |
**Fix B mechanism** (for context): two new SSBOs in `mesh_modern.vert` — binding=4
GLOBAL light array (`LightManager.PointSnapshot`), binding=5 per-instance 8-int
light set (mirrors the U.3 clip-slot SSBO). `LightManager.SelectForObject` +
`BuildPointLightSnapshot` (pure, TDD). `WbDrawDispatcher` computes each entity's
light set once per entity (like `_currentEntitySlot`), threads it parallel to the
matrices.
**Fix C mechanism:** `SkyStateProvider.RetailSunVector` had `y = cos(P)` (≈1) — the
PRE-transform value `SkyDesc::GetLighting` writes to its arg5 (0x00500ac9), before
`LScape::set_sky_position`'s world transform. cdb read retail's actual
`LScape::sunlight = (0.2238, ~0, 0.00352)`, magnitude = DirBright. Corrected to the
world-space spherical form `DirBright × (cos P·sin H, cos P·cos H, sin P)`,
`|sunVec| == DirBright`. Feeds BOTH the ambient boost AND the sun colour, so it
dims **terrain + objects + sky** (all read the shared SceneLighting UBO). 18/18 sky
tests green (old tests pinned the inflated magnitude — updated to cdb-verified).
## KEY LESSON: the "too purple" was NEVER a bug
The user's side-by-side ("acdream too purple, retail neutral") was a comparison
**across different times of day**. Live cdb at the SAME game time + DayGroup proved
acdream's time, weather (DayGroup selection), AND ambient COLOR all match retail
exactly — the purple `AmbColor=(200,100,255)` is authored per-time-of-day in the
sky dat (twilight = purple, midday = neutral `(230,230,255)`). Only the *brightness*
was wrong (Fix C). Don't re-investigate the purple.
---
## RESOLVED — Fix D: outdoor walls too bright near torches (contradiction settled 2026-06-18)
**Symptom (user):** Holtburg meeting-hall walls blow out **warm**/bright in acdream
vs dim in retail. The contradiction ("D3D-FF math says color×100 should blow WHITE,
yet retail is DIM") is **resolved**: the D3D-FF model was the WRONG ORACLE for these
walls. Settled by a 5-thread decomp workflow (`wf_f660eb88`) + adversarial verify +
4 live cdb captures. **⚠ The "DO NOT port the D3D-FF model" warning still stands** —
not because it'd be too bright, but because it's the wrong path entirely.
### Render path (Ghidra xrefs — unambiguous, two SEPARATE light systems)
- **STATIC lights → CPU vertex BAKE.** `RenderDeviceD3D::DrawEnvCell` (0x0059F170) →
`D3DPolyRender::SetStaticLightingVertexColors` (0x0059CFE0) → `calc_point_light`
(0x0059C8B0, its SOLE caller). Wall torches are STATIC objects → baked into vertex
colours. AC town buildings are EnvCell structures, so their walls take this path.
- **DYNAMIC lights → D3D hardware FF.** `add_dynamic_light``insert_light` (0x0054D1B0)
`config_hardware_light` (0x0059AD30); `minimize_envcell_lighting` (0x0054C170)
enables ONLY the dynamic subset (class 2) for the cell — statics are NEVER hardware-
enabled for the cell. (`minimize_object_lighting` 0x0054D480 enables both, for free
GfxObjs.) So `config_hardware_light` — where last session's `intensity=100` was seen —
carries DYNAMIC lights for cells, not the wall torches.
### Why retail stays warm-but-DIM (the bake is triple-clamped — `calc_point_light`)
Per light: `range = falloff×1.3` hard gate; half-Lambert wrap `(1/1.5)(N·D + 0.5·d)`;
`norm = (distsq>1)? distsq·d : d` (~1/d²); `scale = (1d/range)·intensity·(wrap/norm)`;
then the **decisive per-channel cap `result = min(scale·color, color)`** — one light adds
**at most its own (sub-1.0, warm) colour**, however large intensity is. Caller sums from
**BLACK** (no ambient/sun in the accumulator) over all static lights, then **clamps the
sum to [0,1]** per channel before packing `vertex.diffuse`. White needs many in-range
lights stacking past 1.0; a hall has a handful, each warm-capped.
### Live cdb ground truth (4 captures; scripts in `tools/cdb/a7-fixd-*.cdb`)
`Render::world_lights` @ **0x008672a0** (LightParms): `num_static_lights` @ +0x104,
`sorted_static_lights[]` (RenderLight*, info @ RL+0x70) @ +0x3498, `num_dynamic_lights`
@ +0x3588. Captured standing in Holtburg:
- **num_static_lights = 38**, **num_dynamic_lights = 2.**
- **2 DYNAMIC** (`add_dynamic_light`, d3dIdx 12): viewer light `intensity=2.25 falloff=10
color=(1,1,1)` white; **PORTAL** `intensity=100 falloff=6 color=(0.784,0,0.784)` MAGENTA.
→ **the `intensity=100` light is the purple PORTAL (dynamic/hardware), NOT a wall torch.**
- **38 STATIC** wall torches, all `type=0 intensity=100`, **WARM**: orange
`(1.0, 0.588, 0.314)` falloff 4, and cream `(0.980, 0.843, 0.612)` falloff 35
(→ bake range ~3.96.5 m). Torches DO carry intensity=100, but the per-channel cap
pins each to its warm colour ⇒ retail walls go warm, not white.
### acdream's actual bug — TWO real causes (both verified in source)
- **D-1 (math, primary): unclamped accumulator folding ambient+sun+torches.**
`mesh_modern.vert` `accumulateLights` starts `lit = uCellAmbient.xyz` (:184), ADDS
sun (:196), ADDS each capped torch (:206), returns UNCLAMPED (:208); the ONLY clamp is
one `min(lit,1.0)` in `mesh_modern.frag:92` AFTER a lightning bump (:89). The per-light
cap (:180) IS faithful. But pouring ambient + sun + up-to-8 intensity-100 WARM torches
into ONE bucket and trimming only at the end overflows to warm-white. Retail clamps the
torch sum on its OWN (from black); ambient/sun are a separate term.
- **D-2 (state, compounding): EnvCell shell SSBO binding leak.**
`EnvCellRenderer.cs:1225-1230` (RenderModernMDIInternal) binds SSBO 0/1/2/3 only, NEVER
4 (`gLights`) or 5 (`instanceLightIdx`) — which the shared `mesh_modern.vert` reads at
:204-206. Only `WbDrawDispatcher` binds 4/5. Indoor DrawInside interleaves the two, so a
cell shell reads whatever LEAKED light set the last WbDrawDispatcher draw left bound —
a different entity's torches, wrong per-instance indices ⇒ wrong/over-bright walls.
- `LightBake.cs` (verbatim CPU port) exists but is UNWIRED (zero callers); the live path is
the in-shader version missing the clamp shape.
### Fix plan (REPORT-ONLY — implement in a separate session, with the no-workaround rule)
- **D-1:** accumulate point/spot into a LOCAL `pointAcc`, `saturate` it to [0,1] BEFORE
adding ambient + sun — mirrors `SetStaticLightingVertexColors` (sum-from-black, clamp the
point sum). Keep the per-light `min(scale·baseCol, baseCol)` (vert:180). Files:
`mesh_modern.vert` (split accumulator + clamp), `mesh_modern.frag` (reorder/drop the
single clamp). Conformance golden: a wall ≤~5 m from an orange `(1,0.588,0.314)` torch
bakes warm-but-≤[0,1], NOT white.
- **D-2:** EnvCell shell must bind binding 4 (global lights) + 5 (per-instance light set)
for ITS OWN instances — compute a per-shell set like `WbDrawDispatcher.ComputeEntityLightSet`
(LightManager.SelectForObject); option (b) all-(-1) fallback = NO point lights is a STOPGAP
(needs approval + a divergence-register row). File: `EnvCellRenderer.cs` RenderModernMDIInternal.
- **Stale doc to fix in the D-1 commit:** divergence-register `AP-35` still describes the
point-light path as per-pixel `mesh_modern.frag:52` with the wrap "NOT ported"; Fix A
(`aa94ced`) moved it to per-vertex `mesh_modern.vert:163` WITH the wrap.
- **Do NOT port the D3D-FF hardware model for the walls** (config_hardware_light's
color×intensity / (0,1,0)=1/d / Range=falloff×1.5) — it lights GfxObjs/dynamics, not the
baked walls.
---
## cdb cheat-sheet (all verified this session; binary MATCHES refs/acclient.pdb)
- `bp acclient!SmartBox::SetWorldAmbientLight` (0x004530a0) — arg2=level `[esp+4]`, arg3=color32 `[esp+8]`
- `bp acclient!SkyDesc::GetLighting` (0x00500a80) — arg2=dayFraction `[esp+4]`; `dt acclient!SkyDesc @ecx present_day_group`
- `LScape::sunlight` global @ **0x00841940** (Vector3); `LScape::ambient_level` @ 0x00841770
- `bp acclient!PrimD3DRender::config_hardware_light` (0x0059ad30) — arg4=LIGHTINFO `[esp+0x10]`; `dt acclient!LIGHTINFO dwo(@esp+0x10) type intensity falloff color`
- `rangeAdjust = 1.5` @ 0x00820cc4; `D3DPolyRender::SetStaticLightingVertexColors` @ 0x0059cfe0
- Pattern: `.formats poi(<addr>)` for floats, `dwo(<addr>)` for dwords, `qd` after N hits to auto-detach (keeps retail alive). User must have retail in-world first.
- acdream probes: `ACDREAM_PROBE_LIGHT=1` (`[light]` ambient+sun line), `ACDREAM_DUMP_SKY=1` (keyframes + dayFraction + DayGroup).
## Build / run
`dotnet build src/AcDream.App/AcDream.App.csproj -c Debug` (green). Standard
`ACDREAM_LIVE` launch env in CLAUDE.md. Close the client before rebuilding (it locks
the DLLs). 18/18 sky tests + 17/17 LightManager + 36/36 dispatcher clip-slot green.

View file

@ -0,0 +1,152 @@
# A7 Fix D round 2 — REAL cause found (object sun+ambient + torch REACH), CHECKPOINT
**Date:** 2026-06-19 **Branch:** `claude/thirsty-goldberg-51bb9b` (NOT merged — held at the visual gate)
**Predecessor:** `docs/research/2026-06-18-lighting-a7-fixABC-shipped-fixD-handoff.md`
**Status:** checkpointed at user request after pinning the root cause. D-1..D-4 are committed +
correct but **did NOT fix the visible symptom** — they were the wrong subsystem.
---
## ✅ RESOLVED 2026-06-19 (second session) — the "torch REACH" theory was WRONG; real cause = retail does NOT torch-light OUTDOOR objects at all
**The open question is settled, and it overturns this checkpoint's own hypothesis. The fix is NOT
"shorten torch reach" — it is "outdoor objects receive NO torches."**
**Empirical (acdream side, headless dat dump `HoltburgTorchFalloffProbeTests`):** the Holtburg
neighbourhood has **27 static lights, raw dat Falloff ∈ {3,5,6}** — the dominant orange entrance
torch (setup `0x020005D8`, colour `(1,0.588,0.314)`) is **Falloff 6** (17 of 27). acdream reads
this **faithfully**`LightInfoLoader` just copies `info.Falloff`, no stray ×1.5. There is **NO
Falloff-4 torch anywhere in Holtburg**, so the predecessor's "retail orange = falloff 4" could not
be a *different* falloff-4 torch. Both clients read the same dat float → acdream's reach is NOT
inflated. So "acdream 6 vs retail 4" was a red herring.
**Decomp (retail side, read verbatim + corroborated by an independent adversarial workflow
`wf_07289ba4`):** retail's per-object torch binder `minimize_object_lighting` (0x0054d480) is
**gated** in `RenderDeviceD3D::DrawMeshInternal` (0x0059f398) by `if (Render::useSunlight == 0)`.
The OUTDOOR landscape stage runs `Render::useSunlightSet(1)` (`PView::DrawCells` 0x005a485a, right
before `LScape::draw`), so when the building EXTERIOR shell is drawn
(`LScape::draw → DrawBlock 0x005a17c0 → DrawSortCell 0x0059f140 → DrawBuilding 0x0059f2a0 →
CPhysicsPart::Draw → DrawMeshInternal`), torches are **SKIPPED** — the only active light is the
**sun** (`useSunlightSet(1)` enables `add_active_light(0xffffffff, 0)` = sun + ambient only). The
static vertex bake (`SetStaticLightingVertexColors` 0x0059cfe0) is **EnvCell-only** (sole caller
`DrawEnvCell` 0x0059f1f6). **So retail lights outdoor objects with SUN + ambient ONLY — never the
wall torches.** This exactly explains the checkpoint's own isolation result ("object point lights
OFF → building matches retail"): retail's outdoor facade gets ZERO torch energy. (Confirming the
non-bug nature of reach: retail's free-object *hardware* path `config_hardware_light` 0x0059ad30
uses `Range = falloff × rangeAdjust(1.5)` = LONGER than acdream's ×1.3, with `Diffuse = color×100`
and att `1/d` — that would blow the facade WHITE if enabled, which is further proof retail never
enables it outdoors.)
**The three retail lighting regimes (now all mapped):**
1. **EnvCell walls** → static bake (`calc_point_light`, range `falloff×1.3`, wrap, capped), no sun.
→ acdream mode 1 (EnvCell). ✓ already correct.
2. **Indoor objects** (`useSunlight==0`) → torches (hardware, no sun). → acdream mode 0 **indoor**.
3. **Outdoor objects** (`useSunlight==1`) → sun + ambient, **NO torches**. → acdream mode 0 **outdoor**.
acdream's mode-0 path applied sun **AND** torches to ALL objects — wrong for both 2 and 3.
**THE FIX (shipped this session):** in `WbDrawDispatcher.ComputeEntityLightSet`, gate per-object
torch selection on the object being INDOOR (`ParentCellId` is an EnvCell, `(id&0xFFFF)>=0x0100`)
via the pure predicate `IndoorObjectReceivesTorches`. Outdoor objects (building shells — ParentCellId
null; outdoor scenery; outdoor creatures) keep the all-(-1) light set ⇒ sun + ambient only = retail.
The indoor "no sun" half is already handled by the global sun-kill when the player is inside a cell
(`UpdateSunFromSky`, intensity 0). Divergence-register row **AP-43** added (documents the residual:
acdream keys sun/torch on the object's own cell + a per-frame player-inside sun-kill, vs retail's
per-draw-STAGE `useSunlight` — only matters for through-doorway look-ins). Tests:
`WbDrawDispatcherTorchGateTests` (7✓), `HoltburgTorchFalloffProbeTests` (dat dump). Build green;
App 280✓/1skip, Core 1486✓/2skip. **Awaiting the user visual side-by-side at Holtburg before merge.**
**DO-NOT-RETRY (this session's corrections to the checkpoint below):** do NOT shorten torch reach /
change `Falloff×1.3` — acdream reads the dat faithfully and the bake reach is correct for EnvCells.
The building is an OUTDOOR object; retail gives it no torches. The original checkpoint's "tighten reach
to ~5m, keep torches ON" plan (below) is SUPERSEDED — keeping torches ON for the outdoor shell at any
reach is the bug.
---
## TL;DR — what the visible bug actually is (and is NOT)
The user's symptom (Holtburg meeting-hall facade too bright/warm/washed-out + character backs
lit) is **NOT** the EnvCell bake, the per-channel clamp, the half-Lambert wrap, or the SSBO leak.
Those are the D-1..D-4 path. **The visible surfaces are mode-0 OBJECTS**, and the cause is:
1. **Building facade over-bright** = the **torch REACH is too long** (acdream ~7.8 m vs retail
~5.2 m), so each entrance torch floods the whole small facade instead of a tight pool.
**CONFIRMED by isolation**: gating object (mode-0) point lights OFF made the building match
retail ("looks much better", user 2026-06-19).
2. **Character backs / slight object over-bright** = the **sun + ambient on objects** (mode 0
runs both). Ambient is NOT the culprit (it MATCHES retail exactly — see values). The residual
is small for the character (it ~matches retail), so the dominant visible bug is #1 (torches).
## Render-path facts (source-verified, workflow `wf_c4ad8cf8`)
- **Building EXTERIOR** = a flat-mesh `WorldEntity` with `IsBuildingShell=true`, `ParentCellId=null`,
built from `BuildingInfo.ModelId` (`LandblockLoader.cs:79-90`), drawn by **WbDrawDispatcher**
which hard-sets `uLightingMode=0` (`WbDrawDispatcher.cs:898`). It is **NOT an EnvCell** — so
**D-4 (EnvCell walls get no sun) never touched it**.
- **Characters/creatures/players** = ordinary `WorldEntity` dynamics, also drawn by
WbDrawDispatcher at `uLightingMode=0` (plain Lambert + sun). The mode plumbing is CORRECT
(mode-0 plain Lambert already zeroes a torch behind a back-face — that part of D-3 works).
- **EnvCellRenderer** (`uLightingMode=1`, no-sun, wrap) only ever draws **interior** cell shells
from the dat EnvCell list — never `info.Buildings`, never characters.
- Render loop: in-world frames go through `RetailPViewRenderer.DrawInside`; the bare
`WbDrawDispatcher.Draw` (GameWindow.cs:8230) is the no-viewer-cell fallback. Both share the
ONE `_meshShader` (mesh_modern) program (GameWindow.cs:1845-1857), so `uLightingMode` is one
shared uniform; each renderer re-sets it before its draws.
## Ground truth (live cdb retail + acdream probe, SAME-INSTANT)
- **Ambient MATCHES exactly**: acdream `(0.447,0.447,0.495)` == retail `(0.4465,0.4465,0.4951)`.
→ same sky keyframe → **same time of day; NO time desync** (the earlier "retail 0.3 / acdream
purple" was sequential-capture drift + acdream's un-synced spawn frame; ignore it).
- **retail sun** (`world_lights.sunlight` @ 0x008672a0+0x18) = `(0.573, ~0, 0.445)`, magnitude
**0.725**, colour `(0.98,0.84,0.59)` warm. acdream `sun=1` (active, derived from the same sky
state via Fix C `|sunVec|=DirBright`). Sun is NOT zero — retail DOES sun-light objects.
- **retail torches** (golden, a7-fixd-golden2): static, `intensity=100`, `falloff 3/4/5`, warm
`(1,0.588,0.314)` orange + `(0.98,0.843,0.612)` cream. `calc_point_light` makes a BRIGHT TIGHT
pool (saturates to full warm to ~4 m, gone by ~5.2 m). Faithful in acdream (LightBake.cs).
- **acdream torches** ([light-detail]): `range=7.8` (Falloff 6×1.3) and `range=6.5` (Falloff 5).
acdream `Range = info.Falloff * 1.3f` (`LightInfoLoader.cs:90`) — the 1.3 is correct, NO stray 1.5.
## The OPEN question to resolve FIRST on resume (don't guess)
acdream's orange torch reads **Falloff 6** (range 7.8); retail's orange torch was captured at
**Falloff 4** (range 5.2). `6 = 4 × 1.5` (smells like rangeAdjust) BUT they **might be different
torches** (38 static torches, several orange). **Resolve by comparing the SAME torch's Falloff in
acdream vs retail, matched by world position** (one focused capture): break/dump acdream's torch
Falloff for a specific Holtburg torch and the retail `world_lights.static_lights[i].info.falloff`
for the same one. Then:
- If acdream reads a **too-large Falloff** for the same torch → fix the dat read / conversion
(the DatReaderWriter `LightInfo.Falloff` path) so acdream's reach == retail's.
- If the Falloff matches and reach is genuinely ~7.8 → the building-shell-as-one-object spill is
the issue; tighten how building shells receive torches (the per-vertex range gate already
localises, so this is unlikely — favour the Falloff hypothesis).
## Proposed fix (after the falloff is confirmed)
Tighten acdream's torch reach to match retail (≈5 m), keep torches ON. Building facade then shows
a tight warm pool by each flame + dark stone elsewhere (retail-faithful). Files: `LightInfoLoader.cs`
(the Falloff→Range conversion), possibly the DatReaderWriter light read. Add a divergence-register
row if any conversion deviates. Re-verify visually (the diagnostic that confirmed the cause:
object point lights OFF == retail-match).
## State of the committed work (KEEP — all correct, just off-target for the visible bug)
| Commit | What | Verdict |
|---|---|---|
| `180b4af` | D-1 clamp point sum on its own | faithful; keep |
| `39c70f0` | D-2 prep — LightBake conformance test | keep |
| `cf62793` | D-1 shader clamp | keep |
| `c62da82` | D-2 EnvCell shell binds own light set (real leak fix) | keep |
| `b57a53e`/`156dc45` | register AP-35/AP-16 corrections | keep |
| `0980bea` | D-3 objects plain-Lambert / D-4 EnvCell no-sun | keep; correct but doesn't touch the building (it's an object) |
`tools/cdb/a7-fixd-*.cdb` capture scripts are committed. **Diagnostic shader hack reverted**
(working tree clean). Branch NOT merged — finish the torch-reach fix, visual-verify, then merge.
## DO-NOT-RETRY (cost a lot this session)
- Don't re-tune the EnvCell bake / per-channel clamp / wrap / SSBO binding for the building — the
building is a mode-0 OBJECT, none of that path lights it.
- Don't chase a time-of-day / ambient desync — ambient + time MATCH retail exactly (0.446).
- Don't "remove the sun" globally — retail DOES sun-light objects (sun 0.725).
- The visible building bug is the **torch REACH** (confirmed by isolation); start there.