acdream/docs/research/2026-07-06-a7-per-cell-lighting-pseudocode.md
Erik d8984e877f fix #176: light pool tracked the camera via flood scoping - collect residents, anchor at player
The seam-floor purple flicker was NOT a draw z-fight. The in-engine
[seam-*] probe (ACDREAM_PROBE_SEAMDRAW - built because RenderDoc cannot
capture this pipeline: it hides GL_ARB_bindless_texture and the
mandatory-modern startup gate throws; AMD GPU rules out Nsight) killed
every double-draw suspect: ONE shell instance per seam cell at the
lifted z, no floor-coincident entity (portal entities sit at z=-12.05),
zero portal depth fans in the sealed Hub. What it caught instead: the
corridor floor's applied light set flipping wholesale with the flood.

Root cause: c500912b scoped BuildPointLightSnapshot by the per-frame
portal flood, on the research doc's gloss of CEnvCell::visible_cell_table
as "the portal-flood visible set". The named decomp refutes the gloss:
add_visible_cell (0x0052de40) DBObj-LOADS absent cells and inserts them;
a cell activation adds itself + its whole dat visible-cell list
(0x0052e228/0x0052e24a); entries leave only via the flush machinery.
It is the RESIDENT-cell registry - gaze can never remove a cell.
add_dynamic_lights (0x0052d410) walks the WHOLE table per frame
(caller 0x00452d30), and insert_light (0x0054d1b0) caps the pool by
distance to Render::player_pos (0x0054d1dd). Retail's pool is a function
of player position only. Ours followed the camera: turning changed the
flood (probe: 8..41 cells across one turn), the six intensity-100
under-room portal purples entered/left the pool, and the wedge blinked.

Fix: BuildPointLightSnapshot(playerWorldPos) collects ALL registered
(=resident) lit lights; over cap keeps dynamics FIRST (retail's separate
7-slot dynamic pool never competes with statics) then nearest-the-player;
the RebuildScopedLights callback is deleted. Live-verified with the probe:
full-circle turn, flood churning 8..41, the floor set held the same 8
identities on every post-spawn frame. The purple wedge SHAPE stays - it
is cdb-proven retail-faithful.

Residual deviation (AP-85 rewritten): single 128 pool vs retail's
7-dynamic/40-static degrade-scaled dual pools - the Hub now shows
7 purples + viewer where retail's cdb showed 4 + viewer + fixture slots;
if the gate reads the wedge as too purple, the A7 dual-pool cap is the
faithful trim.

Pins: PointSnapshot_HubScaleLightCount_ObjectSelectionIsCameraInvariant
(rewritten to the corrected model),
PointSnapshot_OverCap_DynamicsNeverEvictedByNearerStatics,
PointSnapshot_OverCap_KeepsNearestThePlayer,
PointSnapshot_ResidentCollection_CellTagDoesNotFilter.
Suites: Core 2599+2skip / App 726+2skip / UI 425 / Net 385.

The [seam-*] probes stay until the visual gate passes, then strip.
Correction banner added to 2026-07-06-a7-per-cell-lighting-pseudocode.md;
outcome banner on the z-fight handoff; ISSUES #176 updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 15:12:31 +02:00

303 lines
18 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# A7 dungeon lighting — retail per-cell light model (source-confirmed pseudocode)
> ## ⚠️ CORRECTION #2 (2026-07-06, later the same day — the #176 flicker root cause)
>
> **§1.3's gloss is WRONG: `CEnvCell::visible_cell_table` is NOT "the portal-flood
> visible set".** It is a **static class member** — `HashTable<uint, CEnvCell*>
> CEnvCell::visible_cell_table` (defined 0x0081dd28, ctor 0x006fea4a) — the
> **RESIDENT-EnvCell registry**. Proof from the named decomp:
> - `CEnvCell::add_visible_cell(id)` (0x0052de40) returns the cell if present,
> otherwise **loads it from the dat** (`DBObj::Get(QDID(id, 3))`) and inserts it.
> Entries leave only via the flush machinery (`cell_flush_table` /
> `flush_cells` 0x0052def0) — **camera gaze can never remove a cell**.
> - It is populated at cell activation: a cell adds **itself + its entire dat
> visible-cell list** (`add_visible_cell(this->m_DID.id)` 0x0052e228 +
> `add_visible_cell(this->stab_list[i])` loop 0x0052e24a).
> - `CEnvCell::add_dynamic_lights` (0x0052d410) walks the **whole table**
> (`begin(&visible_cell_table)`), per frame, from 0x00452d30 (SmartBox).
> - `Render::insert_light` (0x0054d1b0) sorts the pool by distance to
> **`Render::player_pos`** (0x0054d1dd0x0054d214; set from
> `player->m_position`, SmartBox 0x00453d3a, viewer fallback 0x00455ab6).
>
> **So retail's pool = all RESIDENT cells' lights, capped nearest-the-PLAYER —
> a function of player position only, camera-invariant.** The §3.1 "slice 1"
> shipped below (flood-scoped `BuildPointLightSnapshot(camPos, visibleCells)`,
> `c500912b`) implemented the wrong gloss and WAS the #176 seam-floor blink: the
> under-room portal purples entered/left the pool as the camera turned (probe
> `[seam-blk]`: flood churned 8→41 cells across a full turn). Fixed same day —
> the scoping + the `RebuildScopedLights` callback are deleted;
> `BuildPointLightSnapshot(playerWorldPos)` collects every registered lit light
> (registration lifecycle = residency) with dynamics-first player-nearest
> capping. §2's "Retail never has 366 candidates" and the through-floor
> paragraph are likewise wrong: retail's Hub pool DOES contain the under-room
> purples (the cdb capture showed them applied to every Hub cell — the wedge is
> faithful). Register row AP-85 rewritten. Lesson (feedback_bn_decomp_field_names
> class): a decomp identifier's NAME is not its semantics — verify what
> POPULATES a structure before adopting its name as its meaning.
**Date:** 2026-07-06 (continuation of the #176/#177 arc)
**Purpose:** the mandated `grep named → decompile → pseudocode → port` step 3 for
the A7 per-cell lighting fix. Captures the RETAIL light-selection model exactly as
read from `docs/research/named-retail/acclient_2013_pseudo_c.txt`, so the port can
match it line-for-line.
> ⚠️ **This document CORRECTS the #176/#177 handoff's framing.** The handoff
> (`2026-07-06-176-177-handoff-A7-lighting.md`) and the digest banner state that
> "retail registers lights per-CELL via `insert_light` 0x0054d1b0" and that
> "retail's `minimize_object_lighting` has NO global camera-nearest pool cap."
> **Both are imprecise.** Reading the source: `insert_light` maintains a GLOBAL
> player-nearest sorted pool with a SMALL cap (40 static + 7 dynamic), functionally
> analogous to acdream's `BuildPointLightSnapshot`. The real per-cell mechanism is
> the *collection phase*: retail rebuilds that global pool **each frame from only
> the currently-VISIBLE cells** (`CEnvCell::add_*_lights` walks the portal-flood
> `visible_cell_table`). That is why retail's tiny cap never bites — the candidate
> pool is pre-scoped by visibility, not by camera distance over the whole dungeon.
> This is a *better* fit for acdream than the handoff's framing, because acdream
> already computes the visible-cell set every frame (the portal flood).
---
## 1. The retail model, as source-confirmed
### 1.1 Each cell owns a light list (`CObjCell` / `CEnvCell`)
- `CObjCell::add_light(this, LIGHTOBJ*)` (`0x0052b1d0`) — appends a light to the
cell's own `light_list` (a `DArray<LIGHTOBJ const*>`), `num_lights` counter.
Populated at cell load: `CEnvCell::UnPack` (`0x0052d470`) unpacks `num_lights`
(line ~310877) and the light list straight from the dat CellStruct; the outdoor
path feeds it from the landblock's static object lights (caller at line ~285976,
`CObjCell::add_light(cell, lights->lightobj + i)`).
- So a light is DATA owned by the cell it sits in — dungeon torches live in the
EnvCell's `light_list`; a landblock's lamp-posts live in the LandCell's list.
### 1.2 A cell pushes its own lights to the global pool
```
CObjCell::add_static_to_global_lights(cell): # 0x0052b350
for lightobj in cell.light_list[0 .. cell.num_lights):
if (lightobj.flags & 1) != 0: # bit 0 set = STATIC light
Render::add_static_light(lightobj.info, cell.m_DID.id, lightobj.frame)
CObjCell::add_dynamic_to_global_lights(cell): # 0x0052b390
for lightobj in cell.light_list[0 .. cell.num_lights):
if (lightobj.flags & 1) == 0: # bit 0 clear = DYNAMIC light
Render::add_dynamic_light(lightobj.info, cell.m_DID.id, lightobj.frame)
```
The cell id (`cell.m_DID.id`) is passed through as `arg6` so the light carries its
owning cell (stored at `+0x6c` on the RenderLight; used by `insert_light` for the
block-offset distance math).
### 1.3 Per frame, ONLY visible cells contribute (the crux)
```
CEnvCell::add_dynamic_lights(): # 0x0052d410
for cell in CEnvCell::visible_cell_table: # the PORTAL-FLOOD visible set
CObjCell::add_dynamic_to_global_lights(cell)
# static counterpart — same function that ends at 0x0052def0 (line ~311650):
for cell in CEnvCell::visible_cell_table: # SAME visible set
cell.init_static_objects()
CObjCell::init_objects(cell)
CObjCell::add_static_to_global_lights(cell)
```
`visible_cell_table` is the set of cells reached by the portal flood from the
viewer's cell (retail `CEnvCell::find_visible_cells` / the `PView` gather). **A
dungeon with 366 fixtures but only 5 visible cells contributes only those 5 cells'
lights to the global pool.** This is the entire reason retail doesn't churn.
### 1.4 The global pool is small and player-sorted (`insert_light`)
```
Render::insert_light(maxCount, &num, lights[], sorted[], info, cellId, frame, base): # 0x0054d1b0
distsq = 0
if info.type == 0: # point light
# squared distance from THIS light to the PLAYER, across the cell block offset
blockOff = LandDefs::get_block_offset(player_pos.objcell_id, cellId)
distsq = |(frame.origin + blockOff) - player_pos.frame.origin|²
# ... write RenderLight fields (color/255, intensity, falloff, cone, distancesq=distsq)
# insertion-sort into sorted[] ascending by distancesq (nearest player first),
# capped at maxCount; when full, evict the farthest-from-player.
Render::add_static_light(info, cellId, frame): # 0x0054d3e0
insert_light(max_static_lights, &world_lights.num_static_lights,
world_lights.static_lights, world_lights.sorted_static_lights,
info, cellId, frame, max_dynamic_lights + 1)
Render::add_dynamic_light(info, cellId, frame): # 0x0054d420
insert_light(max_dynamic_lights, &world_lights.num_dynamic_lights,
world_lights.dynamic_lights, world_lights.sorted_dynamic_lights,
info, cellId, frame, 1)
```
**Cap values:** `max_static_lights` / `max_dynamic_lights` (`0x0081ec94` / `0x0081ec98`)
init to **0x28 = 40** and **0x7 = 7**. Recomputed in `Render::SetDegradeLevelInternal`
(`0x0054c3c0`) as a function of the graphics degrade level (constants 25/50/8/16) —
always small (tens of static, single-digit dynamic). Retail deliberately keeps the
global pool tiny; it can, because §1.3 pre-scopes the input by visibility.
### 1.5 Per-object selection (`minimize_object_lighting`) — this IS acdream's `SelectForObject`
```
Render::minimize_object_lighting(): # 0x0054d480
reset_active_lights_state()
used = 0
# DYNAMIC lights first (priority), pre-sorted nearest-player:
for i in 0 .. num_dynamic_lights:
if used < 8 and remove_object_light(sorted_dynamic_lights[i].info) == keep:
add_active_light(i, 2); used += 1
else: dynamic_light_used[i] = 0
# STATIC lights fill remaining slots:
for i in 0 .. num_static_lights:
if used >= 8: static_light_used[i] = 0; continue
L = sorted_static_lights[i]
if L.info.type != 0: # non-point (directional): always use
add_active_light(i, 1); used += 1
else: # point: sphere-overlap test
reach = L.range + local_object_radius
if |L.pos - local_object_center|² - reach² < 0.0002: # spheres overlap
add_active_light(i, 1); used += 1
else: static_light_used[i] = 0
enable_active_lights()
```
acdream's `LightManager.SelectForObject` already does the sphere-overlap + 8-cap.
The one fidelity gap: retail fills **dynamic-first (priority), then static**, from two
separate player-sorted arrays; acdream selects from one camera-sorted snapshot.
Minor — parity item, not the #176/#177 cause.
### 1.6 Static falloff curve (`calc_point_light`) — fix #2 reference
`calc_point_light` (`0x0059c8b0`) is retail's CPU per-vertex software lighting for
static geometry (accumulates into `CUSTOM_D3D_VERTEX2` r/g/b). Structure:
```
calc_point_light(vertex, &r, &g, &b, info):
d = |info.offset.origin - vertex.pos|
range = info.falloff * static_light_factor # static_light_factor ≈ 1.3
if d < range:
# N·L diffuse gate: 0.5*d + dot(vertex.normal, info.pos - vertex.pos) > 0
if faces_light:
atten = <1/d-ish curve, x87 — SEE WARNING>
f = atten * (1 - d/range) * info.intensity
r += clamp(f * info.color.r, .. info.color.r) # per-channel clamp to the light's own colour
g += clamp(f * info.color.g, ..)
b += clamp(f * info.color.b, ..)
```
> ⚠️ **Do NOT port the exact `atten` curve from this BN pseudo-C.** Lines
> 425331425341 are dense x87 FPU register juggling (`distsq/dist` vs
> `1.5/(distsq·dist)` branch on `distsq ≷ 1`), exactly the "x87 dropout / misread"
> class the project has been burned by twice (see `feedback_bn_decomp_field_names`,
> `feedback_retail_binary_dispatch`). When implementing fix #2, cross-reference a
> SECOND source (ACE / ACViewer static-light port, or the Ghidra decomp) and pin
> the curve with a conformance test before trusting it. The STRUCTURE above
> (range = falloff × static_light_factor, per-vertex N·L, intensity scale, colour
> clamp) is solid; the attenuation exponent is the part to verify.
---
## 2. Why #176/#177 happen in acdream (refined root cause)
acdream `LightManager` registers **every** fixture permanently into `_all` (server
weenie spawns + EnvCell static hydration), then `BuildPointLightSnapshot` caps at
`MaxGlobalLights=128` **nearest-CAMERA** over the WHOLE registered set. In the
Facility Hub (366 fixtures) that evicts 238/frame by camera distance; `SelectForObject`
can only choose from the surviving 128, so an in-range torch of a *visible* cell that
ranks past the cap drops from that cell's 8-set and the per-cell Gouraud lighting pops
as the camera moves (#176 seam flash / #177 stair-room pop-in).
**Retail never has 366 candidates.** It rebuilds `world_lights` each frame from ONLY
the visible cells' `light_list`s (§1.3), so the candidate pool is a handful of cells —
under the 40+7 cap — and nothing gets evicted. The camera-distance cap is a backstop
that essentially never fires because the input is already visibility-scoped.
This also explains the **through-floor purple wash** the cap-raise exposed: acdream's
flat world-space sphere-overlap of all 366 lights let an under-room portal light reach
up through a solid floor. Retail's under-room cell isn't in the corridor's
`visible_cell_table` (the flood doesn't pass through the solid floor), so its light
never enters the pool. Per-cell reach = *the light is only a candidate when its cell
is visibly flooded.*
---
## 3. The fix (materially different from "just uncap MaxGlobalLights")
**Port the visibility-scoped per-frame collection**, not a bigger cap:
1. **Tag each `LightSource` with its owning cell id** (add `CellId` to `LightSource`;
populate at every registration site from the cell/landblock in scope). Retail's
`add_*_light(info, cellId, frame)` carries exactly this.
2. **Build the per-frame point-light pool from ONLY the currently-visible cells**
the portal-flood set the renderer already computes — instead of the whole `_all`
set. This is retail's `add_*_lights over visible_cell_table`. The pool is then
naturally bounded; `MaxGlobalLights` stops biting (can keep 128 or adopt retail's
40+7 as a documented backstop). The Skip'd end-state pin
(`LightManagerTests.PointSnapshot_HubScaleLightCount_ObjectSelectionIsCameraInvariant`)
asserts exactly this: an in-range light of a visible cell is never camera-evicted.
3. **Fix #2 — static curve for stationary fixtures.** Decide `isDynamic` by whether
the light MOVES, not by dat-static-vs-weenie origin. A server-spawned wall lantern
is stationary → static 1/d³ (range × 1.3), reserving `isDynamic` (range × 1.5, 1/d)
for genuinely moving lights (portal swirls, projectiles). See §1.6 warning.
4. **Fix #3 — hunt the striped floor artifact** with the full (now visibility-scoped)
pool on. Invisible at cap 128; see the handoff for the two leading guesses.
5. **THEN uncap / adopt the retail cap** and un-skip the end-state pin.
### 3.1 acdream integration surface — as SHIPPED (slice 1: visible-cell scoping)
The renderers already select per-cell (`EnvCellRenderer.cs:1088`) and per-object
(`WbDrawDispatcher.cs:2095`) from `LightManager.PointSnapshot`; the ONLY defect was
that `PointSnapshot` was built by capping the whole `_all` set at 128 nearest-CAMERA.
The fix scopes that pool to visible cells. Concretely:
1. **`LightSource.CellId`** (new `uint`, 0 = cell-less/global). Retail's per-light cell
(insert_light arg6 → RenderLight +0x6c).
2. **`LightInfoLoader.Load(..., uint cellId = 0)`** propagates it onto each light.
3. **Both registration sites tag the owning cell** from `entity.ParentCellId`:
- Site A live weenie fixtures — `GameWindow.cs:~3682` (`cellId: entity.ParentCellId ?? 0u`).
- Site B dat EnvCell statics — `GameWindow.cs:~7696` (same).
- Viewer fill light keeps `CellId == 0` (always in the pool — retail's per-frame
`add_dynamic_light(&viewer_light, objcell_id)` is unconditional).
4. **`LightManager.BuildPointLightSnapshot(camPos, IReadOnlySet<uint>? visibleCells)`** —
a light joins the pool iff `CellId == 0` OR `visibleCells == null` (outdoor) OR
`visibleCells.Contains(CellId)`. The 128 cap stays as a now-non-biting backstop.
5. **The seam.** The per-frame order is `UpdateViewerLight → Tick → BuildPointLightSnapshot
(null-scope) → SceneLightingUbo.Build → Upload` (`GameWindow.cs:9058-9095`), and the
portal flood + all cell/entity draws happen LATER, INSIDE
`RetailPViewRenderer.DrawInside`. So the scoped rebuild is threaded via a new context
callback: `RetailPViewDrawContext.RebuildScopedLights`, invoked in `DrawInside` right
after `prepareCells` (every cell drawn this frame) is finalized and BEFORE
`PrepareRenderBatches` / the draws (`RetailPViewRenderer.cs:~131`). GameWindow wires it
to `visible => Lighting.BuildPointLightSnapshot(camPos, visible)` (`GameWindow.cs:~9371`).
The renderers hold a reference to the same `_pointSnapshot` list (rebuilt in place), and
`EnvCellRenderer._cellLightSetCache` is `.Clear()`'d every pass, so no stale indices.
`SceneLightingUbo.Build` reads `lights.Active` (Tick), not the snapshot, so it is
unaffected by the relocation. The outdoor `else` path (clipRoot == null: pre-login /
fly) never invokes the callback and keeps the legacy null-scope full pool.
6. **Validation apparatus** — `ACDREAM_PROBE_INDOOR_LIGHT=1` → one rate-limited
`[indoor-light]` line per second with the scoped-pool SET COMPOSITION
(`RenderingDiagnostics.EmitIndoorLight`): `visibleCells / pool / cellLess / registered /
droppedNonVisible / byCell[]`. This is the discriminator the `[light]` COUNTS couldn't
give (#176/#177 lived in set membership).
Fixes #2 (static curve) + #3 (stripe hunt) + the cap decision are follow-on slices.
---
## 4. Source anchors (for the register + future sessions)
| Retail fn | Addr | Role |
|---|---|---|
| `CObjCell::add_light` | 0x0052b1d0 | append light to a cell's own list |
| `CObjCell::add_static_to_global_lights` | 0x0052b350 | push a cell's static lights to the global pool |
| `CObjCell::add_dynamic_to_global_lights` | 0x0052b390 | push a cell's dynamic lights to the global pool |
| `CEnvCell::add_dynamic_lights` | 0x0052d410 | per-frame: walk `visible_cell_table`, collect dynamic |
| (static collector, ends) | 0x0052def0 | per-frame: walk `visible_cell_table`, collect static |
| `CEnvCell::UnPack` | 0x0052d470 | unpack a cell's `num_lights` + `light_list` from dat |
| `Render::insert_light` | 0x0054d1b0 | player-nearest sorted insert into `world_lights`, capped |
| `Render::add_static_light` / `add_dynamic_light` | 0x0054d3e0 / 0x0054d420 | thin wrappers → insert_light |
| `Render::minimize_object_lighting` | 0x0054d480 | per-object ≤8 pick (dynamic-priority, then static sphere-overlap) |
| `Render::SetDegradeLevelInternal` | 0x0054c3c0 | recomputes `max_static/dynamic_lights` from degrade level |
| `calc_point_light` | 0x0059c8b0 | CPU per-vertex static light curve (fix #2 ref) |
| `max_static_lights` / `max_dynamic_lights` | 0x0081ec94 / 0x0081ec98 | init 40 / 7 |