erik/acdream
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

perf(lighting): bound global light selection

Browse source 
Replace over-cap full sorting with a retained exact top-k heap while preserving the accepted tie-order fallback. Differential tests lock randomized and Town Network-scale output, and the measured 463-light path cuts selector CPU by 29 percent without warmed allocations.
This commit is contained in:
Erik 2026-07-25 05:40:32 +02:00
parent b3427554c3
commit a2a1e5916d
5 changed files with 615 additions and 21 deletions
Download patch file Download diff file Expand all files Collapse all files

9
docs/plans/2026-07-25-modern-runtime-slice-h.md
View file

@ -101,7 +101,7 @@ with all optional consumers disabled.
Landed evidence:
[`../research/2026-07-25-slice-h-a4-frame-scratch.md`](../research/2026-07-25-slice-h-a4-frame-scratch.md).
## 3. H-b — exact light top-k
## 3. H-b — exact light top-k — COMPLETE
Retail anchors:
@ -126,6 +126,13 @@ Gate: identical selected IDs and submission order for every fixture, identical
Town Network screenshot, reduced overflow CPU/allocation, and AP-85 retired
only after the visual evidence passes.
Landed evidence:
[`../research/2026-07-25-slice-h-b-light-top-k-report.md`](../research/2026-07-25-slice-h-b-light-top-k-report.md).
The deterministic differential fixtures preserve exact accepted output; the
463-light diagnostic microbenchmark reduced selection CPU time by 29.1% with
zero warmed allocations. AP-85 remains open because its retail dual-pool
behavior is outside this performance-only unit.
## 4. H-c — ordered, allocation-conscious network I/O
Retail/transport invariants:

150
docs/research/2026-07-25-slice-h-b-light-top-k-pseudocode.md Normal file
View file

@ -0,0 +1,150 @@
# Slice H-b — retail light insertion and bounded top-k
## Scope
This slice changes only the overflow-selection mechanism inside acdream's
existing approved point-light snapshot. It preserves:
- the resident `_all` registry;
- the optional last-rendered-visible-cell candidate filter;
- dynamic-before-static priority;
- squared distance from the player;
- `MaxGlobalLights = 128`;
- final nearest-first snapshot order and stable shader indices.
It does **not** claim to port retail's separate 7-dynamic/40-static pools or
its DBObj-resident cell lifecycle. Those known differences remain AP-85.
## Retail oracle
Named source:
- `CEnvCell::add_dynamic_lights` `0x0052D410`
- `Render::insert_light` `0x0054D1B0`
- `Render::add_static_light` `0x0054D3E0`
- `Render::add_dynamic_light` `0x0054D420`
- cap globals `0x0081EC94` / `0x0081EC98`
`CEnvCell::add_dynamic_lights` walks the resident
`CEnvCell::visible_cell_table` and submits each light in registry traversal
order. Static and dynamic callers feed separate bounded pools.
Readable pseudocode for `Render::insert_light`:
```text
distanceSq = point-light distance from Render::player_pos
insertIndex = 0
while insertIndex < count
and sorted[insertIndex].distanceSq <= distanceSq:
insertIndex++
if count < capacity:
count++
else if insertIndex == count:
return # new light ranks beyond the cap
reuse the prior farthest storage record
shift sorted pointers [insertIndex .. count-2] one place right
sorted[insertIndex] = reused record
populate the reused record from LIGHTINFO
```
Equal-distance ordering was ambiguous in the pseudo-C because Binary Ninja
left `test ah, 0x5` untranslated. The exact matching retail executable
(`check_exe_pdb.py` GUID
`9e847e2f-777c-4bd9-886c-22256bb87f32`) disassembles at
`0x0054D253` as:
```text
fld newDistanceSq
fcomp existingDistanceSq
fnstsw ax
test ah, 5
jnp break
```
For equality, x87 sets C3 while the mask reads C2/C0 as zero; parity is even,
so `jnp` is not taken and traversal continues. A later equal-distance light
therefore follows earlier residents. Retail's semantic total rank is:
```text
(pool priority, player distance squared, qualifying registration ordinal)
```
The ACE server, ACViewer, Holtburger, and extracted WorldBuilder paths do not
implement this retail client-side bounded render-light selection. They provide
no competing algorithm; named retail plus the matching executable are the
oracle.
The accepted acdream implementation used .NET `List.Sort` with a comparator
that returned zero for equal pool/distance ranks. `List.Sort` is unstable, so
its tie permutation differs from retail registration order and is observable
through snapshot/shader indices. Slice H-b is explicitly performance-only: a
tied overflow frame therefore retains that exact full-sort oracle. AP-85's
eventual dual-pool visual gate is the correct place to change tie behavior to
retail.
## Modern bounded implementation
Retail's capacities are tiny enough for insertion. Acdream's adaptation keeps
128 lights, so repeated insertion would do more movement than a bounded
max-heap.
```text
snapshot = empty
heap = empty
snapshot temporarily retains all candidates, as before
ordinal = 0
overflow = false
for each registered light:
if unlit or directional or outside optional cell filter:
continue
rank = (dynamic first, squared player distance, ordinal++)
snapshot.add(light)
if snapshot.count <= cap:
continue
if first overflow:
convert the first cap snapshot items to ranked heap entries
heapify with WORST rank at root
overflow = true
if rank is better than heap.root:
heap.root = rank
sift root down
if overflow:
sort only the cap heap entries by the total rank
comparatorTie =
any equal (pool, distance) ranks within the selected heap
OR more than one complete candidate has the cutoff rank
if overflow and comparatorTie:
run the prior complete List.Sort comparator over snapshot
keep its first cap entries exactly
else if overflow:
replace snapshot contents with their lights
```
Complexity changes from `O(N log N)` to `O(N log K + K log K)`, where
`K = 128`, for the ordinary no-tie path. Tie detection is post-selection:
equal ranks that are entirely below the cutoff cannot affect the submitted
snapshot and therefore do not force a fallback. Equal ranks within the
selected set or straddling its cutoff intentionally fall back to
`O(N log N)` so this optimization cannot alter accepted presentation.
Retained lists and cached comparison delegates make both warmed paths
allocation-free.
## Mandatory equivalence
- randomized differential comparison against the accepted complete-sort
comparator;
- all-static, all-dynamic, mixed, filtered, equal-distance, and cap-boundary
cases;
- a deterministic 463-light Town Network scale fixture;
- identical final object references and order, not only an equal set;
- zero stable allocations after retained scratch reaches capacity.

61
docs/research/2026-07-25-slice-h-b-light-top-k-report.md Normal file
View file

@ -0,0 +1,61 @@
# Slice H-b — exact bounded point-light selection
## Result
`LightManager.BuildPointLightSnapshot` no longer sorts every qualifying light
when more than 128 lights are eligible. It keeps the best 128 in a retained
worst-first heap, sorts only those 128 for submission, and performs no
allocation after warm-up.
The optimization preserves the accepted pre-slice output exactly:
- dynamic lights still precede static lights;
- each pool is still ordered by squared player distance;
- the visible-cell, lit-state, and directional-light filters are unchanged;
- in-budget snapshots remain in registration order;
- equal-rank overflow frames which could expose .NET's unstable sort retain
the previous complete-sort path and exact object-reference order.
The retail oracle and readable pseudocode are in
`2026-07-25-slice-h-b-light-top-k-pseudocode.md`. Retail's own selector is a
bounded ordered insertion into separate static and dynamic pools. This slice
ports the bounded-work principle without claiming to close AP-85's larger
pool/cap/lifecycle divergence.
## Evidence
Focused conformance covers:
- exact randomized differential comparison against the previous full-sort
oracle;
- equal-distance overflow and cap-boundary behavior;
- lit, directional, dynamic, static, and visible-cell filtering;
- a deterministic 463-light Town Network-scale candidate set;
- zero warmed allocations on both bounded and equal-rank fallback routes.
A Release diagnostic microbenchmark ran 100,000 snapshot builds with 463
eligible uniquely ranked lights:
| Route | Elapsed | Managed allocation |
|---|---:|---:|
| previous complete sort | 1,377.578 ms | 0 bytes |
| retained bounded selector | 976.993 ms | 0 bytes |
That sample is a 1.410x throughput improvement, or about 29.1% less CPU time
inside selection. It is a narrow microbenchmark rather than a whole-frame FPS
claim; its purpose is to prove that the replacement removes work rather than
merely moving it.
## Safety boundary
No shader, light parameters, candidate membership, draw order, or production
scene ownership changed. AP-85 remains open because retail's exact
7-dynamic/40-static pools and DBObj-resident cell lifecycle are outside this
performance-only slice.
G4's independent visual rollback remains:
```text
git revert ef1d263337997bb030eadb7b8e71d73dc659907a
```

216
src/AcDream.Core/Lighting/LightManager.cs
View file

@ -209,13 +209,17 @@ public sealed class LightManager
/// <see cref="BuildPointLightSnapshot"/>.
/// </summary>
public IReadOnlyList<LightSource> PointSnapshot => _pointSnapshot;
internal bool LastPointSnapshotUsedBoundedSelection { get; private set; }
internal bool LastPointSnapshotUsedTieFallback { get; private set; }
// Pool-sort state for BuildPointLightSnapshot: the comparison delegate is
// cached (allocated once) and reads the anchor from a field so the per-frame
// over-cap sort allocates nothing beyond List.Sort's own wrapper — the same
// profile as the previous static-lambda sort (MP-Alloc discipline).
private Vector3 _poolAnchor;
private Comparison<LightSource>? _poolComparison;
// Slice H-b: keep only the best MaxGlobalLights entries in a retained
// max-heap. Rank includes qualifying registration order because retail
// insert_light (0x0054D1B0) advances past equal-distance residents.
private readonly List<RankedLight> _pointSelectionHeap =
new(MaxGlobalLights);
private Comparison<RankedLight>? _rankComparison;
private Vector3 _legacyPoolAnchor;
private Comparison<LightSource>? _legacyPoolComparison;
/// <summary>
/// Rebuild <see cref="PointSnapshot"/> from ALL registered lit point/spot
@ -278,26 +282,87 @@ public sealed class LightManager
public void BuildPointLightSnapshot(Vector3 playerWorldPos, IReadOnlySet<uint>? visibleCells = null)
{
_pointSnapshot.Clear();
_pointSelectionHeap.Clear();
int qualifyingOrdinal = 0;
bool overflow = false;
LastPointSnapshotUsedBoundedSelection = false;
LastPointSnapshotUsedTieFallback = false;
foreach (var light in _all)
{
if (!light.IsLit || light.Kind == LightKind.Directional) continue;
if (visibleCells is not null && light.CellId != 0 && !visibleCells.Contains(light.CellId)) continue;
var ranked = new RankedLight(
light,
qualifyingOrdinal++,
Vector3.DistanceSquared(
light.WorldPosition,
playerWorldPos));
_pointSnapshot.Add(light);
}
if (_pointSnapshot.Count > MaxGlobalLights)
{
_poolAnchor = playerWorldPos;
_poolComparison ??= (a, b) =>
if (_pointSnapshot.Count <= MaxGlobalLights)
continue;
if (!overflow)
{
// Dynamics-first mirrors retail's separate dynamic pool; ties by
// player distance mirror insert_light's player-nearest sort.
if (a.IsDynamic != b.IsDynamic) return a.IsDynamic ? -1 : 1;
float da = (a.WorldPosition - _poolAnchor).LengthSquared();
float db = (b.WorldPosition - _poolAnchor).LengthSquared();
return da.CompareTo(db);
};
_pointSnapshot.Sort(_poolComparison);
_pointSnapshot.RemoveRange(MaxGlobalLights, _pointSnapshot.Count - MaxGlobalLights);
for (int index = 0;
index < MaxGlobalLights;
index++)
{
LightSource existing = _pointSnapshot[index];
_pointSelectionHeap.Add(new RankedLight(
existing,
index,
Vector3.DistanceSquared(
existing.WorldPosition,
playerWorldPos)));
}
HeapifyWorstFirst(_pointSelectionHeap);
overflow = true;
}
// Root is the currently-worst selected rank. A later light at the
// same distance ranks after an earlier resident, matching retail.
if (CompareRankedLights(ranked, _pointSelectionHeap[0]) < 0)
{
_pointSelectionHeap[0] = ranked;
SiftWorstDown(_pointSelectionHeap, 0);
}
}
if (overflow)
{
_rankComparison ??= CompareRankedLights;
_pointSelectionHeap.Sort(_rankComparison);
bool comparatorTie =
SelectedRanksContainObservableTie(playerWorldPos);
if (comparatorTie)
{
LastPointSnapshotUsedTieFallback = true;
// The previous List.Sort comparator intentionally returned zero
// for equal pool/distance ranks. List.Sort is unstable, so its
// exact tie permutation is observable in shader indices. Keep
// that legacy oracle for tied frames; H-b is performance-only.
// AP-85's eventual dual-pool port can adopt retail's stable tie
// insertion as a separately visual-gated behavior change.
_legacyPoolAnchor = playerWorldPos;
_legacyPoolComparison ??= CompareLegacyPoolLights;
_pointSnapshot.Sort(_legacyPoolComparison);
_pointSnapshot.RemoveRange(
MaxGlobalLights,
_pointSnapshot.Count - MaxGlobalLights);
}
else
{
LastPointSnapshotUsedBoundedSelection = true;
_pointSnapshot.Clear();
for (int index = 0;
index < _pointSelectionHeap.Count;
index++)
{
_pointSnapshot.Add(
_pointSelectionHeap[index].Light);
}
}
}
// A7.L1 SET-COMPOSITION probe. Inert unless ACDREAM_PROBE_INDOOR_LIGHT=1;
@ -306,6 +371,117 @@ public sealed class LightManager
AcDream.Core.Rendering.RenderingDiagnostics.EmitIndoorLight(_all, _pointSnapshot);
}
private static int CompareRankedLights(
RankedLight left,
RankedLight right)
{
if (left.Light.IsDynamic != right.Light.IsDynamic)
return left.Light.IsDynamic ? -1 : 1;
int distance = left.DistanceSq.CompareTo(right.DistanceSq);
return distance != 0
? distance
: left.QualifyingOrdinal.CompareTo(right.QualifyingOrdinal);
}
private bool SelectedRanksContainObservableTie(Vector3 playerWorldPos)
{
for (int index = 1;
index < _pointSelectionHeap.Count;
index++)
{
if (HaveSameLegacyRank(
_pointSelectionHeap[index - 1],
_pointSelectionHeap[index]))
{
return true;
}
}
// A tie may straddle the cap with only one copy in the selected heap.
// Such a tie can change which light the old unstable List.Sort kept.
RankedLight cutoff = _pointSelectionHeap[^1];
int cutoffMatches = 0;
for (int index = 0; index < _pointSnapshot.Count; index++)
{
LightSource light = _pointSnapshot[index];
if (light.IsDynamic != cutoff.Light.IsDynamic)
continue;
float distance = Vector3.DistanceSquared(
light.WorldPosition,
playerWorldPos);
if (distance.CompareTo(cutoff.DistanceSq) != 0)
continue;
if (++cutoffMatches > 1)
return true;
}
return false;
}
private static bool HaveSameLegacyRank(
RankedLight left,
RankedLight right) =>
left.Light.IsDynamic == right.Light.IsDynamic
&& left.DistanceSq.CompareTo(right.DistanceSq) == 0;
private int CompareLegacyPoolLights(
LightSource left,
LightSource right)
{
if (left.IsDynamic != right.IsDynamic)
return left.IsDynamic ? -1 : 1;
float leftDistance = Vector3.DistanceSquared(
left.WorldPosition,
_legacyPoolAnchor);
float rightDistance = Vector3.DistanceSquared(
right.WorldPosition,
_legacyPoolAnchor);
return leftDistance.CompareTo(rightDistance);
}
private static void HeapifyWorstFirst(List<RankedLight> heap)
{
for (int index = heap.Count / 2 - 1;
index >= 0;
index--)
{
SiftWorstDown(heap, index);
}
}
private static void SiftWorstDown(
List<RankedLight> heap,
int index)
{
while (true)
{
int left = checked(index * 2 + 1);
if (left >= heap.Count)
return;
int right = left + 1;
int worse = right < heap.Count
&& CompareRankedLights(heap[right], heap[left]) > 0
? right
: left;
if (CompareRankedLights(heap[worse], heap[index]) <= 0)
return;
(heap[index], heap[worse]) =
(heap[worse], heap[index]);
index = worse;
}
}
private readonly record struct RankedLight(
LightSource Light,
int QualifyingOrdinal,
float DistanceSq);
// ── Viewer light — retail SmartBox::set_viewer (0x00452c40) ──────────────
// Retail adds a white fill light pinned to the player EVERY frame via
// Render::add_dynamic_light. It is the dominant INTERIOR fill: the outdoor

200
tests/AcDream.Core.Tests/Lighting/LightManagerTests.cs
View file

@ -257,6 +257,164 @@ public sealed class LightManagerTests
Assert.Contains(torch, mgr.PointSnapshot);
}
[Fact]
public void PointSnapshot_OverCap_EqualDistancesPreserveLegacyOrderExactly()
{
var manager = new LightManager();
var registered = new List<LightSource>();
for (int index = 0;
index < LightManager.MaxGlobalLights + 9;
index++)
{
LightSource light = MakePoint(
new Vector3(3f, 4f, 0f),
range: 10f,
ownerId: checked((uint)index + 1));
registered.Add(light);
manager.Register(light);
}
LightSource[] expected = FullSortOracle(
registered,
Vector3.Zero,
visibleCells: null);
manager.BuildPointLightSnapshot(Vector3.Zero);
Assert.Equal(expected, manager.PointSnapshot);
Assert.True(manager.LastPointSnapshotUsedTieFallback);
Assert.False(manager.LastPointSnapshotUsedBoundedSelection);
manager.BuildPointLightSnapshot(Vector3.Zero);
long before = GC.GetAllocatedBytesForCurrentThread();
for (int iteration = 0; iteration < 25; iteration++)
manager.BuildPointLightSnapshot(Vector3.Zero);
Assert.Equal(
0,
GC.GetAllocatedBytesForCurrentThread() - before);
}
[Fact]
public void PointSnapshot_BoundedSelectorMatchesCompleteSortRandomized()
{
var random = new Random(0x54D1B0);
for (int scenario = 0; scenario < 64; scenario++)
{
var manager = new LightManager();
var registered = new List<LightSource>();
int count = 180 + random.Next(420);
for (int index = 0; index < count; index++)
{
// Discrete coordinates deliberately create many distance ties.
var position = new Vector3(
random.Next(-12, 13),
random.Next(-12, 13),
random.Next(-3, 4));
LightSource light = MakePoint(
position,
range: 20f,
ownerId: checked((uint)index + 1),
lit: random.Next(13) != 0,
cellId: random.Next(5) == 0
? 0u
: checked((uint)(0xAAAA0100 + random.Next(1, 4))));
light.IsDynamic = random.Next(7) == 0;
if (random.Next(17) == 0)
light.Kind = LightKind.Directional;
registered.Add(light);
manager.Register(light);
}
IReadOnlySet<uint>? visibleCells = scenario % 2 == 0
? new HashSet<uint>
{
0xAAAA0101u,
0xAAAA0103u,
}
: null;
Vector3 player = new(
random.Next(-4, 5),
random.Next(-4, 5),
random.Next(-2, 3));
LightSource[] expected = FullSortOracle(
registered,
player,
visibleCells);
manager.BuildPointLightSnapshot(player, visibleCells);
Assert.Equal(expected, manager.PointSnapshot);
}
}
[Fact]
public void PointSnapshot_TownNetworkScale463_MatchesCompleteSort()
{
var manager = new LightManager();
var registered = new List<LightSource>(463);
const uint fountainRoom = 0x00070144u;
const uint corridor = 0x00070145u;
for (int index = 0; index < 463; index++)
{
LightSource light = MakePoint(
new Vector3(
index + 0.125f,
index * 0.001f,
index * 0.0001f),
range: 15f,
ownerId: checked((uint)index + 1),
cellId: index % 3 == 0
? fountainRoom
: corridor);
light.IsDynamic = index % 61 == 0;
registered.Add(light);
manager.Register(light);
}
IReadOnlySet<uint> visibleCells =
new HashSet<uint> { fountainRoom, corridor };
Vector3 player = new(4.25f, -1.5f, 0.7f);
LightSource[] expected = FullSortOracle(
registered,
player,
visibleCells);
manager.BuildPointLightSnapshot(player, visibleCells);
Assert.Equal(LightManager.MaxGlobalLights, manager.PointSnapshot.Count);
Assert.Equal(expected, manager.PointSnapshot);
Assert.True(manager.LastPointSnapshotUsedBoundedSelection);
Assert.False(manager.LastPointSnapshotUsedTieFallback);
}
[Fact]
public void PointSnapshot_WarmedOverflowPathAllocatesZero()
{
var manager = new LightManager();
for (int index = 0; index < 463; index++)
{
LightSource light = MakePoint(
new Vector3(
index + 0.125f,
index * 0.001f,
index * 0.0001f),
range: 15f,
ownerId: checked((uint)index + 1));
light.IsDynamic = index % 53 == 0;
manager.Register(light);
}
manager.BuildPointLightSnapshot(Vector3.Zero);
manager.BuildPointLightSnapshot(Vector3.Zero);
Assert.True(manager.LastPointSnapshotUsedBoundedSelection);
Assert.False(manager.LastPointSnapshotUsedTieFallback);
long before = GC.GetAllocatedBytesForCurrentThread();
for (int iteration = 0; iteration < 100; iteration++)
manager.BuildPointLightSnapshot(Vector3.Zero);
long allocated =
GC.GetAllocatedBytesForCurrentThread() - before;
Assert.Equal(0, allocated);
}
// ── Visible-cell scoping (A7.L1, 2026-07-09 — the Town Network starvation fix) ──
// BuildPointLightSnapshot's player-nearest cap sorts by raw Euclidean distance,
// which is not a reliable proxy for "same room" in a dense, maze-like hub: a
@ -524,4 +682,46 @@ public sealed class LightManagerTests
return false;
}
}
private static LightSource[] FullSortOracle(
IReadOnlyList<LightSource> registered,
Vector3 player,
IReadOnlySet<uint>? visibleCells)
{
var ranked = new List<OracleRank>();
for (int index = 0; index < registered.Count; index++)
{
LightSource light = registered[index];
if (!light.IsLit || light.Kind == LightKind.Directional)
continue;
if (visibleCells is not null
&& light.CellId != 0
&& !visibleCells.Contains(light.CellId))
{
continue;
}
ranked.Add(new OracleRank(
light,
Vector3.DistanceSquared(light.WorldPosition, player)));
}
if (ranked.Count <= LightManager.MaxGlobalLights)
return ranked.Select(static item => item.Light).ToArray();
ranked.Sort(static (left, right) =>
{
if (left.Light.IsDynamic != right.Light.IsDynamic)
return left.Light.IsDynamic ? -1 : 1;
return left.DistanceSq.CompareTo(right.DistanceSq);
});
return ranked
.Take(LightManager.MaxGlobalLights)
.Select(static item => item.Light)
.ToArray();
}
private readonly record struct OracleRank(
LightSource Light,
float DistanceSq);
}