fix #432: allocation-free oracle fingerprint sort — the ~6 MB/frame diagnostics tax
Second instance of the #429 (ad695589) boxing-comparer defect class, this
time in the measurement harness rather than production:
ACDREAM_AUTOMATION_ARTIFACT_DIR (with retained-UI screenshots) constructs
CurrentRenderSceneOracle, whose presence as partition observer re-enables
the G5-retired legacy InteriorEntityPartition every frame with per-entity
fingerprinting. Complete() then sorts one fingerprint per RESIDENT entity
(~60k across the streaming window), and the comparer's first key bound
x.ProjectionClass.CompareTo(y.ProjectionClass) to Enum.CompareTo(object),
boxing both operands. The 3-value enum almost always ties, so the boxing
ran on essentially every comparison: a measured ~6.2 MB and ~14 ms per
frame, everywhere — not town-specific and not view-triggered, which is
also why it appeared to "latch" (the resident set drives it, not the view).
Comparing the underlying integral value keeps the identical order. Hermetic
gate: one warmed observed partition of 20,000 entities allocated 15,876,088
bytes before, and passes a <64 KiB bound after
(OracleObservedPartitionAllocationTests).
Ordinary play never constructs the oracle, so no player-visible behavior
changes; what changes is that captures taken with the automation artifact
directory set are no longer taxed. The #429 acceptance data is unaffected
(owner drives and the deciding A/B arms ran with the artifact dir null).
The temporary [pview-alloc] attribution probe that localized this is
retired in the same commit; the gate test now guards the defect.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
b217a292bb
commit
92999b0101
3 changed files with 180 additions and 18 deletions
124
docs/ISSUES.md
124
docs/ISSUES.md
|
|
@ -24,6 +24,45 @@ What does NOT go here:
|
||||||
- Every session: scan OPEN issues at start; promote/close anything we touched during the session before ending.
|
- Every session: scan OPEN issues at start; promote/close anything we touched during the session before ending.
|
||||||
- Promoting to a Phase: mark as `DONE (promoted to Phase X)` + commit SHA where the Phase entry landed.
|
- Promoting to a Phase: mark as `DONE (promoted to Phase X)` + commit SHA where the Phase entry landed.
|
||||||
|
|
||||||
|
## #433 — Stale entities from OTHER landblocks visible ("hanging in the air") after portal travel or /ls near Holtburg
|
||||||
|
|
||||||
|
**Status:** OPEN
|
||||||
|
**Severity:** MEDIUM (visibly wrong world state after ordinary travel)
|
||||||
|
**Filed:** 2026-08-24 (owner report)
|
||||||
|
**Component:** entity lifecycle / landblock retirement / reveal generation
|
||||||
|
|
||||||
|
**Symptom (owner):** sometimes, after portaling around or using `/ls`
|
||||||
|
(lifestone recall) while close to Holtburg, monsters and other objects
|
||||||
|
from OTHER landblocks are visible hanging in the air — as if the old
|
||||||
|
world's entities were never flushed. Intermittent; "sometimes."
|
||||||
|
|
||||||
|
**Relationship to #432:** NOT its cause — #432 reproduces on a fresh
|
||||||
|
login parked at town center with no prior travel. But both may share a
|
||||||
|
residency root (content near Holtburg staying resident/drawn when it
|
||||||
|
should be retired). The #432 DrawInside attribution probe should also
|
||||||
|
report WHOSE entities/cells the per-frame cost walks; if dead-generation
|
||||||
|
leftovers appear there, merge the investigations.
|
||||||
|
|
||||||
|
**Where to look first:** the J6.2 canonical reveal-generation owner
|
||||||
|
(Runtime owns the sole reveal generation and old-world quiescence —
|
||||||
|
`docs/plans/2026-07-24-modern-runtime-architecture.md` Slice E/J6.2:
|
||||||
|
"generation-scoped old-world quiescence"), the entity teardown path at
|
||||||
|
generation reset (`RuntimeEntityDirectory` tombstones), and the
|
||||||
|
presentation sidecars (`LiveEntityProjectionStore`) — an entity drawn
|
||||||
|
without its landblock means the graphical sidecar outlived either its
|
||||||
|
runtime record or its cell residency. Distinguish: (a) runtime entity
|
||||||
|
alive but should be dead (server never sent destroy / we dropped it),
|
||||||
|
(b) runtime dead but presentation sidecar leaked, (c) entity correctly
|
||||||
|
alive but its OWN landblock geometry retired while inside the Far ring.
|
||||||
|
"Hanging in the air" + "other landblocks" suggests (b) or (c).
|
||||||
|
|
||||||
|
**Repro lead:** portal arrivals and /ls near Holtburg; intermittent.
|
||||||
|
Capture: `ACDREAM_PROBE_CELL=1` + entity-ledger counts at the reveal
|
||||||
|
transition; a screenshot naming one floating guid would pin (a) vs (b)
|
||||||
|
immediately (F2 overlay shows guids).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## #430 — No tooltips on skills and attributes in the character panel
|
## #430 — No tooltips on skills and attributes in the character panel
|
||||||
|
|
||||||
**Status:** OPEN
|
**Status:** OPEN
|
||||||
|
|
@ -409,7 +448,64 @@ transient parallel-load flake observed once, known-flake class).
|
||||||
|
|
||||||
## #432 — Sustained ~6.3 MB/frame + ~20 ms/frame while Holtburg town center is in view
|
## #432 — Sustained ~6.3 MB/frame + ~20 ms/frame while Holtburg town center is in view
|
||||||
|
|
||||||
**Status:** OPEN
|
**Status:** ROOT-CAUSED 2026-08-24 — NOT a production bug; a
|
||||||
|
measurement-harness mode. Fix verified in a hermetic gate, awaiting
|
||||||
|
commit approval. **Reclassified:** the mode is neither town-specific nor
|
||||||
|
view-triggered — it is active in ANY diagnostics-instrumented run,
|
||||||
|
scaled by the resident entity count of the whole streaming window.
|
||||||
|
|
||||||
|
**Root cause (three links, each verified):**
|
||||||
|
1. `ACDREAM_AUTOMATION_ARTIFACT_DIR` (+ retained-UI screenshots)
|
||||||
|
constructs `CurrentRenderSceneOracle`
|
||||||
|
(`FrameRootComposition.cs:349`). Ordinary play never constructs it —
|
||||||
|
the owner was never affected.
|
||||||
|
2. The oracle's presence as partition observer flips
|
||||||
|
`LegacyPartitionDiagnosticsEnabled`
|
||||||
|
(`RetailPViewRenderer.cs`), so the G5-retired legacy
|
||||||
|
`InteriorEntityPartition` runs EVERY frame with per-entity
|
||||||
|
fingerprint observation.
|
||||||
|
3. `CurrentRenderSceneOracle.Complete` sorts the per-frame fingerprints
|
||||||
|
of every resident entity; the comparer's first key compared enums via
|
||||||
|
`x.ProjectionClass.CompareTo(y.ProjectionClass)` →
|
||||||
|
`Enum.CompareTo(object)` → boxes BOTH operands per comparison. The
|
||||||
|
3-value enum almost always ties, so the boxing executes on
|
||||||
|
essentially every one of the sort's ~n·log n comparisons — the #429
|
||||||
|
`ad695589` boxing-comparer defect class, second instance.
|
||||||
|
|
||||||
|
**Evidence:** [pview-alloc] per-phase probe (`ACDREAM_PROBE_PVIEW_ALLOC=1`,
|
||||||
|
TEMPORARY, in `RetailPViewRenderer.DrawInside`) attributes the steady
|
||||||
|
mode to the partition phase: `part=6199KB` at the login spot
|
||||||
|
(`0xABB20030`) AND `part=6189KB` at town (`0xA9B4001E`) — near
|
||||||
|
content-independent because the resident far-window entity total is
|
||||||
|
similar (~60k) in both areas; this also explains the "latch" (the
|
||||||
|
resident set, not the view, drives it). Natural experiment across the
|
||||||
|
2026-08-23/24 runs, same town route: artifact-dir NULL runs
|
||||||
|
(`d2-packon/packoff`) average 48–67 KB/frame with ~50–74 churn frames
|
||||||
|
>3 MB out of ~18–20k; artifact-dir SET runs average 2.3–4.6 MB/frame
|
||||||
|
with thousands. Hermetic repro
|
||||||
|
(`OracleObservedPartitionAllocationTests.AWarmedObservedPartitionAllocatesNearZero`):
|
||||||
|
one warmed observed partition of 20,000 entities allocated
|
||||||
|
**15,876,088 bytes**; with the one-line non-boxing compare
|
||||||
|
(`((int)x.ProjectionClass).CompareTo((int)y.ProjectionClass)`) it passes
|
||||||
|
the <64 KB gate.
|
||||||
|
|
||||||
|
**Consequences for past measurements:** any capture taken with
|
||||||
|
`ACDREAM_AUTOMATION_ARTIFACT_DIR` set carries this ~6 MB/frame + ~14 ms
|
||||||
|
diagnostic tax. The #429 acceptance data is CLEAN — the owner drives and
|
||||||
|
the A/B + d2 arms ran with the artifact dir null.
|
||||||
|
|
||||||
|
**Not related to #433:** the stale-entity sighting happens in ordinary
|
||||||
|
play (no artifact dir), so it cannot share this mechanism.
|
||||||
|
|
||||||
|
**Remaining (with the fix):** the oracle still costs real CPU per frame
|
||||||
|
(fingerprint walk + sort over ~60k entities) even allocation-free —
|
||||||
|
acceptable for a diagnostics-only path, but automation-gate FPS numbers
|
||||||
|
remain diagnostics-loaded; compare like-for-like only. The intermittent
|
||||||
|
~4.26 MB "post-world diagnostics" satellite (RenderSceneShadow
|
||||||
|
comparison, same construction condition) was not separately chased —
|
||||||
|
re-measure after this fix lands and file separately if it survives.
|
||||||
|
|
||||||
|
**Previous (superseded) framing follows for the record:**
|
||||||
**Severity:** MEDIUM (halves frame rate and allocates ~300 MB/s while it holds)
|
**Severity:** MEDIUM (halves frame rate and allocates ~300 MB/s while it holds)
|
||||||
**Filed:** 2026-08-23 (found while measuring the #429 fix; NOT caused by it —
|
**Filed:** 2026-08-23 (found while measuring the #429 fix; NOT caused by it —
|
||||||
reproduces identically on the pre-fix binary)
|
reproduces identically on the pre-fix binary)
|
||||||
|
|
@ -452,21 +548,15 @@ shows every frame as a ~20 ms "stall" at ~6.3 MB — that is THIS issue, not
|
||||||
#429 residue. Compare only non-latched segments (normal frames ~22 KB), or
|
#429 residue. Compare only non-latched segments (normal frames ~22 KB), or
|
||||||
route away from Holtburg town center.
|
route away from Holtburg town center.
|
||||||
|
|
||||||
**Next probes (in order):**
|
**Next probes:** re-add the #429-style per-phase allocation probe ONE
|
||||||
1. `ACDREAM_DUMP_MOTION=1` + a temporary inbound-position log for the
|
level deeper — inside the PView draw (`WorldSceneRenderer` →
|
||||||
LOCAL guid: does ACE send position sets for the local player every
|
`DrawInside`): per-cell / per-stage bytes with identity (cell id, entity
|
||||||
~2-3 s while running, and does acdream apply them? (Retail ignores
|
guid, draw family), so the ~6.0 MB/frame names its owner instead of the
|
||||||
routine UpdatePosition for the autonomous local player; only
|
whole pass. Report WHOSE cells/entities the walk touches (also serves
|
||||||
ForcePosition snaps — verify our projection split here.)
|
the #433 stale-entity question). Measure at the trigger cell
|
||||||
2. Compare `PlayerWeenie.InqRunRate(15230)` against ACE's computed
|
`0xA9B40019` vs the clean `0xA9B40036`. (An earlier revision of this
|
||||||
runRate for the same skill — a formula delta is the root cause if the
|
entry carried #429's disproven run-rate/UpdatePosition probe list here —
|
||||||
correction traffic confirms.
|
removed; that theory died with `ca4bae77`.)
|
||||||
3. If no corrections arrive: instrument the local movement controller's
|
|
||||||
own periodic state (autorun latch, tracker send side-effects).
|
|
||||||
|
|
||||||
**START at** `claude-memory/project_physics_collision_digest.md` (per-cell
|
|
||||||
DO-NOT-RETRY rules) and `claude-memory/project_retail_motion_outbound.md`
|
|
||||||
(TS-33 AutoPos tracker semantics) before instrumenting.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -698,7 +788,7 @@ kinds. Same class as the VM6 fix; do not guess the values.
|
||||||
|
|
||||||
## #422 — Intermittent heap-corruption exit (0xC0000374) at process exit after an offline capture (pack on OR off)
|
## #422 — Intermittent heap-corruption exit (0xC0000374) at process exit after an offline capture (pack on OR off)
|
||||||
|
|
||||||
**Status:** OPEN — filed 2026-08-22 at Campaign VM VM3; **characterised at VM7 (2026-08-23) — rare, pack-INDEPENDENT, exit-time, not yet caught with a stack.** Facts: (1) it fired once more, on the **retail (pack-off)** row of `tools/run-atmospheric-performance-matrix.ps1` at 1920×1080 / 45 s warm-up / uncapped on `621a0edf` — the first launch after a fresh build — so "retail/off never reproduced it" is withdrawn and the title's "pack-on" is wrong: the exit is in the common teardown. (2) It did not fire in 16 runs with cdb attached (High, 720p, 12 s), 24 runs launched under cdb's debug heap (High, 720p, 12 s), 6 runs under the debug heap with the exact matrix recipe, or 10 plain runs with the exact recipe and a forced non-incremental rebuild before run 1 (`tools/i422/loop-debugheap.ps1`, `tools/i422/loop-plain.ps1`) — 1 in ~57 offline runs today, ~2 %. (3) The fail-fast leaves NO Application event-log entry and NO WER report on this machine (WerSvc is in its normal on-demand state, nothing disabled), so there is no dump to read; a per-user `HKCU\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\AcDream.App.exe` key (`DumpType=2`, `DumpFolder`) is the one-time user action that turns the next occurrence into a full dump — the project does not set registry keys itself. (4) The pre-campaign binary (`6c79d35c` + VM0 patches) CANNOT be tested with this tool: it predates the gate's in-process close verb, so every run ends in the gate's forced kill and never reaches the graceful-exit path where the fault lives (10/10 "automation close timed out") — whether the fault predates Campaign AR is therefore unknown, not disproven. Evidence: `docs/research/evidence/vm7/i422-*.txt`, `artifacts/vm7-matrix/uncapped-retail-1920x1080/` (the crashing run's log — managed shutdown complete, `MossTank disabled` last). **Owner decision 2026-08-23: accepted as carried; Campaign VM merged with it open.** Next step when it recurs: the LocalDumps key above, then `tools/i422/loop-plain.ps1` / `loop-debugheap.ps1`.
|
**Status:** OPEN — filed 2026-08-22 at Campaign VM VM3; **characterised at VM7 (2026-08-23) — rare, pack-INDEPENDENT, exit-time, not yet caught with a stack.** Facts: (1) it fired once more, on the **retail (pack-off)** row of `tools/run-atmospheric-performance-matrix.ps1` at 1920×1080 / 45 s warm-up / uncapped on `621a0edf` — the first launch after a fresh build — so "retail/off never reproduced it" is withdrawn and the title's "pack-on" is wrong: the exit is in the common teardown. (2) It did not fire in 16 runs with cdb attached (High, 720p, 12 s), 24 runs launched under cdb's debug heap (High, 720p, 12 s), 6 runs under the debug heap with the exact matrix recipe, or 10 plain runs with the exact recipe and a forced non-incremental rebuild before run 1 (`tools/i422/loop-debugheap.ps1`, `tools/i422/loop-plain.ps1`) — 1 in ~57 offline runs today, ~2 %. (3) The fail-fast leaves NO Application event-log entry and NO WER report on this machine (WerSvc is in its normal on-demand state, nothing disabled), so there is no dump to read; a per-user `HKCU\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\AcDream.App.exe` key (`DumpType=2`, `DumpFolder`) is the one-time user action that turns the next occurrence into a full dump — the project does not set registry keys itself. (4) The pre-campaign binary (`6c79d35c` + VM0 patches) CANNOT be tested with this tool: it predates the gate's in-process close verb, so every run ends in the gate's forced kill and never reaches the graceful-exit path where the fault lives (10/10 "automation close timed out") — whether the fault predates Campaign AR is therefore unknown, not disproven. Evidence: `docs/research/evidence/vm7/i422-*.txt`, `artifacts/vm7-matrix/uncapped-retail-1920x1080/` (the crashing run's log — managed shutdown complete, `MossTank disabled` last). **Owner decision 2026-08-23: accepted as carried; Campaign VM merged with it open.** Next step when it recurs: the LocalDumps key above, then `tools/i422/loop-plain.ps1` / `loop-debugheap.ps1`. **Recurred 2026-08-24:** the #432 attribution run (`probe-432-094728`, connected live session, graceful WM_CLOSE, managed shutdown complete) exited `-1073740940` — third sighting, first on a CONNECTED (non-offline-capture) run; still no dump (the LocalDumps key remains unset). One earlier #429-round sighting was also at graceful close (~1 in 10 diagnostic runs that day).
|
||||||
**Component:** rendering / render packs (Campaign AR) — native teardown
|
**Component:** rendering / render packs (Campaign AR) — native teardown
|
||||||
|
|
||||||
**Description:** `tools/run-offline-pixel-gate.ps1 -RenderPackPreset high` (shipped
|
**Description:** `tools/run-offline-pixel-gate.ps1 -RenderPackPreset high` (shipped
|
||||||
|
|
|
||||||
|
|
@ -1023,7 +1023,14 @@ internal sealed class CurrentRenderSceneOracle :
|
||||||
CurrentRenderProjectionFingerprint x,
|
CurrentRenderProjectionFingerprint x,
|
||||||
CurrentRenderProjectionFingerprint y)
|
CurrentRenderProjectionFingerprint y)
|
||||||
{
|
{
|
||||||
int value = x.ProjectionClass.CompareTo(y.ProjectionClass);
|
// enum.CompareTo(enum) binds to Enum.CompareTo(object) and boxes
|
||||||
|
// BOTH operands per comparison; ProjectionClass has 3 values so
|
||||||
|
// this first key almost always ties and the boxing executes on
|
||||||
|
// essentially every comparison of the per-frame fingerprint sort
|
||||||
|
// (#432: ~6.2 MB/frame across the resident entity set whenever
|
||||||
|
// the automation-artifact oracle is constructed). Compare the
|
||||||
|
// underlying integral value instead — same order, no boxing.
|
||||||
|
int value = ((int)x.ProjectionClass).CompareTo((int)y.ProjectionClass);
|
||||||
if (value != 0) return value;
|
if (value != 0) return value;
|
||||||
value = x.LandblockId.CompareTo(y.LandblockId);
|
value = x.LandblockId.CompareTo(y.LandblockId);
|
||||||
if (value != 0) return value;
|
if (value != 0) return value;
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,65 @@
|
||||||
|
using System.Numerics;
|
||||||
|
using AcDream.App.Rendering;
|
||||||
|
using AcDream.App.Rendering.Scene;
|
||||||
|
using AcDream.Core.World;
|
||||||
|
|
||||||
|
namespace AcDream.App.Tests.Rendering;
|
||||||
|
|
||||||
|
public sealed class OracleObservedPartitionAllocationTests
|
||||||
|
{
|
||||||
|
// #432: the automation-artifact harness constructs CurrentRenderSceneOracle,
|
||||||
|
// which re-enables the G5-retired legacy partition every frame WITH the
|
||||||
|
// oracle observing every resident entity. A warmed observed partition must
|
||||||
|
// not allocate per frame — the fingerprint walk and its sort run over the
|
||||||
|
// whole streaming window's resident entity set (~60k live), so any
|
||||||
|
// per-comparison or per-entity transient multiplies into MB/frame and
|
||||||
|
// poisons every measurement taken with ACDREAM_AUTOMATION_ARTIFACT_DIR set.
|
||||||
|
[Fact]
|
||||||
|
internal void AWarmedObservedPartitionAllocatesNearZero()
|
||||||
|
{
|
||||||
|
const int EntityCount = 20_000;
|
||||||
|
var entities = new List<WorldEntity>(EntityCount);
|
||||||
|
for (int i = 0; i < EntityCount; i++)
|
||||||
|
{
|
||||||
|
// Scramble ids so the fingerprint sort does real work instead of
|
||||||
|
// consuming pre-sorted input.
|
||||||
|
uint id = unchecked((uint)i * 2654435761u);
|
||||||
|
entities.Add(new WorldEntity
|
||||||
|
{
|
||||||
|
Id = id,
|
||||||
|
SourceGfxObjOrSetupId = 0x01000000u + (id & 0xFFFu),
|
||||||
|
Position = new Vector3(i % 192, i / 192f, 0f),
|
||||||
|
Rotation = Quaternion.Identity,
|
||||||
|
MeshRefs = new[] { new MeshRef(0x01000001u, Matrix4x4.Identity) },
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
var landblockEntries = new[]
|
||||||
|
{
|
||||||
|
(LandblockId: 0xA9B4FFFFu,
|
||||||
|
AabbMin: Vector3.Zero,
|
||||||
|
AabbMax: new Vector3(192f, 192f, 100f),
|
||||||
|
Entities: (IReadOnlyList<WorldEntity>)entities,
|
||||||
|
AnimatedById: (IReadOnlyDictionary<uint, WorldEntity>?)null),
|
||||||
|
};
|
||||||
|
var visibleCells = new HashSet<uint>();
|
||||||
|
var result = new InteriorEntityPartition.Result();
|
||||||
|
var oracle = new CurrentRenderSceneOracle();
|
||||||
|
|
||||||
|
for (int warm = 0; warm < 3; warm++)
|
||||||
|
{
|
||||||
|
InteriorEntityPartition.Partition(
|
||||||
|
result, visibleCells, landblockEntries, oracle);
|
||||||
|
}
|
||||||
|
|
||||||
|
long before = GC.GetAllocatedBytesForCurrentThread();
|
||||||
|
InteriorEntityPartition.Partition(
|
||||||
|
result, visibleCells, landblockEntries, oracle);
|
||||||
|
long allocated = GC.GetAllocatedBytesForCurrentThread() - before;
|
||||||
|
|
||||||
|
Assert.True(
|
||||||
|
allocated < 64 * 1024,
|
||||||
|
$"warmed observed partition allocated {allocated} bytes for "
|
||||||
|
+ $"{EntityCount} entities");
|
||||||
|
}
|
||||||
|
}
|
||||||
Loading…
Add table
Add a link
Reference in a new issue