acdream/docs/research/2026-07-24-runtime-performance-audit.md

Runtime CPU/GPU/memory performance audit

Capture date: 2026-07-24

Benchmark commit: 2c3da8e153123240a289417856d0f937887cdace

Status: Complete, report-only audit. No runtime code, settings, or game behavior were changed during the benchmark.

Compact data:

• 2026-07-24-runtime-performance-audit-summary.json
• 2026-07-24-runtime-performance-audit-checkpoints.csv

Architecture response: ../plans/2026-07-24-modern-runtime-architecture.md

Errata: section 13 records the corrections from the 2026-07-24 adversarial review (test-condition disclosure, single-frame allocation attribution, unit conversions, LOH fragmentation, exception-count provenance). Read it before citing this report's §1/§2/§4 numbers.

---

1. Executive result

Steady capped play was inexpensive. Portal transitions were not.

The client maintained approximately 32 FPS on a 32 Hz Remote Desktop display. Normal update work was approximately 0.4–1.3 ms median, and external GPU 3D use was 7.6% median. The dominant performance failure was the burst of runtime content preparation, publication, and retirement performed around portal transitions:

• Up to approximately 204 MiB allocated in one frame.
• Approximately 2.22 GiB/s maximum managed allocation rate.
• 38 Gen-2 collections over the six-minute route.
• 256 ms maximum recorded GC pause.
• 276 ms maximum frame.

The final live managed heap was approximately 194 MiB even though the process had previously reached 1.66 GiB working set and 2.27 GiB private bytes. Entity, particle, effect, and tracked GPU counts returned to repeatable plateaus when locations were revisited. The audit therefore did not reproduce an unbounded logical-object or GPU-resource leak. It reproduced excessive allocation and retained high-water memory.

Allocation stacks identify runtime mesh/texture preparation as the largest portal source. Update-thread landblock publication and retirement then amplify the burst with complete dictionary/set/sort work and unmetered per-entity operations.

---

2. Test conditions

Hardware and host

Item	Value
CPU	AMD Ryzen 7 9800X3D, 8 cores / 16 logical processors, up to 4.7 GHz
GPU	AMD Radeon RX 9070 XT
GPU driver	32.0.31021.5001
Active display path	Microsoft Remote Display Adapter
Active resolution	2056 × 1290 at 32 Hz
Physical display	2560 × 1440 at 240 Hz
OS	Windows NT 10.0.26200.0
Runtime	.NET 10 / SDK 10.0.300

Client configuration

• Release build.
• Normal graphical client.
• Retail retained UI.
• Developer UI disabled.
• Normal capped presentation.
• ACDREAM_FRAME_PROF=1 enabled for observational frame metrics.
• Local ACE server.
• Graceful client shutdown.

The RDP display and normal presentation cap prevent this capture from establishing uncapped maximum FPS or maximum GPU throughput. Capped and uncapped local-display results must remain separate in future comparisons.

Route

The existing connected R6 route produced nine canonical checkpoints:

1. Caul baseline.
2. Sawato baseline.
3. Rynthid world edge.
4. Aerlinthe.
5. Sawato return.
6. Holtburg.
7. Caul return.
8. Sawato plateau.
9. Caul plateau.

The route includes portal transitions, location revisits, camera turns, input exercises, stable dwell periods, checkpoint capture, and graceful disconnect.

---

3. Measurement methods

The primary process ran for approximately 359 seconds. Measurements combined:

• The permanent [frame-prof] ring profiler.
• System.Runtime counters through dotnet-counters.
• Windows per-process CPU, memory, handle, thread, I/O, and GPU-engine samples.
• A 30-second CPU-sampling EventPipe trace at Aerlinthe.
• A final GC heap dump.
• A short allocation-sampling trace covering startup and Caul.
• A short exception-stack trace covering a connected route segment.
• Canonical world lifecycle/resource checkpoint JSON.

The allocation trace reports sampled allocation intervals. A type attached to an allocation tick is not proof that a single object of that type had the full sampled byte count. Stack attribution is the reliable part of that capture.

The CPU trace was collected while counters were also attached. It is suitable for ranking active code, not for absolute standalone CPU percentages.

---

4. Headline measurements

Measurement	Result
Stable capped rate	approximately 32 FPS / 31.2 ms wall frame
Steady update work	approximately 0.4–1.3 ms median
Normal update p95	approximately 0.5–2.0 ms
CPU normalized p50 / p95 / max	0.05% / 1.30% / 2.46%
CPU as one-core p50 / p95 / max	0.8% / 20.8% / 39.4%
GPU 3D p50 / p95 / max	7.6% / 15.9% / 16.8%
Working set range	776.0–1,661.9 MiB
Private-byte range	1,286.4–2,269.1 MiB
Dedicated GPU range	632.5–818.0 MiB
Handle range	680–704
Thread range	26–34
Managed allocation average	approximately 40.79 MB/s
Managed allocation maximum	2,386,818,304 B/s
Total sampled managed allocation	14,643,958,472 B
Maximum single-frame allocation	approximately 203.6 MiB
Maximum frame	276.4 ms
Worst five-second p95 / p99	52.6 ms / 147 ms
Maximum update p95	37.5 ms
Maximum upload p95	3.5 ms
Total GC pause time	approximately 2.07 s
Maximum one-second GC pause	256 ms
Collections Gen0 / Gen1 / Gen2	99 / 105 / 38
Lock contentions	4,614 total; 952/s maximum
Thread-pool queue	0 throughout capture

Steady per-frame allocation varied by scene:

• Sawato: approximately 27–28 KiB/frame.
• Caul: approximately 42–44 KiB/frame.
• Holtburg: approximately 53.5 KiB/frame outside the portal burst.

Portal windows produced maximum frames of approximately 203.6 MiB, 141.6 MiB, 141.5 MiB, 58.5 MiB, and 35.1 MiB.

---

5. Checkpoint and leak analysis

The complete sanitized checkpoint table is in the companion CSV. Representative revisit comparisons:

Location	Visit	Landblocks	Entities	Emitters	Particles	Tracked GPU MiB
Caul	baseline	625	21,031	1,712	4,562	275.3
Caul	return	625	21,031	1,714	4,530	439.2
Caul	plateau	625	21,025	1,720	4,502	430.7
Sawato	baseline	625	6,659	586	3,021	347.6
Sawato	return	625	6,659	563	2,666	345.6
Sawato	plateau	625	6,667	581	2,808	455.2

Entity, emitter, and particle populations converged. GPU residence warmed and then plateaued rather than increasing without bound. The different Sawato GPU plateau reflects the process-wide cache working set accumulated through the complete route, not an increasing Sawato entity population.

All three Caul materializations recorded viewport-before-ready invariant failures. This is separate evidence that destination work and reveal are not currently one atomic generation-ready transaction.

Final managed heap

• Live heap: 203,771,023 B, approximately 194.3 MiB.
• Live objects: 1,314,128.
• Last observed committed managed memory: approximately 830.6 MiB.

Large retained categories included:

Category	Approximate live size
System.Byte[]	27.5 MiB
DeferredAlphaInstance[]	5.0 MiB
Matrix4x4[]	4.6 MiB
Int32[]	4.2 MiB
CachedBatch[]	4.2 MiB
Single[]	4.0 MiB
WorldEntity objects	3.9 MiB
InstanceLightSet[]	3.5 MiB
Fixed ParticleInstance[]	3.5 MiB

There were approximately 20,000 records in several static presentation registries, including pose, script activation, static owner state, and effect profile tables. Individually they are modest, but together they add portal create/teardown work and several MiB of retained state.

The gap between 194 MiB live heap and much larger committed/private memory is primarily GC segment high-water, fragmentation, native/driver allocations, mapped/cached data, and GPU resource residence.

---

6. CPU attribution

Most CPU samples represented waits and presentation pacing:

Stack/category	Inclusive share of all samples
ManualResetEventSlim.Wait	28.79%
WaitHandle.WaitOneNoCheck	21.68%
LowLevelLifoSemaphore.WaitForSignal	20.48%
Socket.ReceiveFrom	7.23%
FramePacingController.CompleteFrame	6.44%

Named active App work was comparatively small in this capped scene:

App path	Inclusive share of all samples
RenderFrameOrchestrator	0.60%
WorldSceneRenderer	0.48%
RetailPViewRenderer.DrawInside	0.44%
DrawLandscapeThroughOutsideView	0.33%
DrawLandscapeSlice	0.23%
DatDatabaseWrapper.TryGet	0.21%
WbDrawDispatcher.Draw	0.19%
UpdateFrameOrchestrator	0.15%
MeshExtractor.PrepareMeshData	0.13%

The largest named exclusive App sites were InteriorEntityPartition.Partition, WbDrawDispatcher.Draw, terrain clip AABB testing, PView outside work, and deferred-alpha preparation. These are small under the 32 Hz cap but scale with uncapped frame rate.

There was no evidence of a spinning thread-pool worker or a growing thread-pool queue.

---

7. GPU analysis and timing limitation

External GPU 3D utilization was low in this capture, so the client was not GPU-saturated. Tracked GPU resources warmed from approximately 275 MiB to approximately 431 MiB, with OS-reported dedicated memory reaching 818 MiB. Approximately 599 tracked textures and 43–48 atlas arrays were resident near the end of the route.

The difference between tracked and OS-reported GPU memory can include driver allocation, texture-array slack, alignment, metadata, framebuffer resources, and resources not covered by acdream's tracker.

The internal gpu_ms value is not valid active-render time under a cap. GpuFrameTimer.FrameBoundary starts at one frame boundary and completes at the next, so the GPU query includes idle/pacing gaps. Its 23–30 ms result must not be interpreted as GPU saturation. Correct pass/submission timestamp measurement is Slice A of the modern runtime plan.

---

8. Allocation and exception attribution

Runtime mesh and texture preparation

The largest sampled allocation stacks were:

AcDream.Content.MeshExtractor.AddSurfaceToBatch
  -> PrepareGfxObjMeshData
  -> PrepareMeshData
  -> AcDream.App.Rendering.Wb.ObjectMeshManager.ProcessQueue

The path decodes compressed and palette textures into RGBA arrays, clones cached pixels for surface-local alpha changes, constructs vertex/index collections, and produces upload batches. Rebuilding those products during portals creates large short-lived graphs even when the same world content has been seen before.

Relevant implementation: ../src/AcDream.Content/MeshExtractor.cs.

Invalid Setup probes

The focused exception trace captured 1,587 exceptions with no lost events:

• 1,335 ArgumentOutOfRangeException.
• 252 SocketException.

The DAT exceptions followed:

QualifiedDataId<T>.Unpack
  -> Setup.Unpack
  -> DatDatabase.TryGet
  -> RuntimeDatCollection.Get<Setup>
  -> LivePresentationComposition.ResolveActivation
  -> EntityScriptActivator.OnCreate
  -> GpuWorldState.ActivateLandblockPresentation
  -> StreamingController.ApplyResult

ResolveActivation attempts Get<Setup> for every static source ID and catches all exceptions. Many IDs are GfxObj IDs, not Setup IDs. The mesh path already contains the required DID-prefix validation.

Relevant implementation: ../src/AcDream.App/Composition/LivePresentationComposition.cs.

Network receive timeout

NetClient configures a receive timeout and catches SocketError.TimedOut as a normal heartbeat every approximately 250 ms. Every send also materializes datagram.ToArray().

Relevant implementation: ../src/AcDream.Core.Net/NetClient.cs.

---

9. Ranked findings

P0 — Portal content preparation and publication storm

Sources

• MeshExtractor runtime decode and texture cloning.
• StreamingController.DrainAndApply.
• LandblockRetirementCoordinator.RunForEachEntity.
• Landblock render, physics, static presentation, effect, light, and plugin publication.

Impact

• 204 MiB maximum single-frame allocation.
• 276 ms maximum frame.
• Gen-2 GC and committed-heap high-water.
• Destination readiness failures.

Recommendation

• Finish the deduplicated baked pak and consume prepared data at runtime.
• Keep visibility/collision detachment immediate.
• Budget expensive publication/teardown by time, bytes, and entity operations.
• Prepare the destination across portal frames and reveal one ready generation.

Plan mapping: Slices B, C, and E.

P1 — Exception-driven DAT type resolution

Source

LivePresentationComposition.ResolveActivation parses arbitrary source IDs as Setup and catches failure.

Impact

Thousands of constructed/thrown exceptions during streaming, avoidable DAT parsing, noisy diagnostics, and main-thread activation work.

Recommendation

Use explicit DID type metadata, non-allocating typed lookup, negative caching, and prepared Setup presentation metadata. Exceptions must represent corruption or failure, not ordinary type selection.

Plan mapping: Slice C.

P1 — Complete collection rebuilds during static publication

Source

LandblockStaticPresentationPublisher creates entity/snapshot dictionaries, copies previous keys to a set, and sorts IDs/snapshots per publication.

Impact

Portal allocation and update-thread CPU scale with complete landblock contents.

Recommendation

Carry canonical ordered prepared arrays, use cursor-based publication receipts, and publish versioned deltas rather than reconstructing complete collections.

Plan mapping: Slices C, E, and F.

P1 — DAT resolution/cache churn

Sources

• DatCollectionAdapter.ResolveId.
• MeshExtractor.ResolveId().ToList().OrderByDescending().FirstOrDefault().
• Small bounded typed-object cache plus serialized database access.

Impact

Repeated parsing on revisits, allocation from resolution candidates/LINQ, 4,614 lock contentions, and repeated invalid type probes.

Recommendation

Use typed non-allocating resolution, prepared compact metadata, negative/type caches, and remove world render/streaming DAT reads after the pak cutover.

Plan mapping: Slices B and C.

P2 — Whole-world render partition every frame

Source

InteriorEntityPartition.Partition scans every entity in every loaded landblock each render frame. Caul had approximately 21,000 entities.

Impact

CPU cost grows with the complete resident world rather than visible cells and changes. Reused lists reduce allocation but not traversal.

Recommendation

Maintain outdoor-static, per-cell, dynamic, translucent, selectable, and light indices incrementally in a render-only scene database. Existing retail PView remains the visible-cell authority.

Plan mapping: Slices F and G.

P2 — Unconditional diagnostic and retained-UI work

Sources

• WorldSceneDiagnosticsController performs every-landblock AABB tests without an active debug consumer.
• GpuWorldState.LandblockBounds uses an iterator.
• Retained UI reapplies anchors and walks the overlay tree every frame.
• Item cooldown UI scans complete item lists.
• Dialog handling snapshots open-order arrays.

Impact

Moderate capped cost, but it scales directly with uncapped FPS and adds small steady allocations.

Recommendation

Reuse renderer facts, skip inactive diagnostics, add UI dirty-layout/event registries, and replace frame iterators/snapshots with borrowed stable views.

Plan mapping: Slice H.

P2 — High but bounded GPU/cache residency

Sources

• Object mesh and texture caches.
• Standalone and composite texture arrays.
• Fixed 65,536-instance particle CPU/GPU buffer.
• Deferred-alpha scratch arrays that grow to scene high-water.

Impact

633–818 MiB dedicated GPU memory during the route, plus driver/native difference from tracked resources.

Recommendation

Introduce typed asset leases and unified accounting, measure atlas occupancy and fragmentation, use generation-safe cost-aware eviction, and shrink or segment large scratch buffers only with hysteresis.

Do not reduce view distance or particle range as the primary correction.

Plan mapping: Slice D.

P3 — Smaller continuous allocations and background overhead

Sources

• Per-frame RetailPViewFrameInput.
• Yield iterators and periodic liveness sets.
• Full candidate light sorting.
• UDP send copies.
• Socket timeout exceptions.

Impact

Not responsible for the portal storm, but measurable at high uncapped FPS or across many headless sessions.

Recommendation

Use reusable frame views, stable spans, spatial light candidates, span-based socket sends, and cancellable blocking/async receive without exception-driven timeouts.

Plan mapping: Slices H and J.

---

10. Recommended execution order

The audit supports this order:

1. Correct GPU/pass measurement and checkpoint accounting.
2. Complete EnvCell dedup and the full pak bake.
3. Cut world mesh/texture/Setup presentation preparation over to prepared data.
4. Add explicit CPU/staging/GPU residency ownership and accounting.
5. Replace portal budget bypasses with staged cost budgets and one reveal generation.
6. Add the incremental render scene and delta submission.
7. Remove remaining per-frame UI/diagnostic/network allocation.
8. Extract the presentation-independent runtime for Linux/headless use.

The complete slice definitions and acceptance gates are in the linked modern runtime architecture plan.

---

11. Reproduction

Canonical route

dotnet build AcDream.slnx -c Release

$env:ACDREAM_TEST_USER = '<account>'
$env:ACDREAM_TEST_PASS = '<password>'
$env:ACDREAM_FRAME_PROF = '1'

.\tools\run-connected-r6-soak.ps1 -Repository (Get-Location).Path -SkipBuild

Do not place credentials in committed logs or reports.

Runtime counters

After identifying the client PID:

dotnet-counters collect `
  --process-id <pid> `
  --counters System.Runtime `
  --format csv `
  --output dotnet-counters.csv

CPU trace

dotnet-trace collect `
  --process-id <pid> `
  --profile cpu-sampling `
  --duration 00:00:30 `
  --output cpu.nettrace

Final live heap

dotnet-gcdump collect --process-id <pid> --output final.gcdump

For comparable results:

• Use the same commit, Release configuration, route, and camera/input script.
• Record RDP/capped and local/uncapped modes separately.
• Gracefully close the client.
• Capture checkpoint JSON before interpreting process memory.
• Compare live heap, GC committed memory, tracked GPU bytes, and OS GPU memory separately.

---

12. Artifact retention policy

The local raw capture contained 96 files totaling approximately 157.9 MiB. Heavy and environment-specific artifacts were deliberately excluded from Git:

• .nettrace and generated .etlx.
• .gcdump.
• Full dotnet-counters.csv.
• Complete process/GPU sample CSV.
• Full client stdout/stderr.
• Generated trace-reader project binaries.
• Process IDs, timestamps, absolute local paths, and session-specific logs.

Those files are valuable for local forensic work but poor Git artifacts: they are binary or noisy, inflate permanent history, are difficult for code-review tools to inspect, and can expose local/session metadata.

This report plus the compact JSON and checkpoint CSV preserve the benchmark's method, provenance, measurements, interpretation, and decision-driving evidence in a form another engineer or model can review directly.

---

13. Errata — 2026-07-24 adversarial review

An independent adversarial review (43-agent verification against commit e627d9a4, same date) confirmed every cited code mechanism in this report (12/12 file:line citations accurate) but found the following statements wrong or under-qualified. Per the project's historical-record rules the original sections above are left untouched; this appendix is the correction record.

1. Test conditions (§2) — "Normal capped presentation" is wrong. The reproduction script this report cites (tools/run-connected-r6-soak.ps1) unconditionally set ACDREAM_UNCAPPED_RENDER=1, which disables BOTH VSync and the software pacer (FramePacingPolicy.Resolve). The ~32 FPS ceiling came solely from the RDP display path, not client pacing. ACDREAM_DUMP_MOVE_TRUTH=1 (per-movement console logging) was also active for the whole capture and undisclosed. Unresolved provenance question: the CPU trace shows FramePacingController.CompleteFrame at 6.44% inclusive, which should be a near-zero no-op under those flags — either the capture used a different launcher than documented or the trace row is misattributed. Post-review the soak script defaults to capped and takes an explicit -Uncapped switch, and emits an env-disclosure.json per run.
2. Single-frame allocation attribution (§1, §9 P0). The 203.6 MiB "maximum single-frame allocation" is measured with GC.GetAllocatedBytesForCurrentThread() — frame-thread only. MeshExtractor decode runs exclusively on 4 background Task.Run workers (ObjectMeshManager.ProcessQueue), so it structurally cannot appear in that figure. The single-frame spike belongs to the main-thread publication/retirement sources (StreamingController.DrainAndApply / ApplyResult, retirement, static publication). MeshExtractor remains the top source of the PROCESS-WIDE impacts (allocation rate, Gen-2, GC pauses — §8's sampled stacks are process-wide). Consequence: Slice E owns the frame-spike symptom; Slices B/C own the process-wide rate.
3. Unit conversions (§1). 1,661.9 MiB = 1.62 GiB (not 1.66) and 2,269.1 MiB = 2.22 GiB (not 2.27); the printed values divided MiB by 1000.
4. Committed managed memory (§5). 830.6 MiB is the FINAL checkpoint, not the peak: the checkpoint peak was 1,242.1 MiB (caul-return) and the continuous counter maximum 1,360,900,096 B ≈ 1,298 MiB.
5. LOH fragmentation (§5). The companion JSON records LOH size max 807.5 MB with 524.1 MB fragmentation (~65%) — the largest quantified contributor to the committed-vs-live gap. It never appears in the prose above and should anchor the Slice D accounting work.
6. Exception counts (§8). This report's own figures are the focused-trace 1,587 (1,335 ArgumentOutOfRangeException + 252 SocketException). The "4,520 + 1,168" figures quoted by the companion plan document have no derivation here or in the committed artifacts and must not be cited until re-derived from a fresh capture.
7. GC counts (§4). 99/105/38 are per-generation EXCLUSIVE counts from the .NET 10 System.Runtime meter (dotnet.gc.collections), under which gen1 > gen0 is legal; cumulative GC.CollectionCount equivalents are 242/143/38. The "38 Gen-2" headline is correct under either reading.
8. Cache context (§8, §9 P1). Four bounded LRU caches already front the mesh/texture path (GPU render-data 1 GiB / 50 objects; CPU prepared-mesh 128 MiB / 100 entries; decoded-texture 64 MiB / 128 entries; DAT typed object 64 MiB / 256 entries ×4 databases). Revisits rebuild because these are UNDERSIZED relative to the route's working set (~599 tracked textures), not because no caching mechanism exists. Likewise, texture atlas arrays have real fence-gated, budgeted eviction (empty-atlas pool 64 MiB; composite arrays 128/64 MiB) — the route simply never pushed the ceilings, so "bounded" in §7 was bounded-by-route, not policy-exercised.
9. Portal-burst list (§4). The five max-frame values are not attributed to specific transitions or route order, so warm-vs-cold portal behavior is not derivable from the committed data. Cache hit/miss/evict counters were added to the checkpoint JSON post-review to settle this.
10. Dead-code caveat (§9 P2). The "fixed 65,536-instance particle CPU/GPU buffer" belonged to an unreachable code path (ParticleBatcher; never fed by production — the live particle renderer is ParticleRenderer with growable buffers). It was deleted post-review; its ~7 MiB combined residency was dead weight, and its GPU buffer was one concrete instance of the §7 tracked-vs-OS gap (never registered with the GPU tracker).
11. Soak-script dwell bug (§11). The script's own comment requires sampling after the 25-second liveness deadline (LiveEntityLivenessController), but it slept 12 s — return-gate samples could include un-pruned prior-destination records. Fixed post-review (26 s).
12. Reproducibility scope (§12). The §4 headline table's continuous CPU/GPU-engine/memory percentiles were produced by manual sampling not committed to the repo; per-scene steady-allocation figures, four of five portal-burst values, "~20,000 registry records", "599 tracked textures / 43–48 atlas arrays", and the 23–30 ms gpu_ms figure likewise have no surviving committed evidence. They can be reviewed but not independently regenerated; treat them as reproducible-not-reviewed until the Slice A tooling recaptures them.

The full review (verdicts, refutations, and the new findings the audit missed) and the resulting plan amendments are recorded in ../plans/2026-07-24-modern-runtime-architecture.md section 0.