docs(perf): preserve runtime benchmark evidence
Record the sanitized six-minute CPU/GPU/memory audit, compact machine-readable metrics, checkpoint plateaus, attribution, reproduction method, and raw-artifact retention policy. Link the evidence from the consolidated architecture plan and roadmap without committing binary traces or session metadata. Co-authored-by: Codex <codex@openai.com>
This commit is contained in:
Erik
parent
944ee55584
commit
e627d9a403
5 changed files with 776 additions and 0 deletions
|
|
@ -1573,6 +1573,8 @@ port in any phase — no separate listing here.
|
||||||
> memory audit reproduced a larger portal allocation/GC storm (approximately
|
> memory audit reproduced a larger portal allocation/GC storm (approximately
|
||||||
> 204 MiB in one frame, 276 ms maximum frame, 38 Gen-2 collections) while
|
> 204 MiB in one frame, 276 ms maximum frame, 38 Gen-2 collections) while
|
||||||
> confirming stable entity/effect/GPU plateaus and low steady GPU utilization.
|
> confirming stable entity/effect/GPU plateaus and low steady GPU utilization.
|
||||||
|
> The complete sanitized benchmark report is
|
||||||
|
> [`2026-07-24-runtime-performance-audit.md`](../research/2026-07-24-runtime-performance-audit.md).
|
||||||
> The user requested a detailed modern architecture plan. The resulting
|
> The user requested a detailed modern architecture plan. The resulting
|
||||||
> [`2026-07-24-modern-runtime-architecture.md`](2026-07-24-modern-runtime-architecture.md)
|
> [`2026-07-24-modern-runtime-architecture.md`](2026-07-24-modern-runtime-architecture.md)
|
||||||
> unifies the unfinished MP pak/render-world work with future Track LH, preserves
|
> unifies the unfinished MP pak/render-world work with future Track LH, preserves
|
||||||
|
|
|
||||||
|
|
@ -21,6 +21,10 @@ visibility ports, and all behavior already accepted by the user.
|
||||||
|
|
||||||
## 1. Why the architecture should change
|
## 1. Why the architecture should change
|
||||||
|
|
||||||
|
The complete methodology, attribution, sanitized checkpoint data, and raw-
|
||||||
|
artifact retention policy are recorded in
|
||||||
|
[`../research/2026-07-24-runtime-performance-audit.md`](../research/2026-07-24-runtime-performance-audit.md).
|
||||||
|
|
||||||
The 2026-07-24 six-minute, nine-stop connected route established:
|
The 2026-07-24 six-minute, nine-stop connected route established:
|
||||||
|
|
||||||
| Observation | Measurement | Architectural implication |
|
| Observation | Measurement | Architectural implication |
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,10 @@
|
||||||
|
sequence,name,loaded_landblocks,world_entities,animated_entities,particle_emitters,particles,effect_owners,tracked_gpu_mib,managed_mib,managed_committed_mib,fps,frame_ms,reveal_invariant_failures
|
||||||
|
1,caul-baseline,625,21031,18,1712,4562,20896,275.3,239.1,503.3,31.999,31.251,1
|
||||||
|
2,sawato-baseline,625,6659,25,586,3021,5379,347.6,258.4,444.0,31.988,31.262,1
|
||||||
|
3,rynthid,525,9404,19,83,708,9364,362.0,280.2,544.1,32.031,31.220,1
|
||||||
|
4,aerlinthe,374,6660,4,2677,76336,6634,335.0,164.3,564.2,31.986,31.264,1
|
||||||
|
5,sawato-return,625,6659,25,563,2666,5380,345.6,217.7,573.0,32.014,31.237,1
|
||||||
|
6,holtburg,625,10383,51,209,328,10094,417.8,948.6,954.5,32.009,31.242,1
|
||||||
|
7,caul-return,625,21031,18,1714,4530,20901,439.2,260.0,1242.1,32.040,31.211,2
|
||||||
|
8,sawato-plateau,625,6667,32,581,2808,5389,455.2,265.5,1142.6,31.999,31.251,2
|
||||||
|
9,caul-plateau,625,21025,13,1720,4502,20896,430.7,219.7,830.6,32.025,31.225,3
|
||||||
|
159
docs/research/2026-07-24-runtime-performance-audit-summary.json
Normal file
159
docs/research/2026-07-24-runtime-performance-audit-summary.json
Normal file
|
|
@ -0,0 +1,159 @@
|
||||||
|
{
|
||||||
|
"schema": "acdream.runtime-performance-audit.v1",
|
||||||
|
"capture_date": "2026-07-24",
|
||||||
|
"benchmark_commit": "2c3da8e153123240a289417856d0f937887cdace",
|
||||||
|
"report_only": true,
|
||||||
|
"runtime_changed": false,
|
||||||
|
"duration_seconds": 359,
|
||||||
|
"environment": {
|
||||||
|
"configuration": "Release",
|
||||||
|
"developer_ui": false,
|
||||||
|
"retail_ui": true,
|
||||||
|
"frame_profiler": true,
|
||||||
|
"presentation": "normal capped",
|
||||||
|
"display_path": "Remote Desktop",
|
||||||
|
"display_width": 2056,
|
||||||
|
"display_height": 1290,
|
||||||
|
"display_refresh_hz": 32,
|
||||||
|
"cpu": "AMD Ryzen 7 9800X3D",
|
||||||
|
"physical_cores": 8,
|
||||||
|
"logical_processors": 16,
|
||||||
|
"gpu": "AMD Radeon RX 9070 XT",
|
||||||
|
"gpu_driver": "32.0.31021.5001",
|
||||||
|
"os": "Windows NT 10.0.26200.0",
|
||||||
|
"dotnet_sdk": "10.0.300"
|
||||||
|
},
|
||||||
|
"route": [
|
||||||
|
"caul-baseline",
|
||||||
|
"sawato-baseline",
|
||||||
|
"rynthid",
|
||||||
|
"aerlinthe",
|
||||||
|
"sawato-return",
|
||||||
|
"holtburg",
|
||||||
|
"caul-return",
|
||||||
|
"sawato-plateau",
|
||||||
|
"caul-plateau"
|
||||||
|
],
|
||||||
|
"frame": {
|
||||||
|
"stable_fps": 32.0,
|
||||||
|
"stable_wall_frame_ms": 31.2,
|
||||||
|
"steady_update_median_ms_min": 0.4,
|
||||||
|
"steady_update_median_ms_max": 1.3,
|
||||||
|
"steady_update_p95_ms_min": 0.5,
|
||||||
|
"steady_update_p95_ms_max": 2.0,
|
||||||
|
"maximum_frame_ms": 276.4,
|
||||||
|
"worst_window_p95_ms": 52.6,
|
||||||
|
"worst_window_p99_ms": 147.0,
|
||||||
|
"maximum_update_p95_ms": 37.5,
|
||||||
|
"maximum_upload_p95_ms": 3.5,
|
||||||
|
"maximum_single_frame_allocation_mib": 203.6
|
||||||
|
},
|
||||||
|
"process_samples": {
|
||||||
|
"sample_count": 87,
|
||||||
|
"cpu_normalized_percent": {
|
||||||
|
"p50": 0.05,
|
||||||
|
"p95": 1.3,
|
||||||
|
"max": 2.46
|
||||||
|
},
|
||||||
|
"cpu_one_core_percent": {
|
||||||
|
"p50": 0.8,
|
||||||
|
"p95": 20.8,
|
||||||
|
"max": 39.4
|
||||||
|
},
|
||||||
|
"gpu_3d_percent": {
|
||||||
|
"p50": 7.6,
|
||||||
|
"p95": 15.9,
|
||||||
|
"max": 16.8
|
||||||
|
},
|
||||||
|
"working_set_mib": {
|
||||||
|
"min": 776.0,
|
||||||
|
"max": 1661.9
|
||||||
|
},
|
||||||
|
"private_mib": {
|
||||||
|
"min": 1286.4,
|
||||||
|
"max": 2269.1
|
||||||
|
},
|
||||||
|
"gpu_dedicated_mib": {
|
||||||
|
"min": 632.5,
|
||||||
|
"max": 818.0
|
||||||
|
},
|
||||||
|
"handles": {
|
||||||
|
"min": 680,
|
||||||
|
"max": 704
|
||||||
|
},
|
||||||
|
"threads": {
|
||||||
|
"min": 26,
|
||||||
|
"max": 34
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"runtime_counters": {
|
||||||
|
"allocation_rate_average_bytes_per_second": 40791000,
|
||||||
|
"allocation_rate_maximum_bytes_per_second": 2386818304,
|
||||||
|
"total_sampled_allocation_bytes": 14643958472,
|
||||||
|
"gc_pause_total_seconds": 2.07,
|
||||||
|
"gc_pause_maximum_one_second_window_ms": 256,
|
||||||
|
"gen0_collections": 99,
|
||||||
|
"gen1_collections": 105,
|
||||||
|
"gen2_collections": 38,
|
||||||
|
"lock_contentions_total": 4614,
|
||||||
|
"lock_contentions_maximum_per_second": 952,
|
||||||
|
"threadpool_queue_maximum": 0,
|
||||||
|
"working_set_maximum_bytes": 1748021248,
|
||||||
|
"managed_committed_maximum_bytes": 1360900096,
|
||||||
|
"loh_size_maximum_bytes": 807548768,
|
||||||
|
"loh_fragmentation_maximum_bytes": 524062256,
|
||||||
|
"argument_out_of_range_exceptions": 4520,
|
||||||
|
"socket_exceptions": 1168
|
||||||
|
},
|
||||||
|
"final_heap": {
|
||||||
|
"live_bytes": 203771023,
|
||||||
|
"live_mib": 194.3,
|
||||||
|
"live_objects": 1314128,
|
||||||
|
"largest_categories_mib": {
|
||||||
|
"System.Byte[]": 27.5,
|
||||||
|
"DeferredAlphaInstance[]": 5.0,
|
||||||
|
"System.Numerics.Matrix4x4[]": 4.6,
|
||||||
|
"System.Int32[]": 4.2,
|
||||||
|
"CachedBatch[]": 4.2,
|
||||||
|
"System.Single[]": 4.0,
|
||||||
|
"WorldEntity": 3.9,
|
||||||
|
"InstanceLightSet[]": 3.5,
|
||||||
|
"ParticleInstance[]": 3.5
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"focused_exception_trace": {
|
||||||
|
"exceptions": 1587,
|
||||||
|
"lost_events": 0,
|
||||||
|
"argument_out_of_range_exceptions": 1335,
|
||||||
|
"socket_exceptions": 252,
|
||||||
|
"primary_argument_exception_path": "LivePresentationComposition.ResolveActivation -> RuntimeDatCollection.Get<Setup> -> DatDatabase.TryGet -> Setup.Unpack",
|
||||||
|
"socket_exception_path": "WorldSession.NetReceiveLoop -> NetClientWorldSessionTransport.Receive -> Socket.ReceiveFrom"
|
||||||
|
},
|
||||||
|
"cpu_trace_inclusive_percent": {
|
||||||
|
"ManualResetEventSlim.Wait": 28.79,
|
||||||
|
"WaitHandle.WaitOneNoCheck": 21.68,
|
||||||
|
"LowLevelLifoSemaphore.WaitForSignal": 20.48,
|
||||||
|
"Socket.ReceiveFrom": 7.23,
|
||||||
|
"FramePacingController.CompleteFrame": 6.44,
|
||||||
|
"RenderFrameOrchestrator": 0.6,
|
||||||
|
"WorldSceneRenderer": 0.48,
|
||||||
|
"RetailPViewRenderer.DrawInside": 0.44,
|
||||||
|
"WbDrawDispatcher.Draw": 0.19,
|
||||||
|
"UpdateFrameOrchestrator": 0.15,
|
||||||
|
"MeshExtractor.PrepareMeshData": 0.13
|
||||||
|
},
|
||||||
|
"conclusions": {
|
||||||
|
"unbounded_managed_entity_or_particle_leak_reproduced": false,
|
||||||
|
"unbounded_tracked_gpu_resource_leak_reproduced": false,
|
||||||
|
"portal_allocation_gc_storm_reproduced": true,
|
||||||
|
"gpu_saturated_in_capped_rdp_capture": false,
|
||||||
|
"internal_gpu_frame_timer_valid_under_cap": false,
|
||||||
|
"viewport_before_ready_failures": 3
|
||||||
|
},
|
||||||
|
"raw_artifacts": {
|
||||||
|
"tracked_in_git": false,
|
||||||
|
"file_count": 96,
|
||||||
|
"total_mib": 157.88,
|
||||||
|
"exclusion_reason": "Binary/noisy artifacts contain environment-specific process metadata and would permanently inflate Git history."
|
||||||
|
}
|
||||||
|
}
|
||||||
601
docs/research/2026-07-24-runtime-performance-audit.md
Normal file
601
docs/research/2026-07-24-runtime-performance-audit.md
Normal file
|
|
@ -0,0 +1,601 @@
|
||||||
|
# Runtime CPU/GPU/memory performance audit
|
||||||
|
|
||||||
|
**Capture date:** 2026-07-24
|
||||||
|
|
||||||
|
**Benchmark commit:** `2c3da8e153123240a289417856d0f937887cdace`
|
||||||
|
|
||||||
|
**Status:** Complete, report-only audit. No runtime code, settings, or game
|
||||||
|
behavior were changed during the benchmark.
|
||||||
|
|
||||||
|
**Compact data:**
|
||||||
|
|
||||||
|
- [`2026-07-24-runtime-performance-audit-summary.json`](2026-07-24-runtime-performance-audit-summary.json)
|
||||||
|
- [`2026-07-24-runtime-performance-audit-checkpoints.csv`](2026-07-24-runtime-performance-audit-checkpoints.csv)
|
||||||
|
|
||||||
|
**Architecture response:**
|
||||||
|
[`../plans/2026-07-24-modern-runtime-architecture.md`](../plans/2026-07-24-modern-runtime-architecture.md)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Executive result
|
||||||
|
|
||||||
|
Steady capped play was inexpensive. Portal transitions were not.
|
||||||
|
|
||||||
|
The client maintained approximately 32 FPS on a 32 Hz Remote Desktop display.
|
||||||
|
Normal update work was approximately 0.4–1.3 ms median, and external GPU 3D use
|
||||||
|
was 7.6% median. The dominant performance failure was the burst of runtime
|
||||||
|
content preparation, publication, and retirement performed around portal
|
||||||
|
transitions:
|
||||||
|
|
||||||
|
- Up to approximately 204 MiB allocated in one frame.
|
||||||
|
- Approximately 2.22 GiB/s maximum managed allocation rate.
|
||||||
|
- 38 Gen-2 collections over the six-minute route.
|
||||||
|
- 256 ms maximum recorded GC pause.
|
||||||
|
- 276 ms maximum frame.
|
||||||
|
|
||||||
|
The final live managed heap was approximately 194 MiB even though the process
|
||||||
|
had previously reached 1.66 GiB working set and 2.27 GiB private bytes. Entity,
|
||||||
|
particle, effect, and tracked GPU counts returned to repeatable plateaus when
|
||||||
|
locations were revisited. The audit therefore did **not** reproduce an
|
||||||
|
unbounded logical-object or GPU-resource leak. It reproduced excessive
|
||||||
|
allocation and retained high-water memory.
|
||||||
|
|
||||||
|
Allocation stacks identify runtime mesh/texture preparation as the largest
|
||||||
|
portal source. Update-thread landblock publication and retirement then amplify
|
||||||
|
the burst with complete dictionary/set/sort work and unmetered per-entity
|
||||||
|
operations.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Test conditions
|
||||||
|
|
||||||
|
### Hardware and host
|
||||||
|
|
||||||
|
| Item | Value |
|
||||||
|
|---|---|
|
||||||
|
| CPU | AMD Ryzen 7 9800X3D, 8 cores / 16 logical processors, up to 4.7 GHz |
|
||||||
|
| GPU | AMD Radeon RX 9070 XT |
|
||||||
|
| GPU driver | `32.0.31021.5001` |
|
||||||
|
| Active display path | Microsoft Remote Display Adapter |
|
||||||
|
| Active resolution | 2056 × 1290 at 32 Hz |
|
||||||
|
| Physical display | 2560 × 1440 at 240 Hz |
|
||||||
|
| OS | Windows NT 10.0.26200.0 |
|
||||||
|
| Runtime | .NET 10 / SDK 10.0.300 |
|
||||||
|
|
||||||
|
### Client configuration
|
||||||
|
|
||||||
|
- Release build.
|
||||||
|
- Normal graphical client.
|
||||||
|
- Retail retained UI.
|
||||||
|
- Developer UI disabled.
|
||||||
|
- Normal capped presentation.
|
||||||
|
- `ACDREAM_FRAME_PROF=1` enabled for observational frame metrics.
|
||||||
|
- Local ACE server.
|
||||||
|
- Graceful client shutdown.
|
||||||
|
|
||||||
|
The RDP display and normal presentation cap prevent this capture from
|
||||||
|
establishing uncapped maximum FPS or maximum GPU throughput. Capped and uncapped
|
||||||
|
local-display results must remain separate in future comparisons.
|
||||||
|
|
||||||
|
### Route
|
||||||
|
|
||||||
|
The existing connected R6 route produced nine canonical checkpoints:
|
||||||
|
|
||||||
|
1. Caul baseline.
|
||||||
|
2. Sawato baseline.
|
||||||
|
3. Rynthid world edge.
|
||||||
|
4. Aerlinthe.
|
||||||
|
5. Sawato return.
|
||||||
|
6. Holtburg.
|
||||||
|
7. Caul return.
|
||||||
|
8. Sawato plateau.
|
||||||
|
9. Caul plateau.
|
||||||
|
|
||||||
|
The route includes portal transitions, location revisits, camera turns, input
|
||||||
|
exercises, stable dwell periods, checkpoint capture, and graceful disconnect.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Measurement methods
|
||||||
|
|
||||||
|
The primary process ran for approximately 359 seconds. Measurements combined:
|
||||||
|
|
||||||
|
- The permanent `[frame-prof]` ring profiler.
|
||||||
|
- `System.Runtime` counters through `dotnet-counters`.
|
||||||
|
- Windows per-process CPU, memory, handle, thread, I/O, and GPU-engine samples.
|
||||||
|
- A 30-second CPU-sampling EventPipe trace at Aerlinthe.
|
||||||
|
- A final GC heap dump.
|
||||||
|
- A short allocation-sampling trace covering startup and Caul.
|
||||||
|
- A short exception-stack trace covering a connected route segment.
|
||||||
|
- Canonical world lifecycle/resource checkpoint JSON.
|
||||||
|
|
||||||
|
The allocation trace reports sampled allocation intervals. A type attached to
|
||||||
|
an allocation tick is not proof that a single object of that type had the full
|
||||||
|
sampled byte count. Stack attribution is the reliable part of that capture.
|
||||||
|
|
||||||
|
The CPU trace was collected while counters were also attached. It is suitable
|
||||||
|
for ranking active code, not for absolute standalone CPU percentages.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Headline measurements
|
||||||
|
|
||||||
|
| Measurement | Result |
|
||||||
|
|---|---:|
|
||||||
|
| Stable capped rate | approximately 32 FPS / 31.2 ms wall frame |
|
||||||
|
| Steady update work | approximately 0.4–1.3 ms median |
|
||||||
|
| Normal update p95 | approximately 0.5–2.0 ms |
|
||||||
|
| CPU normalized p50 / p95 / max | 0.05% / 1.30% / 2.46% |
|
||||||
|
| CPU as one-core p50 / p95 / max | 0.8% / 20.8% / 39.4% |
|
||||||
|
| GPU 3D p50 / p95 / max | 7.6% / 15.9% / 16.8% |
|
||||||
|
| Working set range | 776.0–1,661.9 MiB |
|
||||||
|
| Private-byte range | 1,286.4–2,269.1 MiB |
|
||||||
|
| Dedicated GPU range | 632.5–818.0 MiB |
|
||||||
|
| Handle range | 680–704 |
|
||||||
|
| Thread range | 26–34 |
|
||||||
|
| Managed allocation average | approximately 40.79 MB/s |
|
||||||
|
| Managed allocation maximum | 2,386,818,304 B/s |
|
||||||
|
| Total sampled managed allocation | 14,643,958,472 B |
|
||||||
|
| Maximum single-frame allocation | approximately 203.6 MiB |
|
||||||
|
| Maximum frame | 276.4 ms |
|
||||||
|
| Worst five-second p95 / p99 | 52.6 ms / 147 ms |
|
||||||
|
| Maximum update p95 | 37.5 ms |
|
||||||
|
| Maximum upload p95 | 3.5 ms |
|
||||||
|
| Total GC pause time | approximately 2.07 s |
|
||||||
|
| Maximum one-second GC pause | 256 ms |
|
||||||
|
| Collections Gen0 / Gen1 / Gen2 | 99 / 105 / 38 |
|
||||||
|
| Lock contentions | 4,614 total; 952/s maximum |
|
||||||
|
| Thread-pool queue | 0 throughout capture |
|
||||||
|
|
||||||
|
Steady per-frame allocation varied by scene:
|
||||||
|
|
||||||
|
- Sawato: approximately 27–28 KiB/frame.
|
||||||
|
- Caul: approximately 42–44 KiB/frame.
|
||||||
|
- Holtburg: approximately 53.5 KiB/frame outside the portal burst.
|
||||||
|
|
||||||
|
Portal windows produced maximum frames of approximately 203.6 MiB, 141.6 MiB,
|
||||||
|
141.5 MiB, 58.5 MiB, and 35.1 MiB.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Checkpoint and leak analysis
|
||||||
|
|
||||||
|
The complete sanitized checkpoint table is in the companion CSV. Representative
|
||||||
|
revisit comparisons:
|
||||||
|
|
||||||
|
| Location | Visit | Landblocks | Entities | Emitters | Particles | Tracked GPU MiB |
|
||||||
|
|---|---|---:|---:|---:|---:|---:|
|
||||||
|
| Caul | baseline | 625 | 21,031 | 1,712 | 4,562 | 275.3 |
|
||||||
|
| Caul | return | 625 | 21,031 | 1,714 | 4,530 | 439.2 |
|
||||||
|
| Caul | plateau | 625 | 21,025 | 1,720 | 4,502 | 430.7 |
|
||||||
|
| Sawato | baseline | 625 | 6,659 | 586 | 3,021 | 347.6 |
|
||||||
|
| Sawato | return | 625 | 6,659 | 563 | 2,666 | 345.6 |
|
||||||
|
| Sawato | plateau | 625 | 6,667 | 581 | 2,808 | 455.2 |
|
||||||
|
|
||||||
|
Entity, emitter, and particle populations converged. GPU residence warmed and
|
||||||
|
then plateaued rather than increasing without bound. The different Sawato GPU
|
||||||
|
plateau reflects the process-wide cache working set accumulated through the
|
||||||
|
complete route, not an increasing Sawato entity population.
|
||||||
|
|
||||||
|
All three Caul materializations recorded `viewport-before-ready` invariant
|
||||||
|
failures. This is separate evidence that destination work and reveal are not
|
||||||
|
currently one atomic generation-ready transaction.
|
||||||
|
|
||||||
|
### Final managed heap
|
||||||
|
|
||||||
|
- Live heap: 203,771,023 B, approximately 194.3 MiB.
|
||||||
|
- Live objects: 1,314,128.
|
||||||
|
- Last observed committed managed memory: approximately 830.6 MiB.
|
||||||
|
|
||||||
|
Large retained categories included:
|
||||||
|
|
||||||
|
| Category | Approximate live size |
|
||||||
|
|---|---:|
|
||||||
|
| `System.Byte[]` | 27.5 MiB |
|
||||||
|
| `DeferredAlphaInstance[]` | 5.0 MiB |
|
||||||
|
| `Matrix4x4[]` | 4.6 MiB |
|
||||||
|
| `Int32[]` | 4.2 MiB |
|
||||||
|
| `CachedBatch[]` | 4.2 MiB |
|
||||||
|
| `Single[]` | 4.0 MiB |
|
||||||
|
| `WorldEntity` objects | 3.9 MiB |
|
||||||
|
| `InstanceLightSet[]` | 3.5 MiB |
|
||||||
|
| Fixed `ParticleInstance[]` | 3.5 MiB |
|
||||||
|
|
||||||
|
There were approximately 20,000 records in several static presentation
|
||||||
|
registries, including pose, script activation, static owner state, and effect
|
||||||
|
profile tables. Individually they are modest, but together they add portal
|
||||||
|
create/teardown work and several MiB of retained state.
|
||||||
|
|
||||||
|
The gap between 194 MiB live heap and much larger committed/private memory is
|
||||||
|
primarily GC segment high-water, fragmentation, native/driver allocations,
|
||||||
|
mapped/cached data, and GPU resource residence.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. CPU attribution
|
||||||
|
|
||||||
|
Most CPU samples represented waits and presentation pacing:
|
||||||
|
|
||||||
|
| Stack/category | Inclusive share of all samples |
|
||||||
|
|---|---:|
|
||||||
|
| `ManualResetEventSlim.Wait` | 28.79% |
|
||||||
|
| `WaitHandle.WaitOneNoCheck` | 21.68% |
|
||||||
|
| `LowLevelLifoSemaphore.WaitForSignal` | 20.48% |
|
||||||
|
| `Socket.ReceiveFrom` | 7.23% |
|
||||||
|
| `FramePacingController.CompleteFrame` | 6.44% |
|
||||||
|
|
||||||
|
Named active App work was comparatively small in this capped scene:
|
||||||
|
|
||||||
|
| App path | Inclusive share of all samples |
|
||||||
|
|---|---:|
|
||||||
|
| `RenderFrameOrchestrator` | 0.60% |
|
||||||
|
| `WorldSceneRenderer` | 0.48% |
|
||||||
|
| `RetailPViewRenderer.DrawInside` | 0.44% |
|
||||||
|
| `DrawLandscapeThroughOutsideView` | 0.33% |
|
||||||
|
| `DrawLandscapeSlice` | 0.23% |
|
||||||
|
| `DatDatabaseWrapper.TryGet` | 0.21% |
|
||||||
|
| `WbDrawDispatcher.Draw` | 0.19% |
|
||||||
|
| `UpdateFrameOrchestrator` | 0.15% |
|
||||||
|
| `MeshExtractor.PrepareMeshData` | 0.13% |
|
||||||
|
|
||||||
|
The largest named exclusive App sites were `InteriorEntityPartition.Partition`,
|
||||||
|
`WbDrawDispatcher.Draw`, terrain clip AABB testing, PView outside work, and
|
||||||
|
deferred-alpha preparation. These are small under the 32 Hz cap but scale with
|
||||||
|
uncapped frame rate.
|
||||||
|
|
||||||
|
There was no evidence of a spinning thread-pool worker or a growing thread-pool
|
||||||
|
queue.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. GPU analysis and timing limitation
|
||||||
|
|
||||||
|
External GPU 3D utilization was low in this capture, so the client was not
|
||||||
|
GPU-saturated. Tracked GPU resources warmed from approximately 275 MiB to
|
||||||
|
approximately 431 MiB, with OS-reported dedicated memory reaching 818 MiB.
|
||||||
|
Approximately 599 tracked textures and 43–48 atlas arrays were resident near the
|
||||||
|
end of the route.
|
||||||
|
|
||||||
|
The difference between tracked and OS-reported GPU memory can include driver
|
||||||
|
allocation, texture-array slack, alignment, metadata, framebuffer resources,
|
||||||
|
and resources not covered by acdream's tracker.
|
||||||
|
|
||||||
|
The internal `gpu_ms` value is not valid active-render time under a cap.
|
||||||
|
`GpuFrameTimer.FrameBoundary` starts at one frame boundary and completes at the
|
||||||
|
next, so the GPU query includes idle/pacing gaps. Its 23–30 ms result must not be
|
||||||
|
interpreted as GPU saturation. Correct pass/submission timestamp measurement is
|
||||||
|
Slice A of the modern runtime plan.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Allocation and exception attribution
|
||||||
|
|
||||||
|
### Runtime mesh and texture preparation
|
||||||
|
|
||||||
|
The largest sampled allocation stacks were:
|
||||||
|
|
||||||
|
```text
|
||||||
|
AcDream.Content.MeshExtractor.AddSurfaceToBatch
|
||||||
|
-> PrepareGfxObjMeshData
|
||||||
|
-> PrepareMeshData
|
||||||
|
-> AcDream.App.Rendering.Wb.ObjectMeshManager.ProcessQueue
|
||||||
|
```
|
||||||
|
|
||||||
|
The path decodes compressed and palette textures into RGBA arrays, clones cached
|
||||||
|
pixels for surface-local alpha changes, constructs vertex/index collections,
|
||||||
|
and produces upload batches. Rebuilding those products during portals creates
|
||||||
|
large short-lived graphs even when the same world content has been seen before.
|
||||||
|
|
||||||
|
Relevant implementation:
|
||||||
|
[`../src/AcDream.Content/MeshExtractor.cs`](../../src/AcDream.Content/MeshExtractor.cs).
|
||||||
|
|
||||||
|
### Invalid Setup probes
|
||||||
|
|
||||||
|
The focused exception trace captured 1,587 exceptions with no lost events:
|
||||||
|
|
||||||
|
- 1,335 `ArgumentOutOfRangeException`.
|
||||||
|
- 252 `SocketException`.
|
||||||
|
|
||||||
|
The DAT exceptions followed:
|
||||||
|
|
||||||
|
```text
|
||||||
|
QualifiedDataId<T>.Unpack
|
||||||
|
-> Setup.Unpack
|
||||||
|
-> DatDatabase.TryGet
|
||||||
|
-> RuntimeDatCollection.Get<Setup>
|
||||||
|
-> LivePresentationComposition.ResolveActivation
|
||||||
|
-> EntityScriptActivator.OnCreate
|
||||||
|
-> GpuWorldState.ActivateLandblockPresentation
|
||||||
|
-> StreamingController.ApplyResult
|
||||||
|
```
|
||||||
|
|
||||||
|
`ResolveActivation` attempts `Get<Setup>` for every static source ID and catches
|
||||||
|
all exceptions. Many IDs are GfxObj IDs, not Setup IDs. The mesh path already
|
||||||
|
contains the required DID-prefix validation.
|
||||||
|
|
||||||
|
Relevant implementation:
|
||||||
|
[`../src/AcDream.App/Composition/LivePresentationComposition.cs`](../../src/AcDream.App/Composition/LivePresentationComposition.cs).
|
||||||
|
|
||||||
|
### Network receive timeout
|
||||||
|
|
||||||
|
`NetClient` configures a receive timeout and catches `SocketError.TimedOut` as a
|
||||||
|
normal heartbeat every approximately 250 ms. Every send also materializes
|
||||||
|
`datagram.ToArray()`.
|
||||||
|
|
||||||
|
Relevant implementation:
|
||||||
|
[`../src/AcDream.Core.Net/NetClient.cs`](../../src/AcDream.Core.Net/NetClient.cs).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Ranked findings
|
||||||
|
|
||||||
|
### P0 — Portal content preparation and publication storm
|
||||||
|
|
||||||
|
**Sources**
|
||||||
|
|
||||||
|
- `MeshExtractor` runtime decode and texture cloning.
|
||||||
|
- `StreamingController.DrainAndApply`.
|
||||||
|
- `LandblockRetirementCoordinator.RunForEachEntity`.
|
||||||
|
- Landblock render, physics, static presentation, effect, light, and plugin
|
||||||
|
publication.
|
||||||
|
|
||||||
|
**Impact**
|
||||||
|
|
||||||
|
- 204 MiB maximum single-frame allocation.
|
||||||
|
- 276 ms maximum frame.
|
||||||
|
- Gen-2 GC and committed-heap high-water.
|
||||||
|
- Destination readiness failures.
|
||||||
|
|
||||||
|
**Recommendation**
|
||||||
|
|
||||||
|
- Finish the deduplicated baked pak and consume prepared data at runtime.
|
||||||
|
- Keep visibility/collision detachment immediate.
|
||||||
|
- Budget expensive publication/teardown by time, bytes, and entity operations.
|
||||||
|
- Prepare the destination across portal frames and reveal one ready generation.
|
||||||
|
|
||||||
|
**Plan mapping:** Slices B, C, and E.
|
||||||
|
|
||||||
|
### P1 — Exception-driven DAT type resolution
|
||||||
|
|
||||||
|
**Source**
|
||||||
|
|
||||||
|
`LivePresentationComposition.ResolveActivation` parses arbitrary source IDs as
|
||||||
|
Setup and catches failure.
|
||||||
|
|
||||||
|
**Impact**
|
||||||
|
|
||||||
|
Thousands of constructed/thrown exceptions during streaming, avoidable DAT
|
||||||
|
parsing, noisy diagnostics, and main-thread activation work.
|
||||||
|
|
||||||
|
**Recommendation**
|
||||||
|
|
||||||
|
Use explicit DID type metadata, non-allocating typed lookup, negative caching,
|
||||||
|
and prepared Setup presentation metadata. Exceptions must represent corruption
|
||||||
|
or failure, not ordinary type selection.
|
||||||
|
|
||||||
|
**Plan mapping:** Slice C.
|
||||||
|
|
||||||
|
### P1 — Complete collection rebuilds during static publication
|
||||||
|
|
||||||
|
**Source**
|
||||||
|
|
||||||
|
`LandblockStaticPresentationPublisher` creates entity/snapshot dictionaries,
|
||||||
|
copies previous keys to a set, and sorts IDs/snapshots per publication.
|
||||||
|
|
||||||
|
**Impact**
|
||||||
|
|
||||||
|
Portal allocation and update-thread CPU scale with complete landblock contents.
|
||||||
|
|
||||||
|
**Recommendation**
|
||||||
|
|
||||||
|
Carry canonical ordered prepared arrays, use cursor-based publication receipts,
|
||||||
|
and publish versioned deltas rather than reconstructing complete collections.
|
||||||
|
|
||||||
|
**Plan mapping:** Slices C, E, and F.
|
||||||
|
|
||||||
|
### P1 — DAT resolution/cache churn
|
||||||
|
|
||||||
|
**Sources**
|
||||||
|
|
||||||
|
- `DatCollectionAdapter.ResolveId`.
|
||||||
|
- `MeshExtractor.ResolveId().ToList().OrderByDescending().FirstOrDefault()`.
|
||||||
|
- Small bounded typed-object cache plus serialized database access.
|
||||||
|
|
||||||
|
**Impact**
|
||||||
|
|
||||||
|
Repeated parsing on revisits, allocation from resolution candidates/LINQ, 4,614
|
||||||
|
lock contentions, and repeated invalid type probes.
|
||||||
|
|
||||||
|
**Recommendation**
|
||||||
|
|
||||||
|
Use typed non-allocating resolution, prepared compact metadata, negative/type
|
||||||
|
caches, and remove world render/streaming DAT reads after the pak cutover.
|
||||||
|
|
||||||
|
**Plan mapping:** Slices B and C.
|
||||||
|
|
||||||
|
### P2 — Whole-world render partition every frame
|
||||||
|
|
||||||
|
**Source**
|
||||||
|
|
||||||
|
`InteriorEntityPartition.Partition` scans every entity in every loaded
|
||||||
|
landblock each render frame. Caul had approximately 21,000 entities.
|
||||||
|
|
||||||
|
**Impact**
|
||||||
|
|
||||||
|
CPU cost grows with the complete resident world rather than visible cells and
|
||||||
|
changes. Reused lists reduce allocation but not traversal.
|
||||||
|
|
||||||
|
**Recommendation**
|
||||||
|
|
||||||
|
Maintain outdoor-static, per-cell, dynamic, translucent, selectable, and light
|
||||||
|
indices incrementally in a render-only scene database. Existing retail PView
|
||||||
|
remains the visible-cell authority.
|
||||||
|
|
||||||
|
**Plan mapping:** Slices F and G.
|
||||||
|
|
||||||
|
### P2 — Unconditional diagnostic and retained-UI work
|
||||||
|
|
||||||
|
**Sources**
|
||||||
|
|
||||||
|
- `WorldSceneDiagnosticsController` performs every-landblock AABB tests without
|
||||||
|
an active debug consumer.
|
||||||
|
- `GpuWorldState.LandblockBounds` uses an iterator.
|
||||||
|
- Retained UI reapplies anchors and walks the overlay tree every frame.
|
||||||
|
- Item cooldown UI scans complete item lists.
|
||||||
|
- Dialog handling snapshots open-order arrays.
|
||||||
|
|
||||||
|
**Impact**
|
||||||
|
|
||||||
|
Moderate capped cost, but it scales directly with uncapped FPS and adds small
|
||||||
|
steady allocations.
|
||||||
|
|
||||||
|
**Recommendation**
|
||||||
|
|
||||||
|
Reuse renderer facts, skip inactive diagnostics, add UI dirty-layout/event
|
||||||
|
registries, and replace frame iterators/snapshots with borrowed stable views.
|
||||||
|
|
||||||
|
**Plan mapping:** Slice H.
|
||||||
|
|
||||||
|
### P2 — High but bounded GPU/cache residency
|
||||||
|
|
||||||
|
**Sources**
|
||||||
|
|
||||||
|
- Object mesh and texture caches.
|
||||||
|
- Standalone and composite texture arrays.
|
||||||
|
- Fixed 65,536-instance particle CPU/GPU buffer.
|
||||||
|
- Deferred-alpha scratch arrays that grow to scene high-water.
|
||||||
|
|
||||||
|
**Impact**
|
||||||
|
|
||||||
|
633–818 MiB dedicated GPU memory during the route, plus driver/native
|
||||||
|
difference from tracked resources.
|
||||||
|
|
||||||
|
**Recommendation**
|
||||||
|
|
||||||
|
Introduce typed asset leases and unified accounting, measure atlas occupancy and
|
||||||
|
fragmentation, use generation-safe cost-aware eviction, and shrink or segment
|
||||||
|
large scratch buffers only with hysteresis.
|
||||||
|
|
||||||
|
Do not reduce view distance or particle range as the primary correction.
|
||||||
|
|
||||||
|
**Plan mapping:** Slice D.
|
||||||
|
|
||||||
|
### P3 — Smaller continuous allocations and background overhead
|
||||||
|
|
||||||
|
**Sources**
|
||||||
|
|
||||||
|
- Per-frame `RetailPViewFrameInput`.
|
||||||
|
- Yield iterators and periodic liveness sets.
|
||||||
|
- Full candidate light sorting.
|
||||||
|
- UDP send copies.
|
||||||
|
- Socket timeout exceptions.
|
||||||
|
|
||||||
|
**Impact**
|
||||||
|
|
||||||
|
Not responsible for the portal storm, but measurable at high uncapped FPS or
|
||||||
|
across many headless sessions.
|
||||||
|
|
||||||
|
**Recommendation**
|
||||||
|
|
||||||
|
Use reusable frame views, stable spans, spatial light candidates, span-based
|
||||||
|
socket sends, and cancellable blocking/async receive without exception-driven
|
||||||
|
timeouts.
|
||||||
|
|
||||||
|
**Plan mapping:** Slices H and J.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Recommended execution order
|
||||||
|
|
||||||
|
The audit supports this order:
|
||||||
|
|
||||||
|
1. Correct GPU/pass measurement and checkpoint accounting.
|
||||||
|
2. Complete EnvCell dedup and the full pak bake.
|
||||||
|
3. Cut world mesh/texture/Setup presentation preparation over to prepared data.
|
||||||
|
4. Add explicit CPU/staging/GPU residency ownership and accounting.
|
||||||
|
5. Replace portal budget bypasses with staged cost budgets and one reveal
|
||||||
|
generation.
|
||||||
|
6. Add the incremental render scene and delta submission.
|
||||||
|
7. Remove remaining per-frame UI/diagnostic/network allocation.
|
||||||
|
8. Extract the presentation-independent runtime for Linux/headless use.
|
||||||
|
|
||||||
|
The complete slice definitions and acceptance gates are in the linked modern
|
||||||
|
runtime architecture plan.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Reproduction
|
||||||
|
|
||||||
|
### Canonical route
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
dotnet build AcDream.slnx -c Release
|
||||||
|
|
||||||
|
$env:ACDREAM_TEST_USER = '<account>'
|
||||||
|
$env:ACDREAM_TEST_PASS = '<password>'
|
||||||
|
$env:ACDREAM_FRAME_PROF = '1'
|
||||||
|
|
||||||
|
.\tools\run-connected-r6-soak.ps1 -Repository (Get-Location).Path -SkipBuild
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not place credentials in committed logs or reports.
|
||||||
|
|
||||||
|
### Runtime counters
|
||||||
|
|
||||||
|
After identifying the client PID:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
dotnet-counters collect `
|
||||||
|
--process-id <pid> `
|
||||||
|
--counters System.Runtime `
|
||||||
|
--format csv `
|
||||||
|
--output dotnet-counters.csv
|
||||||
|
```
|
||||||
|
|
||||||
|
### CPU trace
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
dotnet-trace collect `
|
||||||
|
--process-id <pid> `
|
||||||
|
--profile cpu-sampling `
|
||||||
|
--duration 00:00:30 `
|
||||||
|
--output cpu.nettrace
|
||||||
|
```
|
||||||
|
|
||||||
|
### Final live heap
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
dotnet-gcdump collect --process-id <pid> --output final.gcdump
|
||||||
|
```
|
||||||
|
|
||||||
|
For comparable results:
|
||||||
|
|
||||||
|
- Use the same commit, Release configuration, route, and camera/input script.
|
||||||
|
- Record RDP/capped and local/uncapped modes separately.
|
||||||
|
- Gracefully close the client.
|
||||||
|
- Capture checkpoint JSON before interpreting process memory.
|
||||||
|
- Compare live heap, GC committed memory, tracked GPU bytes, and OS GPU memory
|
||||||
|
separately.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. Artifact retention policy
|
||||||
|
|
||||||
|
The local raw capture contained 96 files totaling approximately 157.9 MiB.
|
||||||
|
Heavy and environment-specific artifacts were deliberately excluded from Git:
|
||||||
|
|
||||||
|
- `.nettrace` and generated `.etlx`.
|
||||||
|
- `.gcdump`.
|
||||||
|
- Full `dotnet-counters.csv`.
|
||||||
|
- Complete process/GPU sample CSV.
|
||||||
|
- Full client stdout/stderr.
|
||||||
|
- Generated trace-reader project binaries.
|
||||||
|
- Process IDs, timestamps, absolute local paths, and session-specific logs.
|
||||||
|
|
||||||
|
Those files are valuable for local forensic work but poor Git artifacts: they
|
||||||
|
are binary or noisy, inflate permanent history, are difficult for code-review
|
||||||
|
tools to inspect, and can expose local/session metadata.
|
||||||
|
|
||||||
|
This report plus the compact JSON and checkpoint CSV preserve the benchmark's
|
||||||
|
method, provenance, measurements, interpretation, and decision-driving evidence
|
||||||
|
in a form another engineer or model can review directly.
|
||||||
Loading…
Add table
Add a link
Reference in a new issue