erik/acdream
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2
acdream/docs
History
Erik 39967e78bd perf #418: parallelize landblock builds across a striped worker pool 
Login publishes the 25x25 window at a flat 32 blocks/s (~27 s in the
tunnel). The reveal-timing probe A/B (695a27b4) showed the consumer
budget env ceilings change nothing, which was read as producer-limited:
one "acdream.streaming.worker" thread, ~31 ms/block. This replaces the
single worker with min(ProcessorCount-2, 8) workers, floor 1.

Design: striped/affinity dispatch. Each worker owns one unbounded lane
channel plus its own high/low priority queues; jobs route to
lane = ((id >> 16) * 2654435761) % N (the low word of a landblock id is
constant, so the id is mixed before reduction). Striping was chosen
over a shared queue + in-flight conflict tracker because it preserves
the per-landblock contract structurally rather than by bookkeeping:
every job for one id lives on one lane, so per-id enqueue order IS
execution and completion-arrival order, and the same-landblock
supersede rules (PromoteToNear removes queued LoadFar/Unload) keep
seeing every queued job for that id. Contract, point by point:

- Per-landblock ordering: same id -> same lane -> serial FIFO.
- ClearLoads: broadcast to every lane inside the same _inboxGate lock
  that serializes enqueues, so any load enqueued before
  ClearPendingLoads() returns sits ahead of its lane's ClearLoads copy
  in that lane's FIFO and is dropped at read time, exactly like the
  single-thread path. Already-dequeued builds still complete (now up
  to one per worker instead of one total); StreamingController's
  SweepCollapsed already unloads those uniformly.
- Priority: per-lane high/low split unchanged. Cross-lane, priority is
  not globally ordered (a lane cannot run another lane's job), which
  the contract permits; near-tier jobs hash-spread across lanes and
  are preferred within each.
- Outbox: SingleWriter flipped to false; nothing assumed single-writer
  (PublishResult already used TryWrite + an Interlocked backlog, and
  the consumer's peek->read head-stability holds because only the
  single reader ever moves the head). Cross-landblock arrival order
  was verified arbitrary-tolerant before relying on it:
  StreamingController.AdmitCompletions classifies each result
  independently into per-priority FIFOs (generation staleness +
  per-landblock retirement blocking); per-landblock arrival order is
  preserved by striping.
- Crash surface: per-worker. The first real crash publishes
  WorkerCrashed (prefixed "worker N:" in pools > 1), sets
  _workerFailure, completes every lane, and cancels the pool (a crash
  still ends all processing, as before); siblings that merely observe
  the closed lanes (ChannelClosedException) exit quietly instead of
  reporting spurious crashes; the outbox completes only when the LAST
  worker exits so no in-flight completions are dropped.
- Disposal: joins every worker under the same _disposeGate; Start
  stays idempotent and dispose-serialized.

Thread-safety audit of the production build closures
(SessionPlayerComposition), per shared object:

- DatCollection (every read in LandblockBuildFactory.BuildLocked:
  LandblockLoader.Load, SceneryGenerator.Generate, SetupMesh.Flatten,
  CellMesh.Build, GfxObjBounds.Get, GfxObjDegradeResolver): NOT
  thread-safe; already serialized under the shared _datLock, which
  BuildLocked holds for the whole read transaction. Unchanged; the
  probe run measured hold 0-13 ms / wait <= 12 ms during the login
  window, so the lock is not the new bottleneck and the build was NOT
  serialized beyond it.
- PakPreparedAssetSource / PakReader (BuildPreparedCollisionClosure,
  outside the lock): immutable TOC array + read-only
  MemoryMappedViewAccessor random-access reads + ConcurrentDictionary
  verdict caches - safe for N concurrent readers (Slice I3 design;
  the headless SharedPreparedCollisionCache wrapper is fully
  lock-protected).
- LandblockMesh.Build (outside the lock): pure math over the dat
  record + the composition-time height table + the immutable
  TerrainBlendingContext record; the shared SurfaceCache is a
  ConcurrentDictionary and BuildSurface is deterministic, so its
  lookup-or-build race is last-write-wins-benign (the code already
  documented exactly this).
- PhysicsDiagnostics probe statics: read-only bools + thread-safe
  Console writes.

MEASURED OUTCOME (gate 4): the timing acceptance did NOT pass, and per
the task contract that is reported, not tuned around. With 8 workers
on this 16-core machine all 625 builds complete in ~203 ms
(ACDREAM_PROBE_TELEPORT BUILD lines t=3475390..3475593) - the producer
is off the critical path - but loaded= still advances at exactly
+32/1000 ms and SUMMARY totalMs measured 27395 and 27503 across two
runs (baseline 26728). The 32/s pacer is in the consumer
admission/publication path and is not governed by the
StreamingWorkBudgetOptions env ceilings. #418 stays IN-PROGRESS on the
consumer side; see docs/ISSUES.md for the evidence chain.

Tests: per-landblock ordering under 4-worker contention, cross-lane
ClearLoads drop, per-lane near-before-far preference, pool-of-1 serial
equivalence, disposal joining every worker, lane-spread guard, and
worker-count validation (LandblockStreamerPoolTests). Two existing
tests asserted a GLOBAL cross-landblock execution order - a serial
implementation detail, not the contract - and now pin workerCount: 1
with justification comments (LoadNear_OvertakesQueuedFarLoads,
TwoQueuedLoads_RetainTheirDistinctOriginAndGeneration).

Gates: Release build 0 errors; App suite 5575 passed / 3 skipped
(5568 + 7 new); Runtime suite 1756/0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 19:34:20 +02:00
..
architecture Merge campaign-newline-fix: retail source-level escape normalization 2026-08-17 14:21:10 +02:00
audit docs: update audit — Sprint 1 items verified (sequence counters + scenery LCG) 2026-04-13 13:51:39 +02:00
plans docs: Campaign CC connected gate PASSED — campaign CLOSED user-accepted 2026-08-16 2026-08-16 19:13:36 +02:00
reference docs(reference): preserve local ACE command catalog 2026-07-27 00:03:58 +02:00
reports docs: the overnight consolidation report, open questions first 2026-07-29 03:54:43 +02:00
research fix(ui): night-round review — F1 real PlaceMarkerOnMap formula 2026-08-17 04:19:29 +02:00
superpowers feat(launcher): complete Campaign LA11 pre-gate support 2026-08-15 00:02:04 +02:00
bugs.md docs: reconcile project status and navigation 2026-07-20 13:00:41 +02:00
ISSUES.md perf #418: parallelize landblock builds across a striped worker pool 2026-08-17 19:34:20 +02:00
README.md docs: reconcile project state and refresh README 2026-07-27 12:53:31 +02:00

README.md

acdream documentation map

This page is the entry point for project documentation. It distinguishes current sources of truth from implementation history so an old plan or issue banner cannot silently override the current program state.

Current snapshot — 2026-07-27

  • Milestone state: M3, “Cast a spell,” landed 2026-07-21. M4, “Live in the world,” is active.
  • M4 gameplay program: resume the pre-M4 world-interaction completion program. Favorite-spell overflow, status Use/Assess, and the complete assessment surface are user-accepted. Equipped-child picking and vendor browse/transactions remain Slices 4–6.
  • Structural/runtime state: all eight GameWindow decomposition slices, Modern Runtime Slices A–J, and the connected visual/lifecycle gates are complete. GameWindow is a 1,622-line composition/callback shell. AcDream.Runtime.GameRuntime owns canonical session, entity/object, gameplay, movement, physics, projectile, environment, and portal state; graphical and no-window hosts borrow the same owner graph.
  • Headless state: Slice K is complete. AcDream.Headless is a presentation-free Windows/Linux host with deterministic commands/events, shared immutable content, multi-session isolation, reconnect, resource telemetry, and 1/5/10/30-session gates. The final two-account native-Linux soak completed ten minutes, logged out through ACE, and converged every ownership ledger.
  • Linux graphical state: Slice L0 and the L1 implementation checkpoint are complete at 66f114b2 and 11501d52. Native Windows passes the active modern-GL/audio/window smoke. WSLg X11/Wayland correctly reject their missing GL_ARB_bindless_texture. Physical-Linux validation and L2–L6 are explicitly deferred; resume at the supported AMD/NVIDIA L1 gate.
  • Completed gameplay gates: R6 locomotion/collision/projectile/teleport/ radar, two-client portal-out/materialization, indoor prepared collision, loot ordering, local/remote ground drops, and selection-marker lifetime.
  • Separate visual verification: issue #225, the shared-alpha lifestone/particle result; its connected resource-lifetime and performance routes pass.
  • Carried behaviour debt: issue #153 (far teleport onto an unstreamed edge), #116 (narrowed slide response), #235 (capped/RDP jump cadence), and the active temporary-stopgap rows in the divergence register.
  • Divergence audit: 189 active rows — IA 18, AD 38, AP 91, TS 38, and UN 4 — plus retained struck/retired historical rows such as TS-37.
  • Latest automated baseline: the Release build succeeds with the 17 test-project warnings tracked by issue #228; 8,826 tests pass and five are intentionally skipped. App passes 3,763 / 3 skips. The L1 Windows supported smoke and WSLg X11/Wayland negative-capability reports all end with zero window/GL/input/audio ownership.

Sources of truth

Read these in this order when deciding what to do next:

  1. plans/2026-05-12-milestones.md — the active playable outcome, freeze boundaries, and visual gates.
  2. plans/2026-04-11-roadmap.md — strategic phase ledger: shipped, active, deferred, and future work.
  3. ISSUES.md — tactical defects and small follow-ups. The status inside an issue is authoritative; physical order is not.
  4. architecture/retail-divergence-register.md — every known place runtime behavior can differ from retail.
  5. architecture/acdream-architecture.md and architecture/code-structure.md — ownership, dependency, update-thread, and extraction rules.
  6. architecture/worldbuilder-inventory.md — rendering/DAT code already owned in-tree versus mechanisms still ours to port.

If these disagree, milestones control the current outcome, the roadmap controls work ordering, the issue status controls the individual defect, and the architecture documents control implementation shape. Reconcile the stale document in the same change; do not leave both claims standing.

Research and implementation records

  • research/named-retail/ is the primary retail oracle: named pseudo-C, headers, symbols, and types from the Sept 2013 build.
  • research/decompiled/ is the older Ghidra fallback.
  • research/ contains focused pseudocode, traces, fixtures, and gate reports. A dated research note records evidence; it does not become a new roadmap.
  • superpowers/specs/ and superpowers/plans/ are per-slice design and execution records. Completed plans remain historical.
  • audit/ contains completion and conformance audits.
  • reference/ace-commands.md preserves the local ACE server's complete in-game command catalog and points to the authoritative per-command help surface.

Durable memory

  • ../claude-memory/MEMORY.md indexes the live subsystem memories and the render/physics digests. Read a domain digest before changing that subsystem, especially its DO-NOT-RETRY table.
  • ../memory/ contains stable engineering references such as the modern rendering pipeline, two-tier streaming, and toolchain notes.

Memory accelerates recall; it does not outrank the canonical documents above. When current truth changes, update the relevant canonical document and distill only the durable lesson into memory.

Historical and deprecated documents

  • bugs.md is the April 2026 bug snapshot. It is preserved for archaeology and is not an active ledger.
  • Dated plans and specs describe the decision at that time. Their completion wording is historical unless the current milestone/roadmap explicitly links the item as active.
  • Old R1→R8 architecture sequencing is superseded. Current execution comes from the milestones and strategic roadmap.

Documentation maintenance rules

  • Update milestone, roadmap, issue, divergence, architecture, and memory claims in the same commit when a shipped change affects them.
  • Keep one issue ID per defect. Narrow an issue in place; do not reuse another issue's number as a shorthand.
  • Mark automated, connected, and visual gates separately. An automated pass is not a visual acceptance, and an RDP throughput sample is not a local-display visual comparison.
  • Preserve research history, but remove stale “current/next” claims from living documents once the state advances.