erik/acdream
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
acdream/docs/architecture/worldbuilder-inventory.md
Erik 932f904e00 fix(pipeline): MP1a - sink delegate restores immediate side-stage enqueue (exception-path faithfulness) 
Coordinator-directed follow-up. The buffer-and-drain seam diverged from
the original on the exception path: pre-MP1a, CollectEmittersFromScript
enqueued particle-preload meshes DIRECTLY into _stagedMeshData
mid-Prepare, so preloads staged before a later throw in the same
Prepare* call (reachable via PrepareEnvCellMeshData side-staging during
its StaticObjects loop, then PrepareCellStructMeshData throwing on a
malformed-dat texture decode) were already safely enqueued. The drain
version only flushed after a successful return — on throw, entries
stranded on the shared extractor until an unrelated successful call
flushed them, and were silently dropped on dispose.

Fix: MeshExtractor takes an Action<ObjectMeshData>? sideStagedSink
constructor parameter; the two CollectEmittersFromScript sites become
_sideStagedSink?.Invoke(meshData) — the original code shape (immediate
hand-off) at those exact lines. ObjectMeshManager wires the sink to
_stagedMeshData.Enqueue, restoring the original immediate-enqueue
semantics including on mid-Prepare throw. _sideStaged buffer,
DrainSideStaged(), and the ProcessQueueAsync drain loop are deleted.
The MP1b bake tool passes its own collector.

Inventory doc updated: MP1a note now records the sink seam and the
Content-owned upload enums, so its no-behavior-change claim is accurate.

dotnet build green; full test suite 4059 passed / 0 failed / 4 skipped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 20:39:42 +02:00

17 KiB
Raw Blame History

WorldBuilder Inventory — what we extracted, adapted, or left behind

Phase O shipped 2026-05-21. The ~33 WB files we actually use have been extracted into our tree. references/WorldBuilder/ stays as a read-reference only — nothing in src/AcDream.* references it as a project dependency. DatCollection is now the only dat reader in process.

Use this document to:

  1. Know where our extracted code lives (look for the "Extracted to" column / notes in each section below).
  2. Know what WB still has that we haven't needed yet — grep references/WorldBuilder/ if you ever need to add something.
  3. Know what WB never had (the 🔴 list) — those are always ours.

Pre-O status (archived for context): As of Phase N.4 (2026-05-08) acdream relied heavily on WorldBuilder as a project reference for rendering and dat-handling. WorldBuilder is MIT-licensed, verified by visual inspection to render the AC world correctly (terrain, scenery, slabs, dungeons, slopes, particles), and uses the same Silk.NET + .NET stack we target.

Post-O integration model: Extracted WB code lives in two locations in our tree (see CLAUDE.md for the full breakdown):

  • src/AcDream.Core/Rendering/Wb/ — pure helpers (no GL): TerrainUtils, TerrainEntry, RegionInfo, SceneryHelpers, TextureHelpers.
  • src/AcDream.App/Rendering/Wb/ — GL infrastructure + mesh pipeline: ObjectMeshManager, WbMeshAdapter, WbDrawDispatcher, texture cache, shader infra, EnvCell/portal/scenery/terrain-blending pipeline classes.

DatCollectionAdapter bridges our IDatCollection to the IDatReaderWriter interface WB's internals expect (O-D7 fallback; ObjectMeshManager has 26 internal _dats.* call sites — above the 20-site inline-swap threshold).

MP1a (2026-07-05): CPU mesh-extraction half moved to AcDream.Content. The GL-free portion of the former ObjectMeshManager — dat read → polygon walk → vertex/index build → inline BCn/palette texture decode → ObjectMeshData — is now MeshExtractor in a new src/AcDream.Content/ assembly (no Silk.NET dependency), so the MP1b bake tool can run the exact same extraction code offline without an OpenGL context. This was a mechanical, verbatim move (namespace + visibility only) per docs/superpowers/plans/2026-07-05-mp1a-content-extraction.md — no behavior change, no divergence-register row.

  • src/AcDream.Content/MeshExtractor.cs — the Prepare* family (PrepareMeshData, PrepareSetupMeshData, PrepareGfxObjMeshData, PrepareEnvCellMeshData, PrepareCellStructMeshData, PrepareCellStructEdgeLineData) + private helpers (CollectParts, CollectEmittersFromScript, ComputeBounds, BuildPolygonIndices, BuildCellStructPolygonIndices) and the decoded-texture cache / ThreadLocal<BcDecoder> that back them.
  • src/AcDream.Content/ObjectMeshData.cs — the CPU-side boundary records: VertexPositionNormalTexture, StagedEmitter, ObjectMeshData, MeshBatchData, TextureBatchData.
  • src/AcDream.Content/TextureKey.cs — the atlas dedup key, lifted out of the GL-owning TextureAtlasManager (which stays in App and now references the lifted struct).
  • src/AcDream.Content/UploadFormats.cs — Content-owned UploadPixelFormat/UploadPixelType enums carried by MeshBatchData/TextureBatchData instead of Silk.NET.OpenGL.PixelFormat/PixelType (Content must stay Silk.NET-free — the bake tool must not ship GL binaries). Underlying values are the GL ABI constants, numerically identical to the Silk.NET members; App casts at its single upload boundary (the AddTexture call in UploadGfxObjMeshData) via a lifted nullable enum conversion — value- and null-preserving.
  • src/AcDream.Content/IDatReaderWriter.cs, EdgeLineBuilder.cs — GL-free dependencies of the extractor, moved (namespace-only) alongside it.
  • Side-stage sink seam: CollectEmittersFromScript pre-loads particle GfxObj meshes mid-extraction and, pre-MP1a, enqueued them directly onto ObjectMeshManager._stagedMeshData. The extractor now takes an Action<ObjectMeshData>? sideStagedSink constructor parameter; App wires it to _stagedMeshData.Enqueue, preserving the original immediate-enqueue semantics exactly — including on a mid-Prepare* throw (preloads staged before a malformed-dat texture-decode exception survive, as they always did). The MP1b bake tool passes its own collector.
  • Stays in src/AcDream.App/Rendering/Wb/: ObjectMeshManager (now a thin wrapper owning the staged-queue/worker-pool/Dispose-quiesce lifecycle and all GL upload — constructs one MeshExtractor and delegates every former Prepare* call site to it), ObjectRenderData/ObjectRenderBatch (hold a GL TextureAtlasManager field), TextureAtlasManager, DatCollectionAdapter (concrete DatCollection-backed implementation of the now-Content-namespaced IDatReaderWriter), GeometryUtils (used only by App-side raycasting, not by extraction), AcSurfaceMetadata/ AcSurfaceMetadataTable (not on the extraction path), Building.cs (explicitly out of scope).
  • AcDream.Core is untouched; AcDream.Content references AcDream.Core (for TextureHelpers, Sphere/BoundingBox via Chorizite.Core.Lib); AcDream.App references AcDream.Content. AcDream.Core does NOT reference AcDream.Content (one-way dependency, per Code Structure Rule 2).
  • Reason: MP1 (docs/superpowers/specs/2026-07-05-modern-pipeline-design.md §6.1) — the bake tool needs the identical mesh/texture extraction code running with no GL context, so baked pak output and live-client output stay byte-identical.

Workflow: Before re-implementing any AC-specific rendering or dat-handling algorithm, check this inventory first. If we already extracted it (🟢 sections), it's in src/AcDream.App/Rendering/Wb/ — use our copy. If WB has it but we haven't extracted it yet, grep references/WorldBuilder/ and extract as needed. Retail decomp remains the oracle for things WB never had (🔴 list).

Attribution: WorldBuilder is MIT-licensed. NOTICE.md includes WB attribution.

Read-reference layout (under references/WorldBuilder/, not project-referenced)

  • Chorizite.OpenGLSDLBackend/ — full OpenGL renderer (Silk.NET). The components we use are extracted into src/AcDream.App/Rendering/Wb/.
  • WorldBuilder.Shared/ — data models, dat parsers, landscape module. The helpers we use are extracted into src/AcDream.Core/Rendering/Wb/.
  • WorldBuilder/ — Avalonia desktop app shell (not taken).
  • WorldBuilder.{Windows,Linux,Mac}/ — platform entry points (not taken).
  • WorldBuilder.Server/ — collab editing backend (not taken).
  • Tests/ + WorldBuilder.Shared.Benchmarks/ — test harness (study only).

Upstream NuGet dependencies (these stay as NuGet packages, we don't vendor them):

Package Version Purpose
Chorizite.Core 0.0.18 Plugin framework — contains Chorizite.Core.Lib.BoundingBox, Chorizite.Core.Render.* interfaces used by every render manager
Chorizite.DatReaderWriter 2.1.x dat parsing (we already use 2.1.7)
Chorizite.DatReaderWriter.Extensions 1.1.x extra dat helpers
BCnEncoder.Net 2.2.x DXT decode (we already use)
SixLabors.ImageSharp 3.1.x image loading
Silk.NET.OpenGL + Silk.NET.SDL 2.23.x GL + windowing (we use Silk's own windowing, they use SDL)
MP3Sharp 1.0.5 MP3 decode

🟢 RENDERING — take wholesale or adapt

These are what makes WB "perfect". Anything in this section, we should use from WB rather than re-implement.

Terrain

Component What it does
TerrainRenderManager Full pipeline (per-chunk GPU buffers, draw orchestration)
LandSurfaceManager Texture blending atlas (palCode, alpha masks, road overlays)
TerrainGeometryGenerator Heightmap → mesh, normals, OnRoad, GetHeight, GetNormal
TerrainChunk 16×16 landblock chunk geometry
TextureAtlasManager Texture atlas builder
VertexLandscape Terrain vertex format

Scenery (procedural placement: trees, bushes, rocks, fences)

Component What it does
SceneryRenderManager Generate + render per-vertex scenery
SceneryHelpers Displace / RotateObj / ScaleObj / ObjAlign / CheckSlope
SceneryInstance Per-spawn instance data

Static objects (buildings, slabs, props — Setup + GfxObj + ObjDesc)

Component What it does
StaticObjectRenderManager Master pipeline for static objects
ObjectRenderManagerBase + BaseObjectRenderManager Common render base
ObjectMeshManager Mesh extraction from Setup/GfxObj, ObjDesc application

Dungeons / interiors

Component What it does
EnvCellRenderManager Dungeon interior cell geometry
PortalRenderManager Portal traversal / visibility

Sky + atmosphere

Component What it does
SkyboxRenderManager Skybox rendering
ParticleEmitterRenderer + ParticleBatcher + ActiveParticleEmitter Particle systems (sky particles, weather, magic)

Visibility / culling

Component What it does
VisibilityManager + VisibilitySnapshot Frustum + cell visibility
Frustum Frustum-cull math

Other rendering helpers

Component What it does
MinimapRenderer Top-down minimap
GlobalMeshBuffer Shared GPU mesh buffer
GpuResourceManager GPU resource lifecycle
InstanceData Instanced draw data
TextureHelpers INDEX16, P8, BGRA, DXT decode + alpha (canonical port)
DebugRenderer + DebugRendererLineDrawer + EdgeLineBuilder Debug primitives

Shaders (22 total)

Located at Chorizite.OpenGLSDLBackend/Shaders/:

Landscape.{vert,frag} · StaticObject.{vert,frag} · StaticObjectModern.{vert,frag} · Particle.{vert,frag} · PortalStencil.{vert,frag} · Outline.{vert,frag} · Simple3D.{vert,frag} · InstancedLine.{vert,frag} · Text.{vert,frag} · UI.{vert,frag} · Gizmo.{vert,frag} (editor-only)

🟢 LOW-LEVEL GL / FRAMEWORK — take or replace with our own

Either take WB's wrappers wholesale, or keep our own and adapt the render managers to use ours. These wrappers are stateless or near-stateless and are the easiest to swap.

Component What it does
OpenGLGraphicsDevice Silk.NET.OpenGL wrapper
OpenGLRenderer Render orchestration
GLSLShader Shader compile/link/uniforms
GLHelpers + GLStateScope GL state utility
ManagedGLFrameBuffer / ManagedGLIndexBuffer / ManagedGLTexture / ManagedGLTextureArray / ManagedGLUniformBuffer / ManagedGLVertexArray / ManagedGLVertexBuffer GL resource wrappers
TextureParameters Sampler config
GpuMemoryTracker Memory tracking
Camera2D / Camera3D / CameraBase / ICamera / CameraController Camera primitives
GameScene + SingleObjectScene + SceneData + ModernRenderData + RenderPass Scene / pass structures

🟢 GEOMETRY / MATH UTILS — take wholesale

Component File
TerrainUtils (OnRoad, GetNormal, GetHeight, GetRoad, palCode) WorldBuilder.Shared/Modules/Landscape/Lib/TerrainUtils.cs
TerrainCacheManager …/Lib/TerrainCacheManager.cs
TerrainRaycast …/Lib/TerrainRaycast.cs
GeometryUtils WorldBuilder.Shared/Lib/GeometryUtils.cs
RaycastingUtils (ray-vs-sphere/AABB/triangle) WorldBuilder.Shared/Lib/RaycastingUtils.cs
DoubleNumerics (double-precision Vector/Matrix) WorldBuilder.Shared/Lib/DoubleNumerics.cs
DatUtils WorldBuilder.Shared/Lib/DatUtils.cs
BoundingBoxExtensions Chorizite.OpenGLSDLBackend/Lib/BoundingBoxExtensions.cs

🟢 DATA MODELS — take selectively

Component What it does
RegionInfo Landblock metadata wrapper (LandblockSizeInUnits, CellSizeInUnits, etc.)
TerrainEntry Per-vertex terrain (Type/Scenery/Road/Height)
MergedLandblock Merged dat data
CellSplitDirection SW-NE vs NE-SW
Cell Generic cell wrapper
ObjectId Object identifier
Position World position
ACEnums AC-specific enums
WbBuildingPortal / WbCellPortal Portal structures
BuildingObject Building data

🟡 EDITOR-ONLY — leave behind / delete in fork

These exist for the editor experience and have no place in a game client. Delete in fork.

  • Modules/Landscape/Tools/* — BrushTool, BucketFillTool, RoadLineTool, RoadVertexTool, InspectorTool, ObjectManipulationTool, Gizmo* (DragHandler, HitTester, Renderer, State), TexturePainting*, SceneRaycaster, LandscapeBrush, LandscapeToolBase, LandscapeToolContext, IToolSettingsProvider, ILandscapeBrush, ILandscapeEditorService, ILandscapeRaycastService, ILandscapeTool, ITexturePaintingTool
  • Modules/Landscape/Commands/* — undo/redo command pattern for editor (Add/Delete/Move/Rename/Reorder/etc.)
  • LandscapeDocument + LandscapeLayer + LandscapeLayerGroup + LandscapeChunk + LandscapeLayerChunk + LandscapeLayerBase — editor document model
  • Modules/Landscape/Models/TerrainPatch* + LandblockChangedEventArgs — editor mutation events
  • Modules/Landscape/Services/ILandscapeCacheService + ILandscapeDataProvider + ILandscapeObjectService + impls — editor data flow
  • All Migrations/* — SQLite schema migrations (project file format)
  • Repositories/* + Services/* — project storage, dat repository, AceDb, SignalR sync, document manager, undo stack, world coordinates, keyword DB, project migration, semantic kernel AI helpers
  • Hubs/* — collaborative editing via SignalR
  • StaticObject (editor model) — replace with our own scene-state data model fed from network
  • BackendGizmoDrawer + GizmoRenderer — editor gizmos
  • ProjectStructures, IProject, Project — editor project files
  • KeyBinding — editor input binding
  • ViewportInputEvent[Extensions] — editor viewport input
  • EditorState — editor state container

🟡 AUDIO / FONT — we already have alternatives

Keep ours; don't take theirs.

  • AudioPlaybackEngine — uses MP3Sharp. We have OpenAL.
  • FontRenderer — uses ImageSharp. We have BitmapFont/StbTrueTypeSharp + ImGui.

🔴 NOT IN WORLDBUILDER — port from retail decomp ourselves

WorldBuilder is a dat editor; it does not have:

  • Network protocol — UDP framing, ISAAC, packet codec, ACE message layer (we have this; oracle is references/holtburger)
  • Physics — collision (CPhysicsObj transitions, BSP queries, sphere sweeps), step-up, walkable validation (we have partial; oracle is the retail decomp at docs/research/named-retail/)
  • Animation — motion sequencer, cycle/non-cycle parts, animation frame interpolation (we have this; oracle is retail decomp)
  • Movement — local player WASD → MoveToState wire, remote-entity motion via UpdateMotion + dead-reckoning (we have this; oracle is references/holtburger + retail decomp)
  • Game UI — chat, vitals, inventory, spell book, allegiance, options (we have this; ImGui-based today, custom-toolkit later)
  • Plugin API — IGameState, IEvents, IActions, IPacketPipeline, IOverlay (we have this — acdream-unique)
  • Game events — combat, allegiance, spell casting, quest events (we have this; oracle is ACE for opcodes + retail for client behavior)
  • Audio — OpenAL pipeline, sound triggers (we have this)
  • TurbineChat + slash commands (we have this)
  • Login + character selection flow (we have this)

What this means for the workflow (post-Phase O)

The CLAUDE.md "grep named → decompile → verify → port" workflow stays the rule for everything in the 🔴 list (network, physics, animation, movement, UI, plugin, audio, chat).

For anything in 🟢 that we've already extracted: the code is in our tree at src/AcDream.{Core,App}/Rendering/Wb/. Read it there — don't grep references/WorldBuilder/ unless you want to compare against the original. Re-porting from retail decomp when we already have a tested port is still how we'd get the scenery edge-vertex bug back.

For anything in 🟢 that we have NOT yet extracted: grep references/WorldBuilder/ to find the source, then extract it using the Phase O pattern (verbatim copy → adapt constructor to accept IDatCollection via DatCollectionAdapter where needed → add to src/AcDream.App/Rendering/Wb/). Do NOT add a new project reference back to WorldBuilder.Shared or Chorizite.OpenGLSDLBackend — Phase O permanently removed those.

When we discover a behavior mismatch with retail (rare — the extracted code is the same as the original), the resolution is: reconcile extracted code ↔ retail decomp ↔ holtburger ↔ ACE ↔ ACViewer (the existing reference hierarchy in CLAUDE.md). Our extracted code ranks at the top of that hierarchy for anything 🟢.