diff --git a/CLAUDE.md b/CLAUDE.md index 469b95c..1731668 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -25,6 +25,19 @@ single source of truth for how the client is structured. All work must align with this document. When the architecture doc and reality diverge, update one or the other โ€” never leave them out of sync. +**WorldBuilder is acdream's rendering + dat-handling base** as of +2026-05-08. Before re-implementing any AC-specific rendering or +dat-handling algorithm, **read `docs/architecture/worldbuilder-inventory.md` +FIRST**. If WorldBuilder has it, port from WorldBuilder (or call into +our fork once wired up), not from retail decomp. WorldBuilder is +MIT-licensed, verified to render the world correctly, and uses the same +Silk.NET stack we target. Re-porting from retail decomp when WB already +has a tested port is how subtle bugs (the scenery edge-vertex bug, the +triangle-Z bug) keep slipping in. Retail decomp remains the oracle for +network, physics, animation, movement, UI, plugin, audio, chat โ€” see +the inventory doc's ๐Ÿ”ด list for the full scope of "we still write this +ourselves". + **Execution phases:** R1โ†’R8 in the architecture doc. Each phase has clear goals, test criteria, and builds on the previous. Don't skip phases. @@ -625,11 +638,18 @@ these, ideally all four: for the palette-indexed formats. See `ACViewer/Render/TextureCache.cs::IndexToColor` for the canonical subpalette overlay algorithm. -- **`references/WorldBuilder/`** โ€” C# + Silk.NET dat editor. Exact-stack - match to acdream for rendering approaches: terrain blending, texture - atlases, shader patterns. Most useful for "how do I do this GL thing - with Silk.NET on net10 idiomatically?" Less useful for protocol or - character appearance (dat editor, not game client). +- **`references/WorldBuilder/`** โ€” **acdream's rendering + dat-handling + BASE (not just a reference).** As of 2026-05-08 acdream is moving to + fork WorldBuilder upstream and depend on the fork for terrain, + scenery, static objects, EnvCells, portals, sky, particles, texture + decoding, mesh extraction, visibility/culling. WorldBuilder is + MIT-licensed, exact-stack match (Silk.NET + .NET), and verified to + render the world correctly. **Before re-porting any rendering or + dat-handling algorithm from retail decomp, check + `docs/architecture/worldbuilder-inventory.md` first.** If WB has it, + use WB's port. If WB doesn't have it (network, physics, animation, + movement, UI, plugin, audio, chat), port from retail decomp as + before. - **`references/Chorizite.ACProtocol/`** โ€” clean-room C# protocol library generated from a protocol XML description. Useful sanity check on field order, packed-dword conventions, type-prefix handling. The @@ -684,12 +704,15 @@ decompiled client code and would have fixed it in minutes. | Domain | Primary Oracle | Secondary | Notes | |--------|---------------|-----------|-------| -| **Any AC-specific algorithm** | **`docs/research/named-retail/`** (PDB-named decomp + verbatim retail header structs from Sept 2013 EoR build) | the existing references below | The retail client itself, fully named. 18,366 functions + 5,371 struct types + 1.4 M lines of pseudo-C in one searchable tree. Beats every other reference for "what does the real client do." | -| **Terrain** (split direction, height sampling, palCode, vertex position, normals) | **ACME `ClientReference.cs`** โ€” decompiled retail client with exact offsets | ACME `TerrainGeometryGenerator.cs` (matches the mesh index buffer) | WorldBuilder original is SUPERSEDED for terrain algorithms. AC2D confirms the same formula. | -| **Terrain blending** (texture atlas, alpha masks, road overlays) | **ACME `LandSurfaceManager.cs`** | WorldBuilder original `LandSurfaceManager.cs` (same code, less tested) | Both use the same TexMerge pipeline. ACME has conformance tests. | -| **GfxObj / Setup rendering** (mesh extraction, multi-part assembly, ObjDesc) | **ACME `StaticObjectManager.cs`** โ€” includes CreaturePalette, GfxObjRemapping, HiddenParts | ACViewer `Render/` namespace | ACME has the complete creature appearance pipeline in one file. | -| **Texture decoding** (INDEX16, P8, DXT, BGRA, alpha) | **ACME `TextureHelpers.cs`** | ACViewer `Render/TextureCache.cs` (palette overlay = `IndexToColor`) | For subpalette overlay specifically, ACViewer's `IndexToColor` is the canonical algorithm. | -| **EnvCell / dungeon rendering** (cell geometry, portal visibility, collision mesh) | **ACME `EnvCellManager.cs`** โ€” portal traversal, mixed landblock detection, collision cache | ACViewer `Physics/Common/EnvCell.cs` | ACME is significantly more complete than original WorldBuilder for dungeons. | +| **Any AC-specific algorithm** | **`docs/research/named-retail/`** (PDB-named decomp + verbatim retail header structs from Sept 2013 EoR build) | the existing references below | The retail client itself, fully named. 18,366 functions + 5,371 struct types + 1.4 M lines of pseudo-C in one searchable tree. Beats every other reference for "what does the real client do." Use for everything in the ๐Ÿ”ด list (network, physics, animation, movement, UI, plugin, audio, chat). | +| **Terrain** (split direction, height sampling, palCode, vertex position, normals) | **WorldBuilder `TerrainGeometryGenerator.cs` + `TerrainUtils.cs`** | retail decomp for cross-check | WB is acdream's terrain base. ACME's port is older/SUPERSEDED by WB. | +| **Terrain blending** (texture atlas, alpha masks, road overlays) | **WorldBuilder `LandSurfaceManager.cs`** | ACME `LandSurfaceManager.cs` (same algo, less complete) | WB is acdream's blending base. | +| **Scenery** (procedural placement: trees, bushes, rocks, fences) | **WorldBuilder `SceneryRenderManager.cs` + `SceneryHelpers.cs`** | retail decomp `CLandBlock::get_land_scenes` | WB is acdream's scenery base. Re-porting from retail decomp is what caused the edge-vertex bug. | +| **GfxObj / Setup rendering** (mesh extraction, multi-part assembly, ObjDesc) | **WorldBuilder `StaticObjectRenderManager.cs` + `ObjectMeshManager.cs`** | ACME `StaticObjectManager.cs` (includes CreaturePalette, GfxObjRemapping, HiddenParts โ€” useful for character appearance which WB doesn't cover) | WB for static objects, ACME for character appearance. | +| **Texture decoding** (INDEX16, P8, DXT, BGRA, alpha) | **WorldBuilder `TextureHelpers.cs`** | ACME `TextureHelpers.cs`; ACViewer's `IndexToColor` is canonical for subpalette overlay | WB is acdream's decode base. | +| **EnvCell / dungeon rendering** (cell geometry, portal visibility, collision mesh) | **WorldBuilder `EnvCellRenderManager.cs` + `PortalRenderManager.cs`** | ACME `EnvCellManager.cs` (more complete for collision); ACViewer `Physics/Common/EnvCell.cs` | WB is acdream's geometry base; ACME for collision until ported. | +| **Particles / sky** (particle systems, weather, sky particles) | **WorldBuilder `SkyboxRenderManager.cs` + `ParticleEmitterRenderer.cs` + `ParticleBatcher.cs`** | retail decomp | WB is acdream's particle base. | +| **Visibility / culling** (frustum, cell visibility) | **WorldBuilder `VisibilityManager.cs` + `Frustum.cs`** | โ€” | WB. | | **Network protocol** (wire format, packet framing, fragment assembly, ISAAC) | **holtburger** `crates/holtburger-session/` | AC2D `cNetwork.cpp` (simpler, good for cross-check) | ACE shows the server side; holtburger + AC2D show the client side. | | **Client behavior** (what to send when, login flow, ack pattern, keepalive) | **holtburger** `crates/holtburger-core/src/client/` | AC2D `cNetwork.cpp` + `cInterface.cpp` | holtburger is the most complete; AC2D is simpler but confirmed working. | | **Movement** (MoveToState format, AutonomousPosition, sequence counters, speed) | **holtburger** `client/movement/` | AC2D `cNetwork.cpp:2592-2664` (0xF61C format) | See `docs/research/2026-04-12-movement-deep-dive.md` for the full cross-reference. | diff --git a/docs/architecture/worldbuilder-inventory.md b/docs/architecture/worldbuilder-inventory.md new file mode 100644 index 0000000..68144c9 --- /dev/null +++ b/docs/architecture/worldbuilder-inventory.md @@ -0,0 +1,250 @@ +# WorldBuilder Inventory โ€” what we take, adapt, or leave + +**Status:** load-bearing reference. As of 2026-05-08 acdream's strategy is +to **rely heavily on WorldBuilder** for rendering and dat-handling rather +than re-port retail algorithms ourselves. WorldBuilder is MIT-licensed, is +verified by visual inspection to render the AC world correctly (terrain, +scenery, slabs, dungeons, slopes, particles), and uses the same Silk.NET ++ .NET stack we already target. + +**Integration model:** **fork upstream WorldBuilder** at +`github.com/Chorizite/WorldBuilder`, depend on our fork, delete editor-only +code, expose hooks for our network state to feed scene data in. Sync with +upstream via merge so we inherit fixes. This document tells you, before +you write code, whether the thing you're about to port already exists in +WorldBuilder. + +**Workflow change:** Before re-implementing any AC-specific rendering or +dat-handling algorithm, **check this inventory first**. If WorldBuilder +has it, port from WorldBuilder (or call into our fork once it's wired +up), not from retail decomp. Retail decomp remains the oracle for things +WorldBuilder lacks โ€” animation, motion, physics collision, networking. + +--- + +## Repo layout (as of cloned snapshot under `references/WorldBuilder/`) + +- **`Chorizite.OpenGLSDLBackend/`** โ€” full OpenGL renderer (Silk.NET). +- **`WorldBuilder.Shared/`** โ€” data models, dat parsers, landscape module. +- **`WorldBuilder/`** โ€” Avalonia desktop app shell (NOT taken). +- **`WorldBuilder.{Windows,Linux,Mac}/`** โ€” platform entry points (NOT taken). +- **`WorldBuilder.Server/`** โ€” collab editing backend (NOT taken). +- **`Tests/` + `WorldBuilder.Shared.Benchmarks/`** โ€” test harness (study). + +**Upstream NuGet dependencies** (these stay as NuGet packages, we don't +vendor them): + +| Package | Version | Purpose | +|---|---|---| +| `Chorizite.Core` | 0.0.18 | Plugin framework โ€” contains `Chorizite.Core.Lib.BoundingBox`, `Chorizite.Core.Render.*` interfaces used by every render manager | +| `Chorizite.DatReaderWriter` | 2.1.x | dat parsing (we already use 2.1.7) | +| `Chorizite.DatReaderWriter.Extensions` | 1.1.x | extra dat helpers | +| `BCnEncoder.Net` | 2.2.x | DXT decode (we already use) | +| `SixLabors.ImageSharp` | 3.1.x | image loading | +| `Silk.NET.OpenGL` + `Silk.NET.SDL` | 2.23.x | GL + windowing (we use Silk's own windowing, they use SDL) | +| `MP3Sharp` | 1.0.5 | MP3 decode | + +--- + +## ๐ŸŸข RENDERING โ€” take wholesale or adapt + +These are what makes WB "perfect". Anything in this section, we should +use from WB rather than re-implement. + +### Terrain + +| Component | What it does | +|---|---| +| `TerrainRenderManager` | Full pipeline (per-chunk GPU buffers, draw orchestration) | +| `LandSurfaceManager` | Texture blending atlas (palCode, alpha masks, road overlays) | +| `TerrainGeometryGenerator` | Heightmap โ†’ mesh, normals, OnRoad, GetHeight, GetNormal | +| `TerrainChunk` | 16ร—16 landblock chunk geometry | +| `TextureAtlasManager` | Texture atlas builder | +| `VertexLandscape` | Terrain vertex format | + +### Scenery (procedural placement: trees, bushes, rocks, fences) + +| Component | What it does | +|---|---| +| `SceneryRenderManager` | Generate + render per-vertex scenery | +| `SceneryHelpers` | Displace / RotateObj / ScaleObj / ObjAlign / CheckSlope | +| `SceneryInstance` | Per-spawn instance data | + +### Static objects (buildings, slabs, props โ€” Setup + GfxObj + ObjDesc) + +| Component | What it does | +|---|---| +| `StaticObjectRenderManager` | Master pipeline for static objects | +| `ObjectRenderManagerBase` + `BaseObjectRenderManager` | Common render base | +| `ObjectMeshManager` | Mesh extraction from Setup/GfxObj, ObjDesc application | + +### Dungeons / interiors + +| Component | What it does | +|---|---| +| `EnvCellRenderManager` | Dungeon interior cell geometry | +| `PortalRenderManager` | Portal traversal / visibility | + +### Sky + atmosphere + +| Component | What it does | +|---|---| +| `SkyboxRenderManager` | Skybox rendering | +| `ParticleEmitterRenderer` + `ParticleBatcher` + `ActiveParticleEmitter` | Particle systems (sky particles, weather, magic) | + +### Visibility / culling + +| Component | What it does | +|---|---| +| `VisibilityManager` + `VisibilitySnapshot` | Frustum + cell visibility | +| `Frustum` | Frustum-cull math | + +### Other rendering helpers + +| Component | What it does | +|---|---| +| `MinimapRenderer` | Top-down minimap | +| `GlobalMeshBuffer` | Shared GPU mesh buffer | +| `GpuResourceManager` | GPU resource lifecycle | +| `InstanceData` | Instanced draw data | +| `TextureHelpers` | INDEX16, P8, BGRA, DXT decode + alpha (canonical port) | +| `DebugRenderer` + `DebugRendererLineDrawer` + `EdgeLineBuilder` | Debug primitives | + +### Shaders (22 total) + +Located at `Chorizite.OpenGLSDLBackend/Shaders/`: + +`Landscape.{vert,frag}` ยท `StaticObject.{vert,frag}` ยท `StaticObjectModern.{vert,frag}` ยท `Particle.{vert,frag}` ยท `PortalStencil.{vert,frag}` ยท `Outline.{vert,frag}` ยท `Simple3D.{vert,frag}` ยท `InstancedLine.{vert,frag}` ยท `Text.{vert,frag}` ยท `UI.{vert,frag}` ยท `Gizmo.{vert,frag}` (editor-only) + +--- + +## ๐ŸŸข LOW-LEVEL GL / FRAMEWORK โ€” take or replace with our own + +Either take WB's wrappers wholesale, or keep our own and adapt the +render managers to use ours. These wrappers are stateless or +near-stateless and are the easiest to swap. + +| Component | What it does | +|---|---| +| `OpenGLGraphicsDevice` | Silk.NET.OpenGL wrapper | +| `OpenGLRenderer` | Render orchestration | +| `GLSLShader` | Shader compile/link/uniforms | +| `GLHelpers` + `GLStateScope` | GL state utility | +| `ManagedGLFrameBuffer` / `ManagedGLIndexBuffer` / `ManagedGLTexture` / `ManagedGLTextureArray` / `ManagedGLUniformBuffer` / `ManagedGLVertexArray` / `ManagedGLVertexBuffer` | GL resource wrappers | +| `TextureParameters` | Sampler config | +| `GpuMemoryTracker` | Memory tracking | +| `Camera2D` / `Camera3D` / `CameraBase` / `ICamera` / `CameraController` | Camera primitives | +| `GameScene` + `SingleObjectScene` + `SceneData` + `ModernRenderData` + `RenderPass` | Scene / pass structures | + +--- + +## ๐ŸŸข GEOMETRY / MATH UTILS โ€” take wholesale + +| Component | File | +|---|---| +| `TerrainUtils` (OnRoad, GetNormal, GetHeight, GetRoad, palCode) | `WorldBuilder.Shared/Modules/Landscape/Lib/TerrainUtils.cs` | +| `TerrainCacheManager` | `โ€ฆ/Lib/TerrainCacheManager.cs` | +| `TerrainRaycast` | `โ€ฆ/Lib/TerrainRaycast.cs` | +| `GeometryUtils` | `WorldBuilder.Shared/Lib/GeometryUtils.cs` | +| `RaycastingUtils` (ray-vs-sphere/AABB/triangle) | `WorldBuilder.Shared/Lib/RaycastingUtils.cs` | +| `DoubleNumerics` (double-precision Vector/Matrix) | `WorldBuilder.Shared/Lib/DoubleNumerics.cs` | +| `DatUtils` | `WorldBuilder.Shared/Lib/DatUtils.cs` | +| `BoundingBoxExtensions` | `Chorizite.OpenGLSDLBackend/Lib/BoundingBoxExtensions.cs` | + +--- + +## ๐ŸŸข DATA MODELS โ€” take selectively + +| Component | What it does | +|---|---| +| `RegionInfo` | Landblock metadata wrapper (LandblockSizeInUnits, CellSizeInUnits, etc.) | +| `TerrainEntry` | Per-vertex terrain (Type/Scenery/Road/Height) | +| `MergedLandblock` | Merged dat data | +| `CellSplitDirection` | SW-NE vs NE-SW | +| `Cell` | Generic cell wrapper | +| `ObjectId` | Object identifier | +| `Position` | World position | +| `ACEnums` | AC-specific enums | +| `WbBuildingPortal` / `WbCellPortal` | Portal structures | +| `BuildingObject` | Building data | + +--- + +## ๐ŸŸก EDITOR-ONLY โ€” leave behind / delete in fork + +These exist for the editor experience and have no place in a game +client. Delete in fork. + +- **`Modules/Landscape/Tools/*`** โ€” `BrushTool`, `BucketFillTool`, + `RoadLineTool`, `RoadVertexTool`, `InspectorTool`, + `ObjectManipulationTool`, `Gizmo*` (DragHandler, HitTester, Renderer, + State), `TexturePainting*`, `SceneRaycaster`, + `LandscapeBrush`, `LandscapeToolBase`, `LandscapeToolContext`, + `IToolSettingsProvider`, `ILandscapeBrush`, `ILandscapeEditorService`, + `ILandscapeRaycastService`, `ILandscapeTool`, `ITexturePaintingTool` +- **`Modules/Landscape/Commands/*`** โ€” undo/redo command pattern for + editor (Add/Delete/Move/Rename/Reorder/etc.) +- **`LandscapeDocument` + `LandscapeLayer` + `LandscapeLayerGroup` + `LandscapeChunk` + `LandscapeLayerChunk` + `LandscapeLayerBase`** โ€” editor document model +- **`Modules/Landscape/Models/TerrainPatch*` + `LandblockChangedEventArgs`** โ€” editor mutation events +- **`Modules/Landscape/Services/ILandscapeCacheService` + `ILandscapeDataProvider` + `ILandscapeObjectService` + impls** โ€” editor data flow +- **All `Migrations/*`** โ€” SQLite schema migrations (project file format) +- **`Repositories/*`** + **`Services/*`** โ€” project storage, dat repository, AceDb, SignalR sync, document manager, undo stack, world coordinates, keyword DB, project migration, semantic kernel AI helpers +- **`Hubs/*`** โ€” collaborative editing via SignalR +- **`StaticObject` (editor model)** โ€” replace with our own scene-state data model fed from network +- **`BackendGizmoDrawer` + `GizmoRenderer`** โ€” editor gizmos +- **`ProjectStructures, IProject, Project`** โ€” editor project files +- **`KeyBinding`** โ€” editor input binding +- **`ViewportInputEvent[Extensions]`** โ€” editor viewport input +- **`EditorState`** โ€” editor state container + +--- + +## ๐ŸŸก AUDIO / FONT โ€” we already have alternatives + +Keep ours; don't take theirs. + +- **`AudioPlaybackEngine`** โ€” uses MP3Sharp. We have OpenAL. +- **`FontRenderer`** โ€” uses ImageSharp. We have BitmapFont/StbTrueTypeSharp + ImGui. + +--- + +## ๐Ÿ”ด NOT IN WORLDBUILDER โ€” port from retail decomp ourselves + +WorldBuilder is a dat editor; it does not have: + +- **Network protocol** โ€” UDP framing, ISAAC, packet codec, ACE message + layer (we have this; oracle is `references/holtburger`) +- **Physics** โ€” collision (CPhysicsObj transitions, BSP queries, sphere + sweeps), step-up, walkable validation (we have partial; oracle is the + retail decomp at `docs/research/named-retail/`) +- **Animation** โ€” motion sequencer, cycle/non-cycle parts, animation + frame interpolation (we have this; oracle is retail decomp) +- **Movement** โ€” local player WASD โ†’ MoveToState wire, remote-entity + motion via UpdateMotion + dead-reckoning (we have this; oracle is + `references/holtburger` + retail decomp) +- **Game UI** โ€” chat, vitals, inventory, spell book, allegiance, options + (we have this; ImGui-based today, custom-toolkit later) +- **Plugin API** โ€” `IGameState`, `IEvents`, `IActions`, `IPacketPipeline`, + `IOverlay` (we have this โ€” acdream-unique) +- **Game events** โ€” combat, allegiance, spell casting, quest events + (we have this; oracle is ACE for opcodes + retail for client behavior) +- **Audio** โ€” OpenAL pipeline, sound triggers (we have this) +- **TurbineChat** + **slash commands** (we have this) +- **Login + character selection flow** (we have this) + +--- + +## What this means for the workflow + +The CLAUDE.md "grep named โ†’ decompile โ†’ verify โ†’ port" workflow stays +the rule for everything in the ๐Ÿ”ด list (network, physics, animation, +movement, UI, plugin, audio, chat). For anything in ๐ŸŸข, the new rule is: +**check this inventory FIRST**. If WB has it, port from WB. Re-porting +from retail decomp when WB already has a tested port is no longer +appropriate โ€” that's how we got the scenery edge-vertex bug. + +When the inventory says "take wholesale or adapt" and we discover a +behavior mismatch with retail (rare โ€” WB is verified), the resolution +is: reconcile WB โ†” retail decomp โ†” holtburger โ†” ACE โ†” ACViewer (the +existing reference hierarchy in CLAUDE.md). WorldBuilder ranks at the +top of that hierarchy for anything ๐ŸŸข.