From 3c3f5267de63703a62f9c149a24aa29bd91a2c5b Mon Sep 17 00:00:00 2001 From: Erik Date: Fri, 10 Apr 2026 22:31:26 +0200 Subject: [PATCH] docs(plan): Phase 4 networking design Design-only doc covering the networking subsystem needed to bring the static Holtburg scene online: UDP packet codec, ISAAC keystream, fragment reassembly, GameMessage dispatch, WorldSession lifecycle, and composite IGameState so server CreateObject messages flow through the same IEvents pipeline the dat-hydrated static entities already use. Also documents that ACE is the authority for the protocol (neither WorldBuilder nor ACViewer implements client networking) and captures the license hygiene plan: read ACE's AGPL source for protocol knowledge, reimplement from scratch, credit in NOTICE.md. Explicit win condition for Phase 4: the foundry statue finally appears, because it's a server weenie, not a dat decoration. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../2026-04-10-phase-4-networking-design.md | 270 ++++++++++++++++++ 1 file changed, 270 insertions(+) create mode 100644 docs/plans/2026-04-10-phase-4-networking-design.md diff --git a/docs/plans/2026-04-10-phase-4-networking-design.md b/docs/plans/2026-04-10-phase-4-networking-design.md new file mode 100644 index 0000000..3322e82 --- /dev/null +++ b/docs/plans/2026-04-10-phase-4-networking-design.md @@ -0,0 +1,270 @@ +# Phase 4: Networking Design + +**Status:** Design (not yet scheduled for implementation) +**Date:** 2026-04-10 +**Prerequisite:** Phases 1-3 complete (dat loading, static world rendering, lighting) +**Primary reference:** `references/ACE/Source/ACE.Server/Network/` (ACE is the authority for the protocol because neither WorldBuilder nor ACViewer implements client-side networking) + +--- + +## 1. Goals + +Phase 4 brings a live server connection online so the static Holtburg scene becomes an actual game session. Minimum bar: + +1. Authenticate against a local ACE server (username/password login). +2. Enter the world as a character (character select + spawn). +3. Receive `CreateObject` messages for dynamic entities (players, creatures, **the foundry statue**) and route them through the same `IGameState`/`IEvents` pipeline the static world already uses. +4. Send `PlayerAutonomousMove` updates so our client shows up on the server's world as a logged-in character. + +**Explicit non-goals for Phase 4:** +- Combat / spellcasting +- Inventory manipulation +- Chat UI (ingest only, render later) +- Visual character appearance (clothing, palettes, animation priorities) — use placeholder meshes until Phase 5 +- Server list / patch server / account creation — hardcode a dev endpoint + +--- + +## 2. Why ACE is the reference + +The protocol is baked into the retail client. There are three practical sources: + +| Source | What it has | License | Fit | +|---|---|---|---| +| `references/ACE/` | Full server-side protocol: packet framing, ISAAC, fragment reassembly, GameMessage encoding, every opcode | AGPL | Authority. Read-only reference; design our own implementation. | +| `references/WorldBuilder/` | Nothing — it's an offline dat viewer/editor | MIT | N/A | +| `references/ACViewer/` | Nothing — offline dat viewer | GPL | N/A | +| `references/holtburger/` | Rust `ac-protocol` crate with handshake + packet parsing | AGPL | Architecture reference only — confirms what works, don't copy code | + +**Licensing note:** we read ACE for protocol knowledge; we do not paste AGPL code into acdream. The wire format is a fact about the game, not ACE's IP. Structures and algorithms get reimplemented from our own understanding of the code. + +--- + +## 3. Architecture overview + +``` +┌──────────────────────────────────────────────────────────────┐ +│ AcDream.App (host) │ +│ GameWindow ──uses──▶ WorldSession ──publishes──▶ IEvents│ +└──────────────────────────────────────────────────────────────┘ + │ + ▼ +┌──────────────────────────────────────────────────────────────┐ +│ AcDream.Core.Net │ +│ │ +│ NetClient (UDP socket pump, send/recv loop) │ +│ │ │ +│ ▼ │ +│ PacketCodec (ISAAC, xor, CRC, headers, fragments) │ +│ │ │ +│ ▼ │ +│ FragmentAssembler (multi-fragment GameMessages) │ +│ │ │ +│ ▼ │ +│ GameMessageReader ──▶ IGameMessageHandler[] (dispatch) │ +│ │ │ +│ ▼ │ +│ WorldSession │ +│ (applies to IGameState) │ +└──────────────────────────────────────────────────────────────┘ +``` + +New project: **`AcDream.Core.Net`**. Contains no rendering, no Silk.NET. Pure .NET 10, `System.Net.Sockets`. Testable with a loopback fake server. + +New project: **`AcDream.Core.Net.Tests`**. Mostly xUnit with canned packet captures. + +--- + +## 4. Module breakdown + +### 4.1 `PacketCodec` + +Handles the Turbine/AC packet format: + +- `PacketHeader` (20 bytes: sequence, flags, checksum, session, table, time, size, iteration) +- `PacketHeaderFlags` enum +- Optional header fields gated by flags (ack, time sync, ISAAC CRC seed, ...) +- Body: 0..N `MessageFragment`s +- Fragment header: 16 bytes (size, group, sequence, id, count, index, queue) + +Key algorithms to reimplement (read `ACE.Server.Network.Packet.cs`, `PacketHeader.cs`, `MessageFragment.cs`): + +1. **Header checksum** — simple 32-bit sum. +2. **Body checksum** — includes fragment bodies with a rotating transform. +3. **ISAAC stream** — the server sends a seed in the connect response; both sides generate a keystream and XOR it into the CRC field of each outbound packet. ACE has a complete `IsaacRandom.cs`. +4. **CRC-masking of outbound packets** — client XORs ISAAC keystream word into the CRC. + +**Test strategy:** capture 5-10 packets from a real ACE session with Wireshark, save as hex fixtures under `tests/fixtures/net/`, unit-test encode/decode round-trip. + +### 4.2 `NetClient` + +Socket pump. Single UDP socket, non-blocking receive on a dedicated thread, outbound queue. + +```csharp +public sealed class NetClient : IDisposable +{ + public NetClient(IPEndPoint server, PacketCodec codec); + public event Action>? PacketReceived; // raw body after codec + public void Start(); + public void Send(GameMessage msg, MessageGroup group); + public void Dispose(); +} +``` + +- Keeps a sliding window of unacked sequences for retransmit. +- Sends ack bundles on a timer (~500ms). +- Tracks server time offset for `NetworkBundle` time fields. + +### 4.3 `FragmentAssembler` + +Some GameMessages exceed the per-packet fragment budget (~460 bytes of payload). AC splits them into multiple fragments sharing a `messageId` and differing `index`. Assembler holds partial buffers keyed by `(session, messageId)`, releases a complete `GameMessage` when all `count` fragments have arrived. + +### 4.4 `GameMessageReader` and handlers + +```csharp +public interface IGameMessageHandler +{ + GameMessageOpcode Opcode { get; } + void Handle(BinaryReader reader, WorldSession session); +} +``` + +Phase 4 opcodes (minimum viable client): + +| Opcode | Name | Action | +|---|---|---| +| `0xF7B0` | GameEvent (wraps sub-opcodes) | Dispatch to sub-handler | +| `0xF745` | CreateObject | Hydrate `WorldEntitySnapshot`, publish `EntitySpawned` | +| `0xF747` | DeleteObject | Publish `EntityDespawned` | +| `0xF748` | UpdateObject | Mutate snapshot, publish `EntityUpdated` | +| `0xF7E1` | CharacterList (GameEvent sub) | Fill character select state | +| `0xF7E2` | LoginCharacterResponse | Transition to in-world | +| `0xF749` | PlayerDescription | Patch local character stats | +| `0xF74C` | PlayerTeleport | Move camera target | + +Everything else is ignored (logged at debug) until later phases. + +### 4.5 `WorldSession` + +Owns the connection lifecycle state machine: + +``` +Disconnected ──▶ Connecting ──▶ Authenticated ──▶ CharacterSelect ──▶ InWorld ──▶ Disconnected +``` + +Hosts: +- `IGameState` implementation for dynamic entities (works alongside the static `DatWorldState` from Phase 2) +- Heartbeat timer (ping the server every ~15s) +- Outbound `PlayerAutonomousMove` pump when in-world + +### 4.6 Integration with `IGameState` / `IEvents` + +Phase 2's `IGameState` has `IEnumerable Entities`. Today it's fed only by dat hydration. + +Phase 4 change: make the game state a **composite**. Static entities (stabs, buildings, scenery, interiors) live forever. Dynamic entities (from `CreateObject`) live in a mutable dictionary keyed by `ObjectId`. Both are enumerated through the same `Entities` property and routed through the same `IEvents.EntitySpawned` with replay-on-subscribe semantics the plugin system already expects. + +**This is how the foundry statue finally appears.** It's a weenie with a fixed ObjectId in the server DB; the server sends us a `CreateObject` for it the moment we enter the landblock, we hydrate the snapshot (referencing the same `GfxObj` id the dats already provide), publish `EntitySpawned`, and the existing `MeshRenderer` pipeline from Phase 2 picks it up with zero changes. + +--- + +## 5. Authentication flow (minimum viable) + +The retail login dance was a mess (patch server → login server → world server). ACE short-circuits most of it. Target ACE's simplified path: + +1. **Connect UDP** to ACE's login port (default 9000). +2. Send `LoginRequest` packet with account + password. +3. Server replies with `ConnectRequest` (contains ISAAC seeds). +4. Client replies with `ConnectResponse`. +5. Server sends `CharacterList` (wrapped in a GameEvent). +6. UI shows character list; user picks one; client sends `CharacterLogin(guid)`. +7. Server replies with a storm of packets: PlayerDescription, CreateObject for self, CreateObject for nearby entities, landblock info, ... +8. Client transitions to `InWorld` and starts rendering everything from state. + +**Credentials handling** is a sharp edge: per ``, I cannot auto-fill passwords. The UI presents a password field the user types into directly; NetClient reads from the bound field only after user clicks "Login". Keep credentials out of logs. + +--- + +## 6. Plugin exposure + +Phase 4 is where plugins stop being a party trick and start being useful. Two new `IPluginHost` services: + +```csharp +public interface INetworkService +{ + ConnectionState State { get; } + event Action StateChanged; + void SendChat(ChatChannel channel, string text); + void SendGameAction(GameAction action); // allowlist only +} +``` + +Allowlist of `GameAction`s plugins may send: movement (`PlayerAutonomousMove`), chat (`Chat`, `Tell`), inventory-read-only actions. **No casting, no trade, no commerce** until we have a permission UI. This protects users from malicious macros. + +`IEvents` gets new channels: +- `EntitySpawned` / `EntityUpdated` / `EntityDespawned` now fire for dynamic entities too +- `ChatMessage` (server-origin text) +- `CharacterPositionChanged` (self) + +This is enough surface for a radar plugin, a chat logger, a follow-bot, or a fishing macro — the four canonical "is the plugin system real?" demos. + +--- + +## 7. Test strategy + +Phase 4 is the first phase where xUnit alone isn't enough: UDP, real server, timing. Layered approach: + +1. **Unit tests** (fast, deterministic): + - `PacketCodec` round-trip on hex fixtures + - `IsaacRandom` keystream matches ACE's golden values + - `FragmentAssembler` with out-of-order arrivals, duplicates, missing fragments + - Each `IGameMessageHandler` handled in isolation with a synthetic reader + +2. **Fake server integration** (medium): + - `AcDream.Core.Net.Tests.FakeServer` — tiny in-process UDP loop that scripts a login + single `CreateObject` and asserts `WorldSession` reaches `InWorld` and publishes the expected `EntitySpawned` + - Runs in CI + +3. **Live ACE manual smoke** (slow, manual only): + - Local ACE docker container, standard dev account + - Launch acdream, log in, verify Holtburg renders with dynamic entities on top + - **This is when the foundry statue should finally appear in-game.** Visual confirmation = Phase 4 done. + +--- + +## 8. Risks and sharp edges + +| Risk | Mitigation | +|---|---| +| ISAAC mismatch → every packet rejected | Golden-vector test from ACE's `IsaacRandom` before touching the network | +| Fragment reassembly bugs → corrupted GameMessages | Fuzz the assembler with randomized fragment orderings | +| `CreateObject` contains ~80 optional fields gated by `PhysicsDescriptionFlag` and `ObjectDescriptionFlag` bitmasks; partial implementations silently drop data | Parse **all** flags to consume the right bytes even if we don't store the values; integration test verifies subsequent messages still parse correctly | +| Credentials leaking into logs | Wrap the auth packet in a `// SECRET` region and strip in the packet logger | +| Plugin malware via `INetworkService.SendGameAction` | Strict allowlist enum; anything else throws | +| License contamination from reading ACE | All network code goes in `AcDream.Core.Net` with a `NOTICE.md` crediting "protocol reimplemented from observation of ACE (AGPL)", code written from scratch | + +--- + +## 9. Out of scope (later phases) + +- **Phase 5:** character appearance + animation (clothing tables, palette swaps, motion interp) +- **Phase 6:** physics (collision, gravity — port ACViewer's `Physics/` since it matches retail exactly) +- **Phase 7:** full GameAction surface (combat, magic, trade) with a permission UI for plugins +- **Phase 8:** chat UI and command entry +- **Phase 9:** multi-landblock streaming (load/unload as player moves) + +--- + +## 10. Deliverables checklist + +- [ ] `src/AcDream.Core.Net/` project created +- [ ] `PacketCodec` + `IsaacRandom` with golden-vector tests passing +- [ ] `FragmentAssembler` with fuzz tests passing +- [ ] `GameMessageReader` with handlers for the 8 Phase 4 opcodes +- [ ] `NetClient` with UDP pump + retransmit +- [ ] `WorldSession` state machine + composite `IGameState` +- [ ] Fake-server integration test: login → CreateObject → EntitySpawned published +- [ ] `INetworkService` exposed via `IPluginHost` +- [ ] `IEvents.ChatMessage` channel +- [ ] Minimal login UI (username + password text input, "Login" button) +- [ ] Manual smoke: log into local ACE, enter Holtburg, see the foundry statue +- [ ] NOTICE.md crediting ACE as protocol reference +- [ ] Memory handoff file after merge