3c3f5267de
Design-only doc covering the networking subsystem needed to bring the static Holtburg scene online: UDP packet codec, ISAAC keystream, fragment reassembly, GameMessage dispatch, WorldSession lifecycle, and composite IGameState so server CreateObject messages flow through the same IEvents pipeline the dat-hydrated static entities already use. Also documents that ACE is the authority for the protocol (neither WorldBuilder nor ACViewer implements client networking) and captures the license hygiene plan: read ACE's AGPL source for protocol knowledge, reimplement from scratch, credit in NOTICE.md. Explicit win condition for Phase 4: the foundry statue finally appears, because it's a server weenie, not a dat decoration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
270 lines
14 KiB
Markdown
270 lines
14 KiB
Markdown
# Phase 4: Networking Design
|
|
|
|
**Status:** Design (not yet scheduled for implementation)
|
|
**Date:** 2026-04-10
|
|
**Prerequisite:** Phases 1-3 complete (dat loading, static world rendering, lighting)
|
|
**Primary reference:** `references/ACE/Source/ACE.Server/Network/` (ACE is the authority for the protocol because neither WorldBuilder nor ACViewer implements client-side networking)
|
|
|
|
---
|
|
|
|
## 1. Goals
|
|
|
|
Phase 4 brings a live server connection online so the static Holtburg scene becomes an actual game session. Minimum bar:
|
|
|
|
1. Authenticate against a local ACE server (username/password login).
|
|
2. Enter the world as a character (character select + spawn).
|
|
3. Receive `CreateObject` messages for dynamic entities (players, creatures, **the foundry statue**) and route them through the same `IGameState`/`IEvents` pipeline the static world already uses.
|
|
4. Send `PlayerAutonomousMove` updates so our client shows up on the server's world as a logged-in character.
|
|
|
|
**Explicit non-goals for Phase 4:**
|
|
- Combat / spellcasting
|
|
- Inventory manipulation
|
|
- Chat UI (ingest only, render later)
|
|
- Visual character appearance (clothing, palettes, animation priorities) — use placeholder meshes until Phase 5
|
|
- Server list / patch server / account creation — hardcode a dev endpoint
|
|
|
|
---
|
|
|
|
## 2. Why ACE is the reference
|
|
|
|
The protocol is baked into the retail client. There are three practical sources:
|
|
|
|
| Source | What it has | License | Fit |
|
|
|---|---|---|---|
|
|
| `references/ACE/` | Full server-side protocol: packet framing, ISAAC, fragment reassembly, GameMessage encoding, every opcode | AGPL | Authority. Read-only reference; design our own implementation. |
|
|
| `references/WorldBuilder/` | Nothing — it's an offline dat viewer/editor | MIT | N/A |
|
|
| `references/ACViewer/` | Nothing — offline dat viewer | GPL | N/A |
|
|
| `references/holtburger/` | Rust `ac-protocol` crate with handshake + packet parsing | AGPL | Architecture reference only — confirms what works, don't copy code |
|
|
|
|
**Licensing note:** we read ACE for protocol knowledge; we do not paste AGPL code into acdream. The wire format is a fact about the game, not ACE's IP. Structures and algorithms get reimplemented from our own understanding of the code.
|
|
|
|
---
|
|
|
|
## 3. Architecture overview
|
|
|
|
```
|
|
┌──────────────────────────────────────────────────────────────┐
|
|
│ AcDream.App (host) │
|
|
│ GameWindow ──uses──▶ WorldSession ──publishes──▶ IEvents│
|
|
└──────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌──────────────────────────────────────────────────────────────┐
|
|
│ AcDream.Core.Net │
|
|
│ │
|
|
│ NetClient (UDP socket pump, send/recv loop) │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ PacketCodec (ISAAC, xor, CRC, headers, fragments) │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ FragmentAssembler (multi-fragment GameMessages) │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ GameMessageReader ──▶ IGameMessageHandler[] (dispatch) │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ WorldSession │
|
|
│ (applies to IGameState) │
|
|
└──────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
New project: **`AcDream.Core.Net`**. Contains no rendering, no Silk.NET. Pure .NET 10, `System.Net.Sockets`. Testable with a loopback fake server.
|
|
|
|
New project: **`AcDream.Core.Net.Tests`**. Mostly xUnit with canned packet captures.
|
|
|
|
---
|
|
|
|
## 4. Module breakdown
|
|
|
|
### 4.1 `PacketCodec`
|
|
|
|
Handles the Turbine/AC packet format:
|
|
|
|
- `PacketHeader` (20 bytes: sequence, flags, checksum, session, table, time, size, iteration)
|
|
- `PacketHeaderFlags` enum
|
|
- Optional header fields gated by flags (ack, time sync, ISAAC CRC seed, ...)
|
|
- Body: 0..N `MessageFragment`s
|
|
- Fragment header: 16 bytes (size, group, sequence, id, count, index, queue)
|
|
|
|
Key algorithms to reimplement (read `ACE.Server.Network.Packet.cs`, `PacketHeader.cs`, `MessageFragment.cs`):
|
|
|
|
1. **Header checksum** — simple 32-bit sum.
|
|
2. **Body checksum** — includes fragment bodies with a rotating transform.
|
|
3. **ISAAC stream** — the server sends a seed in the connect response; both sides generate a keystream and XOR it into the CRC field of each outbound packet. ACE has a complete `IsaacRandom.cs`.
|
|
4. **CRC-masking of outbound packets** — client XORs ISAAC keystream word into the CRC.
|
|
|
|
**Test strategy:** capture 5-10 packets from a real ACE session with Wireshark, save as hex fixtures under `tests/fixtures/net/`, unit-test encode/decode round-trip.
|
|
|
|
### 4.2 `NetClient`
|
|
|
|
Socket pump. Single UDP socket, non-blocking receive on a dedicated thread, outbound queue.
|
|
|
|
```csharp
|
|
public sealed class NetClient : IDisposable
|
|
{
|
|
public NetClient(IPEndPoint server, PacketCodec codec);
|
|
public event Action<ReadOnlyMemory<byte>>? PacketReceived; // raw body after codec
|
|
public void Start();
|
|
public void Send(GameMessage msg, MessageGroup group);
|
|
public void Dispose();
|
|
}
|
|
```
|
|
|
|
- Keeps a sliding window of unacked sequences for retransmit.
|
|
- Sends ack bundles on a timer (~500ms).
|
|
- Tracks server time offset for `NetworkBundle` time fields.
|
|
|
|
### 4.3 `FragmentAssembler`
|
|
|
|
Some GameMessages exceed the per-packet fragment budget (~460 bytes of payload). AC splits them into multiple fragments sharing a `messageId` and differing `index`. Assembler holds partial buffers keyed by `(session, messageId)`, releases a complete `GameMessage` when all `count` fragments have arrived.
|
|
|
|
### 4.4 `GameMessageReader` and handlers
|
|
|
|
```csharp
|
|
public interface IGameMessageHandler
|
|
{
|
|
GameMessageOpcode Opcode { get; }
|
|
void Handle(BinaryReader reader, WorldSession session);
|
|
}
|
|
```
|
|
|
|
Phase 4 opcodes (minimum viable client):
|
|
|
|
| Opcode | Name | Action |
|
|
|---|---|---|
|
|
| `0xF7B0` | GameEvent (wraps sub-opcodes) | Dispatch to sub-handler |
|
|
| `0xF745` | CreateObject | Hydrate `WorldEntitySnapshot`, publish `EntitySpawned` |
|
|
| `0xF747` | DeleteObject | Publish `EntityDespawned` |
|
|
| `0xF748` | UpdateObject | Mutate snapshot, publish `EntityUpdated` |
|
|
| `0xF7E1` | CharacterList (GameEvent sub) | Fill character select state |
|
|
| `0xF7E2` | LoginCharacterResponse | Transition to in-world |
|
|
| `0xF749` | PlayerDescription | Patch local character stats |
|
|
| `0xF74C` | PlayerTeleport | Move camera target |
|
|
|
|
Everything else is ignored (logged at debug) until later phases.
|
|
|
|
### 4.5 `WorldSession`
|
|
|
|
Owns the connection lifecycle state machine:
|
|
|
|
```
|
|
Disconnected ──▶ Connecting ──▶ Authenticated ──▶ CharacterSelect ──▶ InWorld ──▶ Disconnected
|
|
```
|
|
|
|
Hosts:
|
|
- `IGameState` implementation for dynamic entities (works alongside the static `DatWorldState` from Phase 2)
|
|
- Heartbeat timer (ping the server every ~15s)
|
|
- Outbound `PlayerAutonomousMove` pump when in-world
|
|
|
|
### 4.6 Integration with `IGameState` / `IEvents`
|
|
|
|
Phase 2's `IGameState` has `IEnumerable<WorldEntitySnapshot> Entities`. Today it's fed only by dat hydration.
|
|
|
|
Phase 4 change: make the game state a **composite**. Static entities (stabs, buildings, scenery, interiors) live forever. Dynamic entities (from `CreateObject`) live in a mutable dictionary keyed by `ObjectId`. Both are enumerated through the same `Entities` property and routed through the same `IEvents.EntitySpawned` with replay-on-subscribe semantics the plugin system already expects.
|
|
|
|
**This is how the foundry statue finally appears.** It's a weenie with a fixed ObjectId in the server DB; the server sends us a `CreateObject` for it the moment we enter the landblock, we hydrate the snapshot (referencing the same `GfxObj` id the dats already provide), publish `EntitySpawned`, and the existing `MeshRenderer` pipeline from Phase 2 picks it up with zero changes.
|
|
|
|
---
|
|
|
|
## 5. Authentication flow (minimum viable)
|
|
|
|
The retail login dance was a mess (patch server → login server → world server). ACE short-circuits most of it. Target ACE's simplified path:
|
|
|
|
1. **Connect UDP** to ACE's login port (default 9000).
|
|
2. Send `LoginRequest` packet with account + password.
|
|
3. Server replies with `ConnectRequest` (contains ISAAC seeds).
|
|
4. Client replies with `ConnectResponse`.
|
|
5. Server sends `CharacterList` (wrapped in a GameEvent).
|
|
6. UI shows character list; user picks one; client sends `CharacterLogin(guid)`.
|
|
7. Server replies with a storm of packets: PlayerDescription, CreateObject for self, CreateObject for nearby entities, landblock info, ...
|
|
8. Client transitions to `InWorld` and starts rendering everything from state.
|
|
|
|
**Credentials handling** is a sharp edge: per `<user_privacy>`, I cannot auto-fill passwords. The UI presents a password field the user types into directly; NetClient reads from the bound field only after user clicks "Login". Keep credentials out of logs.
|
|
|
|
---
|
|
|
|
## 6. Plugin exposure
|
|
|
|
Phase 4 is where plugins stop being a party trick and start being useful. Two new `IPluginHost` services:
|
|
|
|
```csharp
|
|
public interface INetworkService
|
|
{
|
|
ConnectionState State { get; }
|
|
event Action<ConnectionState> StateChanged;
|
|
void SendChat(ChatChannel channel, string text);
|
|
void SendGameAction(GameAction action); // allowlist only
|
|
}
|
|
```
|
|
|
|
Allowlist of `GameAction`s plugins may send: movement (`PlayerAutonomousMove`), chat (`Chat`, `Tell`), inventory-read-only actions. **No casting, no trade, no commerce** until we have a permission UI. This protects users from malicious macros.
|
|
|
|
`IEvents` gets new channels:
|
|
- `EntitySpawned` / `EntityUpdated` / `EntityDespawned` now fire for dynamic entities too
|
|
- `ChatMessage` (server-origin text)
|
|
- `CharacterPositionChanged` (self)
|
|
|
|
This is enough surface for a radar plugin, a chat logger, a follow-bot, or a fishing macro — the four canonical "is the plugin system real?" demos.
|
|
|
|
---
|
|
|
|
## 7. Test strategy
|
|
|
|
Phase 4 is the first phase where xUnit alone isn't enough: UDP, real server, timing. Layered approach:
|
|
|
|
1. **Unit tests** (fast, deterministic):
|
|
- `PacketCodec` round-trip on hex fixtures
|
|
- `IsaacRandom` keystream matches ACE's golden values
|
|
- `FragmentAssembler` with out-of-order arrivals, duplicates, missing fragments
|
|
- Each `IGameMessageHandler` handled in isolation with a synthetic reader
|
|
|
|
2. **Fake server integration** (medium):
|
|
- `AcDream.Core.Net.Tests.FakeServer` — tiny in-process UDP loop that scripts a login + single `CreateObject` and asserts `WorldSession` reaches `InWorld` and publishes the expected `EntitySpawned`
|
|
- Runs in CI
|
|
|
|
3. **Live ACE manual smoke** (slow, manual only):
|
|
- Local ACE docker container, standard dev account
|
|
- Launch acdream, log in, verify Holtburg renders with dynamic entities on top
|
|
- **This is when the foundry statue should finally appear in-game.** Visual confirmation = Phase 4 done.
|
|
|
|
---
|
|
|
|
## 8. Risks and sharp edges
|
|
|
|
| Risk | Mitigation |
|
|
|---|---|
|
|
| ISAAC mismatch → every packet rejected | Golden-vector test from ACE's `IsaacRandom` before touching the network |
|
|
| Fragment reassembly bugs → corrupted GameMessages | Fuzz the assembler with randomized fragment orderings |
|
|
| `CreateObject` contains ~80 optional fields gated by `PhysicsDescriptionFlag` and `ObjectDescriptionFlag` bitmasks; partial implementations silently drop data | Parse **all** flags to consume the right bytes even if we don't store the values; integration test verifies subsequent messages still parse correctly |
|
|
| Credentials leaking into logs | Wrap the auth packet in a `// SECRET` region and strip in the packet logger |
|
|
| Plugin malware via `INetworkService.SendGameAction` | Strict allowlist enum; anything else throws |
|
|
| License contamination from reading ACE | All network code goes in `AcDream.Core.Net` with a `NOTICE.md` crediting "protocol reimplemented from observation of ACE (AGPL)", code written from scratch |
|
|
|
|
---
|
|
|
|
## 9. Out of scope (later phases)
|
|
|
|
- **Phase 5:** character appearance + animation (clothing tables, palette swaps, motion interp)
|
|
- **Phase 6:** physics (collision, gravity — port ACViewer's `Physics/` since it matches retail exactly)
|
|
- **Phase 7:** full GameAction surface (combat, magic, trade) with a permission UI for plugins
|
|
- **Phase 8:** chat UI and command entry
|
|
- **Phase 9:** multi-landblock streaming (load/unload as player moves)
|
|
|
|
---
|
|
|
|
## 10. Deliverables checklist
|
|
|
|
- [ ] `src/AcDream.Core.Net/` project created
|
|
- [ ] `PacketCodec` + `IsaacRandom` with golden-vector tests passing
|
|
- [ ] `FragmentAssembler` with fuzz tests passing
|
|
- [ ] `GameMessageReader` with handlers for the 8 Phase 4 opcodes
|
|
- [ ] `NetClient` with UDP pump + retransmit
|
|
- [ ] `WorldSession` state machine + composite `IGameState`
|
|
- [ ] Fake-server integration test: login → CreateObject → EntitySpawned published
|
|
- [ ] `INetworkService` exposed via `IPluginHost`
|
|
- [ ] `IEvents.ChatMessage` channel
|
|
- [ ] Minimal login UI (username + password text input, "Login" button)
|
|
- [ ] Manual smoke: log into local ACE, enter Holtburg, see the foundry statue
|
|
- [ ] NOTICE.md crediting ACE as protocol reference
|
|
- [ ] Memory handoff file after merge
|