acdream/docs/plans/2026-04-10-phase-4-networking-design.md

Phase 4: Networking Design

Status: Design (not yet scheduled for implementation) Date: 2026-04-10 Prerequisite: Phases 1-3 complete (dat loading, static world rendering, lighting) Primary reference: references/ACE/Source/ACE.Server/Network/ (ACE is the authority for the protocol because neither WorldBuilder nor ACViewer implements client-side networking)

---

1. Goals

Phase 4 brings a live server connection online so the static Holtburg scene becomes an actual game session. Minimum bar:

1. Authenticate against a local ACE server (username/password login).
2. Enter the world as a character (character select + spawn).
3. Receive CreateObject messages for dynamic entities (players, creatures, the foundry statue) and route them through the same IGameState/IEvents pipeline the static world already uses.
4. Send PlayerAutonomousMove updates so our client shows up on the server's world as a logged-in character.

Explicit non-goals for Phase 4:

• Combat / spellcasting
• Inventory manipulation
• Chat UI (ingest only, render later)
• Visual character appearance (clothing, palettes, animation priorities) — use placeholder meshes until Phase 5
• Server list / patch server / account creation — hardcode a dev endpoint

---

2. Why ACE is the reference

The protocol is baked into the retail client. There are three practical sources:

Source	What it has	License	Fit
references/ACE/	Full server-side protocol: packet framing, ISAAC, fragment reassembly, GameMessage encoding, every opcode	AGPL	Authority. Read-only reference; design our own implementation.
references/WorldBuilder/	Nothing — it's an offline dat viewer/editor	MIT	N/A
references/ACViewer/	Nothing — offline dat viewer	GPL	N/A
references/holtburger/	Rust ac-protocol crate with handshake + packet parsing	AGPL	Architecture reference only — confirms what works, don't copy code

Licensing note: we read ACE for protocol knowledge; we do not paste AGPL code into acdream. The wire format is a fact about the game, not ACE's IP. Structures and algorithms get reimplemented from our own understanding of the code.

---

3. Architecture overview

┌──────────────────────────────────────────────────────────────┐
│                      AcDream.App (host)                     │
│   GameWindow  ──uses──▶  WorldSession  ──publishes──▶  IEvents│
└──────────────────────────────────────────────────────────────┘
                │
                ▼
┌──────────────────────────────────────────────────────────────┐
│                   AcDream.Core.Net                           │
│                                                              │
│  NetClient (UDP socket pump, send/recv loop)                 │
│     │                                                        │
│     ▼                                                        │
│  PacketCodec (ISAAC, xor, CRC, headers, fragments)           │
│     │                                                        │
│     ▼                                                        │
│  FragmentAssembler (multi-fragment GameMessages)             │
│     │                                                        │
│     ▼                                                        │
│  GameMessageReader ──▶  IGameMessageHandler[] (dispatch)     │
│                                 │                            │
│                                 ▼                            │
│                          WorldSession                        │
│                            (applies to IGameState)           │
└──────────────────────────────────────────────────────────────┘

New project: AcDream.Core.Net. Contains no rendering, no Silk.NET. Pure .NET 10, System.Net.Sockets. Testable with a loopback fake server.

New project: AcDream.Core.Net.Tests. Mostly xUnit with canned packet captures.

---

4. Module breakdown

4.1 PacketCodec

Handles the Turbine/AC packet format:

• PacketHeader (20 bytes: sequence, flags, checksum, session, table, time, size, iteration)
• PacketHeaderFlags enum
• Optional header fields gated by flags (ack, time sync, ISAAC CRC seed, ...)
• Body: 0..N MessageFragments
• Fragment header: 16 bytes (size, group, sequence, id, count, index, queue)

Key algorithms to reimplement (read ACE.Server.Network.Packet.cs, PacketHeader.cs, MessageFragment.cs):

1. Header checksum — simple 32-bit sum.
2. Body checksum — includes fragment bodies with a rotating transform.
3. ISAAC stream — the server sends a seed in the connect response; both sides generate a keystream and XOR it into the CRC field of each outbound packet. ACE has a complete IsaacRandom.cs.
4. CRC-masking of outbound packets — client XORs ISAAC keystream word into the CRC.

Test strategy: capture 5-10 packets from a real ACE session with Wireshark, save as hex fixtures under tests/fixtures/net/, unit-test encode/decode round-trip.

4.2 NetClient

Socket pump. Single UDP socket, non-blocking receive on a dedicated thread, outbound queue.

public sealed class NetClient : IDisposable
{
    public NetClient(IPEndPoint server, PacketCodec codec);
    public event Action<ReadOnlyMemory<byte>>? PacketReceived; // raw body after codec
    public void Start();
    public void Send(GameMessage msg, MessageGroup group);
    public void Dispose();
}

• Keeps a sliding window of unacked sequences for retransmit.
• Sends ack bundles on a timer (~500ms).
• Tracks server time offset for NetworkBundle time fields.

4.3 FragmentAssembler

Some GameMessages exceed the per-packet fragment budget (~460 bytes of payload). AC splits them into multiple fragments sharing a messageId and differing index. Assembler holds partial buffers keyed by (session, messageId), releases a complete GameMessage when all count fragments have arrived.

4.4 GameMessageReader and handlers

public interface IGameMessageHandler
{
    GameMessageOpcode Opcode { get; }
    void Handle(BinaryReader reader, WorldSession session);
}

Phase 4 opcodes (minimum viable client):

Opcode	Name	Action
0xF7B0	GameEvent (wraps sub-opcodes)	Dispatch to sub-handler
0xF745	CreateObject	Hydrate WorldEntitySnapshot, publish EntitySpawned
0xF747	DeleteObject	Publish EntityDespawned
0xF748	UpdateObject	Mutate snapshot, publish EntityUpdated
0xF7E1	CharacterList (GameEvent sub)	Fill character select state
0xF7E2	LoginCharacterResponse	Transition to in-world
0xF749	PlayerDescription	Patch local character stats
0xF74C	PlayerTeleport	Move camera target

Everything else is ignored (logged at debug) until later phases.

4.5 WorldSession

Owns the connection lifecycle state machine:

Disconnected ──▶ Connecting ──▶ Authenticated ──▶ CharacterSelect ──▶ InWorld ──▶ Disconnected

Hosts:

• IGameState implementation for dynamic entities (works alongside the static DatWorldState from Phase 2)
• Heartbeat timer (ping the server every ~15s)
• Outbound PlayerAutonomousMove pump when in-world

4.6 Integration with IGameState / IEvents

Phase 2's IGameState has IEnumerable<WorldEntitySnapshot> Entities. Today it's fed only by dat hydration.

Phase 4 change: make the game state a composite. Static entities (stabs, buildings, scenery, interiors) live forever. Dynamic entities (from CreateObject) live in a mutable dictionary keyed by ObjectId. Both are enumerated through the same Entities property and routed through the same IEvents.EntitySpawned with replay-on-subscribe semantics the plugin system already expects.

This is how the foundry statue finally appears. It's a weenie with a fixed ObjectId in the server DB; the server sends us a CreateObject for it the moment we enter the landblock, we hydrate the snapshot (referencing the same GfxObj id the dats already provide), publish EntitySpawned, and the existing MeshRenderer pipeline from Phase 2 picks it up with zero changes.

---

5. Authentication flow (minimum viable)

The retail login dance was a mess (patch server → login server → world server). ACE short-circuits most of it. Target ACE's simplified path:

1. Connect UDP to ACE's login port (default 9000).
2. Send LoginRequest packet with account + password.
3. Server replies with ConnectRequest (contains ISAAC seeds).
4. Client replies with ConnectResponse.
5. Server sends CharacterList (wrapped in a GameEvent).
6. UI shows character list; user picks one; client sends CharacterLogin(guid).
7. Server replies with a storm of packets: PlayerDescription, CreateObject for self, CreateObject for nearby entities, landblock info, ...
8. Client transitions to InWorld and starts rendering everything from state.

Credentials handling is a sharp edge: per <user_privacy>, I cannot auto-fill passwords. The UI presents a password field the user types into directly; NetClient reads from the bound field only after user clicks "Login". Keep credentials out of logs.

---

6. Plugin exposure

Phase 4 is where plugins stop being a party trick and start being useful. Two new IPluginHost services:

public interface INetworkService
{
    ConnectionState State { get; }
    event Action<ConnectionState> StateChanged;
    void SendChat(ChatChannel channel, string text);
    void SendGameAction(GameAction action); // allowlist only
}

Allowlist of GameActions plugins may send: movement (PlayerAutonomousMove), chat (Chat, Tell), inventory-read-only actions. No casting, no trade, no commerce until we have a permission UI. This protects users from malicious macros.

IEvents gets new channels:

• EntitySpawned / EntityUpdated / EntityDespawned now fire for dynamic entities too
• ChatMessage (server-origin text)
• CharacterPositionChanged (self)

This is enough surface for a radar plugin, a chat logger, a follow-bot, or a fishing macro — the four canonical "is the plugin system real?" demos.

---

7. Test strategy

Phase 4 is the first phase where xUnit alone isn't enough: UDP, real server, timing. Layered approach:

1. Unit tests (fast, deterministic):

   • PacketCodec round-trip on hex fixtures
   • IsaacRandom keystream matches ACE's golden values
   • FragmentAssembler with out-of-order arrivals, duplicates, missing fragments
   • Each IGameMessageHandler handled in isolation with a synthetic reader

2. Fake server integration (medium):

   • AcDream.Core.Net.Tests.FakeServer — tiny in-process UDP loop that scripts a login + single CreateObject and asserts WorldSession reaches InWorld and publishes the expected EntitySpawned
   • Runs in CI

3. Live ACE manual smoke (slow, manual only):

   • Local ACE docker container, standard dev account
   • Launch acdream, log in, verify Holtburg renders with dynamic entities on top
   • This is when the foundry statue should finally appear in-game. Visual confirmation = Phase 4 done.

---

8. Risks and sharp edges

Risk	Mitigation
ISAAC mismatch → every packet rejected	Golden-vector test from ACE's IsaacRandom before touching the network
Fragment reassembly bugs → corrupted GameMessages	Fuzz the assembler with randomized fragment orderings
CreateObject contains ~80 optional fields gated by PhysicsDescriptionFlag and ObjectDescriptionFlag bitmasks; partial implementations silently drop data	Parse all flags to consume the right bytes even if we don't store the values; integration test verifies subsequent messages still parse correctly
Credentials leaking into logs	Wrap the auth packet in a // SECRET region and strip in the packet logger
Plugin malware via INetworkService.SendGameAction	Strict allowlist enum; anything else throws
License contamination from reading ACE	All network code goes in AcDream.Core.Net with a NOTICE.md crediting "protocol reimplemented from observation of ACE (AGPL)", code written from scratch

---

9. Out of scope (later phases)

• Phase 5: character appearance + animation (clothing tables, palette swaps, motion interp)
• Phase 6: physics (collision, gravity — port ACViewer's Physics/ since it matches retail exactly)
• Phase 7: full GameAction surface (combat, magic, trade) with a permission UI for plugins
• Phase 8: chat UI and command entry
• Phase 9: multi-landblock streaming (load/unload as player moves)

---

10. Deliverables checklist

• src/AcDream.Core.Net/ project created
• PacketCodec + IsaacRandom with golden-vector tests passing
• FragmentAssembler with fuzz tests passing
• GameMessageReader with handlers for the 8 Phase 4 opcodes
• NetClient with UDP pump + retransmit
• WorldSession state machine + composite IGameState
• Fake-server integration test: login → CreateObject → EntitySpawned published
• INetworkService exposed via IPluginHost
• IEvents.ChatMessage channel
• Minimal login UI (username + password text input, "Login" button)
• Manual smoke: log into local ACE, enter Holtburg, see the foundry statue
• NOTICE.md crediting ACE as protocol reference
• Memory handoff file after merge