# Phase M — Network Stack Conformance — Design Spec **Date:** 2026-05-10 **Status:** Draft (sections 1–3 of 8 written; sections 4–8 pending; opcode matrix in flight) **Phase identifier:** M (per `docs/plans/2026-04-11-roadmap.md:414`) **Supersedes the planned-but-never-written** `docs/superpowers/specs/2026-05-02-network-stack-conformance.md` **Related research:** - [`docs/research/2026-05-10-holtburger-network-stack-study.md`](../../research/2026-05-10-holtburger-network-stack-study.md) — first-pass parity study, source of recent commit references - [`docs/research/2026-05-10-phase-m-opcode-matrix.md`](../../research/2026-05-10-phase-m-opcode-matrix.md) — opcode coverage matrix (in flight; this spec links to it as the source of "done") **Reference repos:** - `references/holtburger/` — fast-forwarded to `629695a` on 2026-05-10 - `references/ACE/` — server-side opcode authority - `docs/research/named-retail/` — Sept 2013 EoR PDB-named decomp --- ## 1. Goal and non-goals ### 1.1 Goal Build a **complete, layered, testable network protocol library** for acdream that covers every wire opcode a 2013 EoR retail client receives, sends, or both — independent of whether each opcode is yet wired into game state. The library is delivered behind three interfaces (`INetTransport`, `IReliableSession`, `IGameProtocol`); the existing `WorldSession` shrinks to a thin behavior consumer on top. Every parser, builder, and transport feature is unit-tested with golden-vector fixtures and survives a live ACE smoke loop before the phase ships. The bar is **Bar C — "wireable on demand."** For every in-scope opcode: - A typed message struct exists (record with named fields, no raw byte arrays) - A parser exists if inbound (wire bytes → typed message) - A builder exists if outbound (typed message → wire bytes) - A round-trip test exists where applicable - A golden-vector test exists pinning at least one canonical wire encoding - Either the opcode is dispatched to a typed event observable by `WorldSession`, or its dispatch is documented as deferred to the gameplay phase that needs it (with the deferred-target named, e.g., "wired in Phase F") The **behavior layer** (what to DO with each message in game state) remains the responsibility of the gameplay phase that needs it. Phase M does not wire `HouseStatusUpdate` into a house-status panel; it ensures `HouseStatusUpdate` parses correctly into a typed event so the future Phase consuming it has zero protocol work. ### 1.2 What "complete" is measured against The opcode coverage matrix at `docs/research/2026-05-10-phase-m-opcode-matrix.md` is the **source of truth** for "done." Per opcode it cites: - Holtburger coverage (`references/holtburger/crates/holtburger-{session,protocol,core}/`) - ACE coverage (`references/ACE/Source/ACE.Server/Network/GameMessages/Messages/` for outbound; `Source/ACE.Server/Network/Handlers/` for inbound accept rules) - Named retail decomp (`docs/research/named-retail/acclient_2013_pseudo_c.txt`, `symbols.json` — by `class::method` or address) - Acdream's current state (parser? builder? wired? deferred? unknown?) - Phase M target (parse, build, both, or "skip with documented justification") An opcode is **in scope** if any of: - Holtburger or ACE actively sends/receives it - The named retail decomp shows the 2013 client invoking it - It appears in observed live ACE traffic on `127.0.0.1:9000` An opcode is **out of scope** if all of: - Holtburger doesn't touch it - ACE marks it server-internal-only or post-2013 (visible in ACE's commit history or comments) - The named retail decomp shows no client-side reference - It hasn't been observed on live ACE Out-of-scope opcodes get one row in the matrix with the justification, no code work. ### 1.3 Non-goals - **Not** reimplementing ACE server behavior. Validations, accept rules, and game-side decisions live in ACE; we mirror only what the client must produce or consume. - **Not** replacing acdream's stricter inbound checksum verification. Our `PacketCodec` validates more aggressively than retail did (per the existing class doc); we keep that unless named retail proves it's wrong. - **Not** rewriting renderer, animation, audio, UI, plugin, or chat layers. Those have their own phases. The new network stack must compile under, and run alongside, the current rendering and gameplay code. - **Not** introducing async/await across the codebase. The current `Tick()`-driven recv-loop model is preserved; layer extraction is structural, not asynchrony-restructuring. (We MAY add a dedicated network thread if M.7's runtime work warrants it, but that decision is internal to M.7.) - **Not** handling opcodes that are ACE-only invented for emulation purposes (e.g., debug echos that retail never had). The matrix calls these out per row. - **Not** optimizing for throughput. Correctness first. Allocation profile and CPU cost tuning is a follow-up phase if the live loop measurably regresses. - **Not** plugin-API exposure of network internals. The plugin API gets typed-event subscriptions where useful; raw packet introspection is dev-only. ### 1.4 What ships at the end of Phase M When M.8 closes: - `src/AcDream.Net/` (new namespace) contains `INetTransport`, `IReliableSession`, `IGameProtocol`, their concrete implementations, and the typed message library. - `src/AcDream.Core.Net/WorldSession.cs` is a behavior consumer ~200–400 LOC, not the current 1213 LOC monolith. - The `tests/AcDream.Net.Tests/` project covers every protocol-layer surface with unit tests. - A `tools/network-conformance-replay/` harness can replay a captured ACE session and verify byte-perfect outputs. - `dotnet build` green, `dotnet test` green, live ACE smoke green: login → walk → chat → combat action → portal → logout, verified by user. - The roadmap entry for Phase M moves from "PLANNED" to "shipped" with a one-line summary and commit reference. --- ## 2. Coverage definition ### 2.1 The opcode matrix The matrix is a markdown table at `docs/research/2026-05-10-phase-m-opcode-matrix.md`, grouped by layer: 1. **Transport flags** — every value in `PacketHeaderFlags` (LoginRequest, ConnectRequest, AckSequence, EncryptedChecksum, BlobFragments, RequestRetransmit, RejectRetransmit, EchoRequest, EchoResponse, Flow, ServerSwitch, TimeSync, Disconnect, …). Each row says what the flag means, who sets it, and what acdream must do on receive. 2. **Optional-header fields** — every variable-length section (RequestRetransmit list, RejectRetransmit list, AckSequence, ConnectRequest payload, LoginRequest payload, CICMD, TimeSync, EchoRequest/EchoResponse times, Flow). Each row defines the byte layout and our parse/build status. 3. **GameMessage opcodes** — every top-level opcode the client sees (0xF658 CharacterList, 0xF745 CreateObject, 0xF74C UpdateMotion, 0xF7B0 GameEvent envelope, 0xF7DE TurbineChat, 0xEA60 AdminEnvirons, …) and every top-level opcode the client sends (0xF7C8 CharacterEnterWorldRequest, 0xF657 CharacterEnterWorld, 0xF61C MoveToState, 0xF61B JumpAction, 0xF753 AutonomousPosition, 0xF7E4 DddInterrogationResponse, …). 4. **GameEvent sub-opcodes** — every entry in `GameEventType.cs` (94 currently named; ~70+ currently unhandled). Each row identifies the parsing target plus the acdream wiring status. 5. **GameAction sub-opcodes** — every typed game-action ID (Talk, Tell, Channel, Use, UseWithTarget, MoveToObject, JumpAbsolute, CastSpell, Appraise, Identify, AttackTargetMelee/Missile, Allegiance ops, Inventory ops, Social ops, Skill/Attribute raise, Train, …). Each row has these columns: | Code | Direction | Name | Named-retail symbol or address | Holtburger | ACE | acdream today | Phase M target | Notes | Cell values for "Holtburger" / "ACE" / "acdream today": - **`P`** — parses inbound - **`B`** — builds outbound - **`PB`** — both - **`W`** — wired (parser/builder + dispatched to typed event consumed somewhere) - **`–`** — not implemented - **`N/A`** — not applicable for this side (e.g., a server-only message in ACE column) "Phase M target" cell values: - **`PB+W`** — must parse, build (if outbound), wire to a typed event by phase end - **`PB`** — must parse, build (if outbound), no wiring required - **`P+W`** — inbound only, must parse and dispatch typed event - **`–defer:`** — explicitly deferred to a named gameplay phase - **`–skip:`** — out of scope, with justification ### 2.2 Inbound parser obligations For every in-scope inbound opcode: - A typed C# record represents the message. Fields are named, typed, and ordered to match the wire layout (so a future reader can map field-to-byte without re-reading the parser). - The parser is a static method on the record (`public static MyMessage Parse(ref BinaryReader r)`), throws `InvalidOperationException` on malformed input with a message containing the opcode and offset. - A round-trip test exists if the opcode is also outbound. A golden-vector test always exists with at least one specific captured wire encoding. - The parser dispatches to a typed event on `IGameProtocol` (`event Action OnMyMessage`). If wiring to game state is deferred, the matrix row says `–defer:` and the typed event still exists — gameplay-phase wiring is then a one-line subscription. ### 2.3 Outbound builder obligations For every in-scope outbound opcode: - A typed C# record represents the message. - A `Build(ref BinaryWriter w)` instance method writes the wire encoding. - A golden-vector test pins at least one specific wire encoding. - The high-level entry point lives on `IGameProtocol` (`Send(MyAction act)` or `Send(MyMessage msg)`). - `WorldSession` exposes a behavior-friendly wrapper (`SendTalk(string text)` rather than `_protocol.Send(new TalkMessage { … })`) only for opcodes the user-facing app currently triggers. Less-used outbound builders stay on `IGameProtocol` directly until a gameplay phase needs the convenience wrapper. ### 2.4 Three test fixture sources - **Golden vectors.** Hand-computed bytes for representative messages. Source: named retail decomp (extract via `tools/pdb-extract/`), holtburger captures, or by manual trace. Stored in `tests/AcDream.Net.Tests/Fixtures/Golden/.bin` plus a sibling `.json` describing the fields. - **Live capture replay.** A captured session log (raw datagrams + timestamps) replayed offline against the new stack. Captures come from running acdream itself with a `ACDREAM_PCAP=1` env-var that dumps every datagram to disk. The first capture is recorded once Phase M.7's runtime is in place; subsequent captures replace it as features land. - **Live ACE smoke.** Per-sub-phase, a live `dotnet run` against `127.0.0.1:9000` that exercises the relevant features. Final M.8 smoke covers login → walk → chat → combat action → teleport → reconnect → logout end-to-end. ### 2.5 Acceptance for an in-scope opcode An opcode is "done" for Phase M when: 1. Its matrix row is filled completely. 2. The typed message struct exists and matches the documented byte layout. 3. The parser and/or builder exist and pass round-trip tests where applicable. 4. At least one golden-vector test pins a canonical encoding. 5. The typed event is exposed on `IGameProtocol` (inbound) or the high-level send method exists (outbound). 6. The matrix row's `acdream today` column is updated to match `Phase M target`. The opcode-class agents working on the matrix produce the per-row data. Phase M.6 implementation work is then "for each row in the matrix where target ≠ today, write the code and tests." --- ## 3. Three-layer architecture ### 3.1 Layer overview ``` ┌─────────────────────────────────────────────────────────────┐ │ WorldSession (behavior layer — not part of Phase M's │ │ protocol library; consumes IGameProtocol) │ └────────────────────────────┬────────────────────────────────┘ │ subscribes to typed events, │ calls Send(IGameMessage|IGameAction) ┌────────────────────────────▼────────────────────────────────┐ │ IGameProtocol — typed message routing │ │ • opcode dispatch table │ │ • GameAction sequence counter │ │ • per-message typed events │ │ • outbound: typed message → bytes via builder │ └────────────────────────────┬────────────────────────────────┘ │ delivers fully-assembled GameMessage │ payloads; receives outbound payloads ┌────────────────────────────▼────────────────────────────────┐ │ IReliableSession — wire correctness │ │ • PacketCodec (header + optional + body framing, CRC, │ │ ISAAC c2s/s2c, fragment header layout) │ │ • inbound ordering buffer + RequestRetransmit issuing │ │ • outbound packet cache + retransmit on server request │ │ • ACK queue + piggyback │ │ • EchoRequest reply, TimeSync forwarding │ │ • port-switch state machine │ │ • fragment assembly (inbound) + splitting (outbound) │ └────────────────────────────┬────────────────────────────────┘ │ INetTransport.Send(bytes, endpoint) │ INetTransport.TryReceive(out bytes, out endpoint) ┌────────────────────────────▼────────────────────────────────┐ │ INetTransport — UDP only │ │ • Send / TryReceive / Close │ │ • no protocol knowledge │ │ • UdpNetTransport (prod) / MockTransport (test) │ └─────────────────────────────────────────────────────────────┘ ``` **Hard rules on direction:** - Higher layers know about lower layers; lower layers do not know about higher layers. - `IGameProtocol` does not call into `INetTransport`; it must go through `IReliableSession`. - `WorldSession` does not directly construct UDP packets, ISAAC streams, or fragment headers. - A unit test for any layer can mock the layer below it. ### 3.2 `INetTransport` ```csharp public interface INetTransport : IDisposable { /// /// Send a single UDP datagram to the given endpoint. Synchronous. /// Returns the number of bytes sent (always == datagram.Length on /// success). Throws on socket error. /// int Send(ReadOnlySpan datagram, IPEndPoint remote); /// /// Non-blocking receive. Returns false if no datagram is available. /// On true, datagram contains the bytes (caller must not retain /// the returned span past the next call) and remote contains the /// source endpoint. /// bool TryReceive(out ReadOnlySpan datagram, out IPEndPoint remote); /// /// Local endpoint we are bound to (after construction). /// IPEndPoint LocalEndpoint { get; } } ``` **Concrete implementations:** - `UdpNetTransport` — wraps `UdpClient` + `Socket`. Sets a 2 MiB recv buffer (matches holtburger). Bound to `0.0.0.0:0` by default; constructor accepts an explicit local endpoint for tests that need port reproducibility. - `MockTransport` — in-memory channel with two queues: outbound (datagrams the SUT sent) and inbound (datagrams the test wants the SUT to receive). Tests assert against outbound, inject into inbound. No threads, no async, no time. **Forbidden in `INetTransport`:** - Any knowledge of `PacketHeader`, `PacketHeaderFlags`, ISAAC, fragments, GameMessages. - Dispatching to event handlers (it returns bytes; routing is the next layer up). - Owning a recv loop. The recv loop lives in `IReliableSession.Tick()` or its async equivalent. ### 3.3 `IReliableSession` This is the largest layer. It owns the wire. ```csharp public interface IReliableSession : IDisposable { /// Drive the recv loop once. Call from the host loop or a /// dedicated network thread. Drains all available inbound datagrams, /// fires events for completed GameMessages, flushes pending ACKs and /// retransmits, and emits time-sync updates. void Tick(); /// Send a GameMessage payload. The reliable session /// allocates a sequence number, encodes the header, computes the /// CRC (encrypted if flags require), splits into fragments if the /// payload exceeds the single-fragment limit, and ships via /// INetTransport. void SendGameMessage(ReadOnlySpan payload); /// Send a control packet (handshake, disconnect, echo response). /// Bypasses the GameMessage path; caller supplies the optional-header /// content directly. void SendControl(PacketHeaderFlags flags, ReadOnlySpan optionalContent); /// Begin the handshake. Drives LoginRequest → /// ConnectRequest → ConnectResponse → CharacterList ready, then /// transitions to "ready for EnterWorld" state. void BeginHandshake(string account, string password); /// Advance from CharacterSelection to InWorld. Sends /// CharacterEnterWorldRequest; waits for ServerReady; sends /// CharacterEnterWorld. void EnterWorld(uint characterGuid, string account); /// Disconnect cleanly. Sends Disconnect packet with /// client_id, then flushes and closes the transport. void Disconnect(); // Events surfaced upward: event Action> OnGameMessageReceived; // payload only event Action OnTimeSync; // server time event Action OnHandshakeStateChanged; event Action OnDisconnected; event Action OnEchoStatsUpdated; // optional, dev-mode } ``` **Concrete implementation:** `ReliableSession`. Composes seven sub-components: 1. `PacketCodec` — pure functions: encode, decode, CRC, fragment header pack/parse. Stateless except for the ISAAC streams it borrows. 2. `IsaacStreamPair` — owns `IsaacRandom c2s, s2c` plus a shared "search-and-stash" implementation for out-of-order encrypted-checksum recovery (port from holtburger `crypto.rs:73-93`). 3. `InboundOrderingBuffer` — `BTreeMap`-equivalent (`SortedDictionary` works in C#). Tracks `last_server_seq`, gaps, and feeds `RequestRetransmit` when gaps exceed the rate-limit threshold (1 second, max 115 seq IDs in a 256-seq window — match holtburger constants). 4. `OutboundPacketCache` — LRU dictionary (`max=512`) of recently-sent packets keyed by sequence. On server-issued `RequestRetransmit`, looks up + re-encrypts with current ISAAC + `RETRANSMISSION` flag. Uses `Iteration` field correctly. 5. `AckQueue` — pending-ack list. `IReliableSession.Tick` flushes via piggyback on the next outbound data packet; if no data goes out within the idle threshold, sends a standalone ACK packet. Piggybacks are automatic on every `SendGameMessage`. 6. `FragmentAssembler` — inbound: keyed by `(sequence, fragmentId)`, with TTL eviction (default 30s) for orphaned partials. Outbound: splits payloads >448 bytes into multiple fragments with consistent `id`/`count`/`index`/`queue` per holtburger and ACE conventions. 7. `HandshakeMachine` — state machine: `Idle` → `LoginSent` → `ConnectRequestReceived` → `ConnectResponseQueued` (with 200ms deferred send, non-blocking) → `PortPending` → `PortConfirmed` → `Ready` → `EnterWorldSent` → `InWorld`. Each transition is logged with timestamps for diagnostic replay. **Forbidden in `IReliableSession`:** - Knowing the structure of GameMessage payloads beyond "they are bytes." - Dispatching to typed events for specific opcodes. - Calling into `WorldSession` or game state. ### 3.4 `IGameProtocol` ```csharp public interface IGameProtocol : IDisposable { /// Send a typed game action (0xF7B1 envelope, bumps the /// per-action sequence counter). The implementation builds the /// payload and hands it to IReliableSession.SendGameMessage. void Send(IGameAction action); /// Send a non-action GameMessage (e.g., 0xF657 /// CharacterEnterWorld, 0xF7C8 CharacterEnterWorldRequest, 0xF7E4 /// DddInterrogationResponse, 0xF753 AutonomousPosition, /// 0xF61C MoveToState). void Send(IGameMessage message); // Inbound typed events (one per in-scope opcode): event Action OnCharacterList; event Action OnCreateObject; event Action OnUpdateMotion; event Action OnUpdatePosition; event Action OnDddInterrogation; event Action OnPlayerCreate; event Action OnPlayerTeleport; event Action OnTurbineChat; // ...one per opcode in the matrix... // GameEvent sub-opcode events (one per sub-opcode): event Action OnChannelBroadcast; event Action OnTell; event Action OnUpdateHealth; // ...one per sub-opcode in the matrix... // Unknown / unhandled: event Action OnUnknownMessage; // includes opcode, raw bytes, telemetry } ``` The dispatch table is generated from the opcode matrix at build time (or maintained by hand from the matrix; this is a M.6 sub-decision). Every in-scope opcode has its own typed event; unknown opcodes go to `OnUnknownMessage` with full byte payload so devtools can render them. **Forbidden in `IGameProtocol`:** - Direct UDP I/O. - ISAAC, CRC, fragment work. - Holding onto game state (Characters, current player guid, login state — those live in `WorldSession`). ### 3.5 `WorldSession` (the behavior consumer — not protocol library) After Phase M, `WorldSession` is a thin layer: ```csharp public sealed class WorldSession : IDisposable { private readonly IGameProtocol _protocol; private readonly IReliableSession _reliable; // High-level state public CharacterListEntry[] Characters { get; private set; } public CharacterListEntry? CurrentCharacter { get; private set; } public uint? PlayerGuid { get; private set; } // High-level commands (convenience wrappers around _protocol.Send) public void Login(string account, string password) { ... } public void EnterWorld(int characterIndex) { ... } public void SendTalk(string text) { ... } public void SendTell(string target, string text) { ... } public void SendMove(MoveToState moveState) { ... } // Subscribes to _protocol events in the constructor; routes them // to public events GameWindow / plugins consume. public event Action OnCreateObject; public event Action OnUpdateMotion; // ...etc, mirroring _protocol.On... but at the WorldSession surface // so callers don't reach into the protocol layer directly. } ``` Target line count after migration: 200–400 LOC vs the current 1213 LOC. ### 3.6 Layer dependencies and project structure New project: **`src/AcDream.Net/`**. - `AcDream.Net.Transport` namespace — `INetTransport`, `UdpNetTransport`, `MockTransport`. - `AcDream.Net.Reliable` namespace — `IReliableSession`, `ReliableSession`, sub-components (`PacketCodec`, `IsaacStreamPair`, `InboundOrderingBuffer`, `OutboundPacketCache`, `AckQueue`, `FragmentAssembler`, `HandshakeMachine`), plus `PacketHeader`, `PacketHeaderFlags`, `PacketHeaderOptional`, `MessageFragment` (moved here from `AcDream.Core.Net.Packets`). - `AcDream.Net.Protocol` namespace — `IGameProtocol`, `GameProtocol`, every typed message record, every typed event payload record. Subdivided by class: `Protocol/Messages/`, `Protocol/Events/`, `Protocol/Actions/`. The existing `src/AcDream.Core.Net/` namespace is **deleted at end of phase**. `WorldSession` moves to `src/AcDream.Core/` (it's behavior, not network plumbing). Any helpers in the old namespace migrate into `AcDream.Net.*` if still needed; otherwise they're deleted. Project references: - `AcDream.Net` references `AcDream.Core` (for `IPlatformLogger`, shared types). - `AcDream.Core` references `AcDream.Net` (for the interfaces — `WorldSession` needs `IGameProtocol`, `IReliableSession`). This implies one logical cycle that's broken by interface-only references: `AcDream.Net` only references `AcDream.Core`'s types that don't transitively depend on network code (i.e., logging + result types). If the cycle resists clean breaking, the fallback is a third project `AcDream.Net.Abstractions` for the interfaces, with `AcDream.Net.Implementation` and `AcDream.Core` both depending on it. ### 3.7 What stays out of the architecture (and where it goes) - **Auth / GLS ticket flow** — currently absent. If Phase M needs to support GLS-ticketed login (real retail server flow, not just account/password against ACE), it lives in `AcDream.Net.Reliable.HandshakeMachine` as an additional pre-LoginRequest stage. For now, ACE only accepts account/password, so this is documented as a non-goal until a real-server phase. - **Plugin packet introspection** — surface lives on `WorldSession` (or a separate dev-tool API), not in the protocol library. Exposing raw fragments to plugins is risky; we expose typed events. - **Capture/replay tooling** — lives in `tools/network-conformance-replay/`, depends on `AcDream.Net` but not vice-versa. --- ## 4. Migration strategy ### 4.1 Worktree branch model Phase M ships entirely on a long-lived feature branch off `main`: - Branch name: `claude/phase-m-network-stack` - Worktree path: `.claude/worktrees/phase-m-network-stack/` (per existing repo convention) - All sub-phase commits land on this branch. - `main` is untouched until M.8 acceptance gates close. - Live-ACE testing of the new stack happens by `dotnet run` from the worktree. - Live-ACE testing of the old stack continues to happen from `main`. ### 4.2 Branch lifetime and rebase cadence - **Estimated lifetime:** 6–8 weeks (per cost estimate in §8). - **Rebase cadence:** weekly minimum, plus an immediate rebase whenever any of the following lands on main: - Touches `src/AcDream.Core.Net/`, `src/AcDream.App/Input/PlayerMovementController.cs`, or any networking-adjacent code - Updates `references/holtburger/` (we re-pull and re-baseline our research) - Updates `docs/research/named-retail/` (new symbols may invalidate matrix rows) - Modifies the roadmap in any way that changes Phase M scope - **Conflict resolution policy:** - Wire-format conflicts (main lands a fix to `MoveToState` while we're rewriting it): we adopt the main fix into the new stack, file an issue to verify the same behavior is reproduced post-port. - Test conflicts (main adds a test that exercises the old `WorldSession`): the test moves to test the new stack via the same call site after migration; if the call site is gone, the test is rewritten against the new equivalent. - Build conflicts: standard rebase resolution. - **Frequency check:** if rebase frequency exceeds 2× per week or rebase work consistently exceeds 30 minutes, the branch is too stale. Pause feature work, catch up, then resume. ### 4.3 What ships on the branch vs in separate commits to main - All Phase M code: branch only. - All Phase M tests: branch only. - Roadmap updates (ongoing status, not the final "shipped" entry): cherry-pick to main as the phase progresses, so other agents see status. - Research notes (e.g., new opcode-matrix updates, new findings against ACE/holtburger): land directly on main since they're useful to other phases independent of M. - The opcode matrix doc itself: lives on main from the start (it's reference data, not protected by the migration). ### 4.4 Final merge: M.8 ship gate When M.8 closes: 1. Branch is rebased one final time against current `main`. 2. Full `dotnet build` + `dotnet test` green on the branch. 3. Live-ACE smoke run from the worktree by user: login → walk → chat → combat → portal → logout. 4. Old `src/AcDream.Core.Net/` deleted in a final branch commit (NOT before — this is the load-bearing flip). 5. Branch merged to main as a single `--no-ff` merge commit, message names every sub-phase shipped. 6. Roadmap entry for Phase M moves to "shipped" in the same merge. 7. Memory crib written summarizing the architecture for future sessions. ### 4.5 Rollback path If post-merge live ACE breaks unexpectedly, the rollback is: - `git revert` the merge commit on main - File a bug with the live-ACE failure mode - Cherry-pick the fix onto a new branch off the reverted main - Re-merge Since the merge is a single commit, revert is mechanical. The 6–8 weeks of work isn't lost — it's reachable via the original branch tip + the revert undoing the merge. ### 4.6 Work-in-flight protocol During Phase M, other agents may want to work on other features. The protocol: - Other agents work off main as usual. - They are NOT permitted to touch `src/AcDream.Core.Net/` or any file the spec lists as Phase-M-owned. - If they need to add a new outbound message (e.g., a new gameplay phase needs a new opcode), they file an issue tagged `phase-m-followup` and we incorporate post-merge. - The Phase M branch is the only place network changes happen until M.8 closes. This is enforced by convention, not tooling. The Phase M agent (or human equivalent) communicates in commits + roadmap updates about what's locked. --- ## 5. Sub-phase definitions of done Each sub-phase has: **entry criteria**, **exit criteria**, **conformance test gates**, and an **hour estimate**. ### 5.1 M.1 — Audit & parity map **Entry:** Phase M kickoff. `references/holtburger/` is at known commit (`629695a` as of 2026-05-10). **Exit:** - Opcode matrix at `docs/research/2026-05-10-phase-m-opcode-matrix.md` is filled to ≥95% completeness across all five sections (transport flags, optional headers, GameMessages, GameEvents, GameActions). - For every row marked `–skip:`, the reason is documented and ratified by spec review. - For every row marked `–defer:`, the deferred phase exists in the roadmap. - A meta-section at the top of the matrix lists totals: "in-scope opcodes: N", "currently-implemented: M", "Phase M target delta: N-M". **Conformance gates:** - Spot-check 10 randomly-selected rows by hand against all three sources (holtburger / ACE / named retail). Discrepancies block exit. **Hour estimate:** 16 hours. **Notes:** the holtburger study at `docs/research/2026-05-10-holtburger-network-stack-study.md` is a partial M.1 deliverable. M.1 completion includes building the formal matrix table from that study + per-opcode source citation. ### 5.2 M.2 — Layer extraction (skeleton) **Entry:** M.1 exit gates green. **Exit:** - New project `src/AcDream.Net/` exists with three namespaces (`Transport` / `Reliable` / `Protocol`). - All three interfaces (`INetTransport`, `IReliableSession`, `IGameProtocol`) compile with their full signatures from §3. - `MockTransport` and `UdpNetTransport` implement `INetTransport` with passing unit tests. - Stub implementations of `IReliableSession` and `IGameProtocol` exist (throw `NotImplementedException` on member calls; pass interface compliance tests via the mock). - The new project compiles. The old `src/AcDream.Core.Net/` is unchanged and still works. **Conformance gates:** - `dotnet build` green. - `dotnet test` green for any tests in `tests/AcDream.Net.Tests/` (which at this point covers only `MockTransport` and `UdpNetTransport`). **Hour estimate:** 40 hours. ### 5.3 M.3 — Reliability core **Entry:** M.2 exit gates green. **Exit:** - `IReliableSession`'s `ReliableSession` implementation is functionally complete: codec, ISAAC pair with search-and-stash, inbound ordering buffer, outbound packet cache, retransmit (both directions), `Iteration` field handling, RequestRetransmit issuing on gaps with rate-limit, RejectRetransmit handling. - Sub-component unit tests pass. - An integration test connects to a `MockTransport`, simulates an entire ACE session (login → walk → disconnect) with synthetic loss/reorder, verifies state. - Holtburger study items 1.4 (port-switch race) and 1.7 (retransmit machinery) and ISAAC search-mode (item 6) are landed in this sub-phase. **Conformance gates:** - 100% of unit tests pass. - Integration test with synthetic 5% packet loss: 100% of GameMessages are eventually delivered; no false positives in retransmit requests. - Integration test with synthetic 10% reordering: 100% of GameMessages are delivered in correct order; ISAAC search-mode keys are correctly stashed and consumed. **Hour estimate:** 40 hours. ### 5.4 M.4 — ACK and control-packet policy **Entry:** M.3 exit gates green. **Exit:** - ACK queue with piggyback works: every outbound `SendGameMessage` on `IReliableSession` carries the latest server seq automatically; standalone ACKs flush only when no data goes out within an idle threshold. - EchoRequest handling: inbound EchoRequest triggers an outbound EchoResponse with mirrored time field. - Disconnect packet carries `client_id` (study item 5). - LoginComplete is sent on every PlayerTeleport and on first PlayerCreate (study item 1.2 — but the dispatch happens at the protocol layer, M.6, not here; M.4 ensures the underlying control-packet send path is correct). - Idle ping/timeout: 1 Hz net tick, 15s timeout. **Conformance gates:** - ACK piggyback test: send a series of GameMessages, verify each carries the most recent server seq. - EchoResponse test: receive synthetic EchoRequest, verify EchoResponse goes out within 1 frame with correct time. - Idle timeout test: don't send anything for 15s, verify keepalive fires and timeout doesn't trigger. **Hour estimate:** 16 hours. ### 5.5 M.5 — Fragment and payload completeness **Entry:** M.4 exit gates green. **Exit:** - Inbound fragment assembly with TTL eviction (default 30s) for orphaned partials. - Outbound multi-fragment splitting for payloads >448 bytes. Handles correct `id` / `count` / `index` / `queue` per fragment. - Round-trip tests for: single-fragment, 2-fragment, 5-fragment payloads. **Conformance gates:** - Round-trip test with a 2KB payload: 5 fragments, all assembled correctly on receive. - TTL test: orphan a fragment, verify it's evicted at 30s. - Capture from holtburger or ACE of a real multi-fragment packet (e.g., long appraise text), our fragment assembler reproduces the same field values byte-perfect. **Hour estimate:** 24 hours. ### 5.6 M.6 — Typed protocol surface **Entry:** M.5 exit gates green. Opcode matrix complete (M.1 exit + any deltas from M.2-M.5). **Exit:** - For every opcode marked `PB+W`, `PB`, or `P+W` in the matrix: - Typed message struct exists in `AcDream.Net.Protocol.Messages`, `Events`, or `Actions`. - Parser/builder exists. - Typed event exists on `IGameProtocol` for inbound opcodes. - Round-trip test passes if applicable. - Golden-vector test pins at least one canonical encoding. - The dispatch table in `GameProtocol` routes inbound bytes to the correct typed event. - Unknown opcodes route to `OnUnknownMessage` with full byte payload. **Conformance gates:** - 100% of in-scope opcodes have green tests. - A "round-trip every opcode" meta-test exists that, given a list of golden-vector samples, encodes + decodes each and asserts bit-for-bit equivalence. - The MoveToState wire-format audit (study items 1.1.a-e) lands as part of M.6 — i.e., the new typed `MoveToStateMessage` builder produces wire output matching holtburger's `common.rs:122-186` encoding. **Hour estimate:** 80 hours. **Note:** This is the largest sub-phase. M.6 is parallelizable via agent dispatch — one agent per opcode class (transport flags, GameMessages, GameEvents, GameActions). Estimated single-developer time is 80h; with effective agent dispatch on the implementation, calendar time may compress to 3-5 days. ### 5.7 M.7 — Runtime loop and diagnostics **Entry:** M.6 exit gates green. **Exit:** - The new stack drives a recv loop that drains all available inbound, fires events, flushes pending ACKs/retransmits/ECHO replies, all within a single `Tick()`. - Decode/order/reassembly is moved out of the render tick into either (a) the same render-tick `Tick()` call or (b) a dedicated network thread, depending on M.7's internal decision (logged in the sub-phase commit). - Byte counters: per-direction, per-opcode, exposed via `IGameProtocol.GetTelemetry()`. - Packet capture: `ACDREAM_PCAP=1` env-var dumps every datagram to disk in a parseable format. - Replay tool: `tools/network-conformance-replay/` reads a capture, replays it against the new stack, asserts no decode errors and matching event sequence. - Dev-panel diagnostics: a debug overlay shows current handshake state, ACK depth, retransmit queue depth, byte counters. **Conformance gates:** - A 5-minute live ACE session captures a clean replay; replay against the new stack: zero decode errors. - The render thread's per-frame budget for network work is < 0.5ms median (measured via existing perf instrumentation). **Hour estimate:** 16 hours. ### 5.8 M.8 — Conformance tests and live validation **Entry:** M.7 exit gates green. **Exit:** - All `tests/AcDream.Net.Tests/` tests green: unit, round-trip, golden-vector, integration with synthetic loss/reorder, replay-against-capture. - Live ACE smoke: login → walk to lifestone → chat in /general → engage NPC for combat (one attack) → portal recall → logout. User-confirmed visually + via decode-error counter (must be 0). - The `WorldSession` shrinkage is complete: pre-migration ~1213 LOC, post-migration ≤400 LOC. - The `src/AcDream.Core.Net/` namespace is deleted. - Memory crib written: `memory/project_phase_m_network.md` summarizing layer architecture, key gotchas discovered during implementation, location of opcode matrix. - Roadmap updated: Phase M moves from "PLANNED" to "shipped" with merge commit reference. **Conformance gates:** - All M.1–M.7 exit gates remain green. - Final live ACE smoke green. **Hour estimate:** 24 hours. ### 5.9 Total | Sub-phase | Hours | Cumulative | |-----------|-------|------------| | M.1 — Audit & matrix | 16 | 16 | | M.2 — Layer extraction | 40 | 56 | | M.3 — Reliability core | 40 | 96 | | M.4 — ACK + control | 16 | 112 | | M.5 — Fragments | 24 | 136 | | M.6 — Typed protocol | 80 | 216 | | M.7 — Runtime + diagnostics | 16 | 232 | | M.8 — Tests + live val | 24 | 256 | **Total: 256 hours ≈ 32 working days ≈ 6.4 weeks single-developer.** Realistic with subagent parallelization on M.6 (typed-message implementation) and M.1 (matrix population): 4-6 weeks calendar time. --- ## 6. Conformance test plan ### 6.1 Test surfaces per layer | Layer | Test surface | Backing project | |-------|--------------|-----------------| | Transport | Mock + Udp behavior, recv-buffer sizing, error paths | `tests/AcDream.Net.Tests/Transport/` | | Reliable | Codec round-trip, CRC encrypted+unencrypted, ISAAC search edge cases, ordering buffer scenarios, retransmit cycles, ACK piggyback, Echo, port-switch state machine, fragment assembly + splitting | `tests/AcDream.Net.Tests/Reliable/` | | Protocol | Per-opcode round-trip + golden-vector + unknown-opcode telemetry | `tests/AcDream.Net.Tests/Protocol/` | | End-to-end | Replay-against-capture, live-ACE smoke | `tests/AcDream.Net.Tests/Replay/` + `tools/network-conformance-replay/` | ### 6.2 Golden-vector library structure ``` tests/AcDream.Net.Tests/Fixtures/Golden/ ├── Transport/ │ ├── login_request.bin │ ├── connect_request.bin │ ├── ack_only.bin │ ├── echo_request.bin │ └── ... ├── Messages/ │ ├── 0xF658_character_list.bin │ ├── 0xF61C_movetostate_run_forward.bin │ ├── 0xF753_autonomous_position.bin │ └── ... ├── Events/ │ ├── 0x0147_channel_broadcast.bin │ ├── 0x02BD_tell.bin │ └── ... └── manifests/ └── all-golden.json # (filename, opcode, decoded fields, source citation) ``` Each `.bin` has a sibling `.json` with the decoded fields and source attribution (holtburger capture / named retail trace / ACE-generated). ### 6.3 Live capture replay `tools/network-conformance-replay/` is a small console app: - Reads a `.pcap`-like capture from disk (binary format defined as part of M.7). - For each datagram, hands bytes to a fresh `ReliableSession` + `GameProtocol`. - Asserts: no decode errors, every typed event fires in the expected order (event order is part of the capture metadata), final session state matches the capture's recorded final state. - Output: PASS/FAIL with detailed first-failure diff. ### 6.4 Live ACE smoke flows Two tiers: - **Per-sub-phase smoke** (lightweight, automated where possible): - M.3: handshake completes; CharacterList received; clean disconnect. - M.4: 60-second idle session with ECHO traffic flowing both ways; 0 disconnects. - M.5: a multi-fragment payload from ACE (e.g., long appraise text) parses correctly. - M.6: every opcode the live session naturally produces (login → walk → chat → portal) parses to its typed event. - **M.8 final smoke** (manual, user-driven): - Account login: user enters credentials, picks +Acdream, enters world. - Walk: WASD around Holtburg for 30s; observe local + retail-observer view (via parallel retail client) for blippy movement. - Chat: /general "hello", /tell to a name, /a (allegiance), /f (fellowship). - Combat: target a guard, swing once, observe damage notification + animation. - Portal recall: cast Portal Recall, watch teleport. - Logout: clean disconnect, verify ACE shows session ended. - Decode-error counter must be 0 throughout. ### 6.5 What's not tested at this layer - Game-state correctness: that's per-feature in gameplay phases. - Rendering correctness: that's the existing renderer test surface. - Plugin behavior: separate test surface. --- ## 7. Risk register | # | Risk | Probability | Impact | Mitigation | |---|------|-------------|--------|------------| | 1 | **Branch drift** — main moves faster than expected, rebase work overwhelms. | Medium | High (could double phase calendar time) | Weekly rebase minimum + watchpoints on key files. Pause and catch up if conflict effort exceeds 30min/week. | | 2 | **Opcode ambiguity** — three sources (holtburger / ACE / named retail) disagree on a field layout. | Medium | Medium (delays the affected M.6 row) | Per-row triage: cross-check against live ACE traffic if available; file a research note documenting disagreement; pick the source with strongest evidence; revisit if a real-server-deploy phase invalidates the choice. | | 3 | **ISAAC stream desync** — search-mode port has a subtle bug that corrupts the keystream. | Low | Critical (silent corruption looks like ACE incompat) | Parallel-run old + new ISAAC for 1 week in dev mode; log every divergence; smoke-test with synthetic out-of-order injection. | | 4 | **Live ACE incompat** — new stack works in unit tests but real ACE rejects something subtle. | Medium | High (blocks M.8) | Per-sub-phase live smoke (not just final). Catches incompats early. | | 5 | **Dead-builder integration drift** — Phase B.4 surface (Use/UseWithTarget/PickUp) was built without wiring; we may rebuild without verifying the wiring works. | Medium | Medium (fixes one bug, introduces another) | Every typed builder must have a golden-vector test. The matrix row's "Phase M target" includes "verified against live ACE" for any opcode previously dead-built. | | 6 | **`Iteration` field** — current code always writes 0; if retail uses non-zero iteration on retransmits in a way ACE validates, we get rejected. | Low | Medium (breaks retransmit specifically) | M.3's retransmit test exercises iteration values 0, 1, 2; live-ACE smoke with synthetic loss to trigger real retransmits. | | 7 | **Project structure refactor breaks downstream code** — moving `WorldSession` or deleting `AcDream.Core.Net` shifts a namespace many files reference. | High | Low (compile errors are immediate) | M.8 deletion is the last commit; entire branch compiles up to that point; deletion + namespace fix lands in one commit, single rebuild. | | 8 | **Threading model regression** — if M.7 introduces a network thread, render-thread races appear. | Medium | High (intermittent crashes) | Default to keeping single-threaded model; threading is opt-in via a flag for one test session before becoming default. | | 9 | **Test fixture rot** — golden vectors capture a 2026-05 ACE version; future ACE versions diverge. | Low | Low (fixtures still valid for retail-conformance baseline) | Golden vectors are pinned to retail behavior, not ACE-specific. Live capture replay is from acdream itself (most reproducible). | | 10 | **Calendar overrun** — 6.4 weeks expands to 12+ weeks. | Medium | Medium (delays Phase F+ gameplay phases) | Mid-phase checkpoint at M.4 close (week 3 in plan). If hours-spent ≥ 1.5× estimate, scope-cut M.6 to "matrix-deferred opcodes only, batch the long tail to M.6.b post-merge." | --- ## 8. Cost estimate ### 8.1 Summary **Total estimate: 256 hours ≈ 6.4 working weeks single-developer.** With effective subagent dispatch (especially on M.1 matrix population and M.6 typed-message implementation), realistic calendar compression to **4–6 weeks**. ### 8.2 Cost breakdown by sub-phase (repeating for visibility) | Sub-phase | Hours | Calendar weeks | Subagent-friendly? | |-----------|-------|----------------|--------------------| | M.1 — Audit & matrix | 16 | 0.4 | Yes (per-class agents) | | M.2 — Layer extraction | 40 | 1.0 | Limited (architecture-driven, single voice) | | M.3 — Reliability core | 40 | 1.0 | Limited (ISAAC + ordering buffer interact) | | M.4 — ACK + control | 16 | 0.4 | Limited | | M.5 — Fragments | 24 | 0.6 | Limited | | M.6 — Typed protocol | 80 | 2.0 | **Yes (per-opcode-class agents)** | | M.7 — Runtime + diagnostics | 16 | 0.4 | Limited | | M.8 — Tests + live val | 24 | 0.6 | Limited (live val needs human) | | **Total** | **256** | **6.4** | | ### 8.3 Critical path ``` M.1 → M.2 → M.3 → M.4 → M.5 → M.6 → M.7 → M.8 (mostly sequential within a single-developer flow) ``` M.1 can partially overlap M.2 (matrix work continues while skeleton lands). M.3 / M.4 / M.5 are conceptually parallel within the reliable layer, but practically sequenced because they share state. M.6 is the parallelization cliff — agents work on different opcode classes simultaneously. M.7 / M.8 are sequential. ### 8.4 Resource assumptions - One primary developer driving the architecture and integration. - Subagent dispatch budget: liberal (acdream's sustained pattern is to use Sonnet agents heavily for bounded chunks; per CLAUDE.md "Subagent policy"). - Live ACE on `127.0.0.1:9000` available throughout for smoke tests. - User available for M.8 final visual gate (the only step that genuinely needs human eyes). ### 8.5 What buys schedule slack If budget compresses (e.g., 4 weeks max), the following are scope-cuts in order: 1. **Long-tail GameEvent sub-opcodes** (House*, Trade*, Book*, Vendor*, Barber*, Allegiance updates, ContractTracker*) — 30+ rows that gameplay phases will need eventually but not for M.8 acceptance. Move to a `M.6.b` follow-up. 2. **Outbound multi-fragment splitting** (M.5 second half) — defer until a gameplay phase needs >448-byte outbound payload. 3. **M.7 dev-panel diagnostics** — keep the byte counters and capture, drop the visual overlay. 4. **M.8 replay harness** — keep the smoke gate, drop the automated replay testing (move to follow-up). These cuts get total down to ~150–180 hours / 4 weeks if necessary. The architecture is preserved; the long-tail completeness regresses to "covers everything observed in live ACE during normal play, not the long tail." --- ## Status & next steps **Spec status as of 2026-05-10:** Sections 1–8 written. Awaiting: 1. **Opcode matrix construction** (M.1's main deliverable). Dispatch agents: one per opcode class. Output: `docs/research/2026-05-10-phase-m-opcode-matrix.md`. 2. **Roadmap update.** Phase M entry shrinks to a one-paragraph summary + status table + pointer to this spec. M.0 sub-lane folds into M.3 / M.4 / M.6 (no longer ships separately). **When implementation starts:** create the worktree, branch off main, begin M.1 matrix completion → M.2 skeleton.