acdream/docs/superpowers/specs/2026-05-10-phase-m-network-stack-design.md

Phase M — Network Stack Conformance — Design Spec

Date: 2026-05-10 Status: Draft (sections 1–3 of 8 written; sections 4–8 pending; opcode matrix in flight) Phase identifier: M (per docs/plans/2026-04-11-roadmap.md:414) Supersedes the planned-but-never-written docs/superpowers/specs/2026-05-02-network-stack-conformance.md

Related research:

• docs/research/2026-05-10-holtburger-network-stack-study.md — first-pass parity study, source of recent commit references
• docs/research/2026-05-10-phase-m-opcode-matrix.md — opcode coverage matrix (in flight; this spec links to it as the source of "done")

Reference repos:

• references/holtburger/ — fast-forwarded to 629695a on 2026-05-10
• references/ACE/ — server-side opcode authority
• docs/research/named-retail/ — Sept 2013 EoR PDB-named decomp

---

1. Goal and non-goals

1.1 Goal

Build a complete, layered, testable network protocol library for acdream that covers every wire opcode a 2013 EoR retail client receives, sends, or both — independent of whether each opcode is yet wired into game state. The library is delivered behind three interfaces (INetTransport, IReliableSession, IGameProtocol); the existing WorldSession shrinks to a thin behavior consumer on top. Every parser, builder, and transport feature is unit-tested with golden-vector fixtures and survives a live ACE smoke loop before the phase ships.

The bar is Bar C — "wireable on demand." For every in-scope opcode:

• A typed message struct exists (record with named fields, no raw byte arrays)
• A parser exists if inbound (wire bytes → typed message)
• A builder exists if outbound (typed message → wire bytes)
• A round-trip test exists where applicable
• A golden-vector test exists pinning at least one canonical wire encoding
• Either the opcode is dispatched to a typed event observable by WorldSession, or its dispatch is documented as deferred to the gameplay phase that needs it (with the deferred-target named, e.g., "wired in Phase F")

The behavior layer (what to DO with each message in game state) remains the responsibility of the gameplay phase that needs it. Phase M does not wire HouseStatusUpdate into a house-status panel; it ensures HouseStatusUpdate parses correctly into a typed event so the future Phase consuming it has zero protocol work.

1.2 What "complete" is measured against

The opcode coverage matrix at docs/research/2026-05-10-phase-m-opcode-matrix.md is the source of truth for "done." Per opcode it cites:

• Holtburger coverage (references/holtburger/crates/holtburger-{session,protocol,core}/)
• ACE coverage (references/ACE/Source/ACE.Server/Network/GameMessages/Messages/ for outbound; Source/ACE.Server/Network/Handlers/ for inbound accept rules)
• Named retail decomp (docs/research/named-retail/acclient_2013_pseudo_c.txt, symbols.json — by class::method or address)
• Acdream's current state (parser? builder? wired? deferred? unknown?)
• Phase M target (parse, build, both, or "skip with documented justification")

An opcode is in scope if any of:

• Holtburger or ACE actively sends/receives it
• The named retail decomp shows the 2013 client invoking it
• It appears in observed live ACE traffic on 127.0.0.1:9000

An opcode is out of scope if all of:

• Holtburger doesn't touch it
• ACE marks it server-internal-only or post-2013 (visible in ACE's commit history or comments)
• The named retail decomp shows no client-side reference
• It hasn't been observed on live ACE

Out-of-scope opcodes get one row in the matrix with the justification, no code work.

1.3 Non-goals

• Not reimplementing ACE server behavior. Validations, accept rules, and game-side decisions live in ACE; we mirror only what the client must produce or consume.
• Not replacing acdream's stricter inbound checksum verification. Our PacketCodec validates more aggressively than retail did (per the existing class doc); we keep that unless named retail proves it's wrong.
• Not rewriting renderer, animation, audio, UI, plugin, or chat layers. Those have their own phases. The new network stack must compile under, and run alongside, the current rendering and gameplay code.
• Not introducing async/await across the codebase. The current Tick()-driven recv-loop model is preserved; layer extraction is structural, not asynchrony-restructuring. (We MAY add a dedicated network thread if M.7's runtime work warrants it, but that decision is internal to M.7.)
• Not handling opcodes that are ACE-only invented for emulation purposes (e.g., debug echos that retail never had). The matrix calls these out per row.
• Not optimizing for throughput. Correctness first. Allocation profile and CPU cost tuning is a follow-up phase if the live loop measurably regresses.
• Not plugin-API exposure of network internals. The plugin API gets typed-event subscriptions where useful; raw packet introspection is dev-only.

1.4 What ships at the end of Phase M

When M.8 closes:

• src/AcDream.Net/ (new namespace) contains INetTransport, IReliableSession, IGameProtocol, their concrete implementations, and the typed message library.
• src/AcDream.Core.Net/WorldSession.cs is a behavior consumer ~200–400 LOC, not the current 1213 LOC monolith.
• The tests/AcDream.Net.Tests/ project covers every protocol-layer surface with unit tests.
• A tools/network-conformance-replay/ harness can replay a captured ACE session and verify byte-perfect outputs.
• dotnet build green, dotnet test green, live ACE smoke green: login → walk → chat → combat action → portal → logout, verified by user.
• The roadmap entry for Phase M moves from "PLANNED" to "shipped" with a one-line summary and commit reference.

---

2. Coverage definition

2.1 The opcode matrix

The matrix is a markdown table at docs/research/2026-05-10-phase-m-opcode-matrix.md, grouped by layer:

1. Transport flags — every value in PacketHeaderFlags (LoginRequest, ConnectRequest, AckSequence, EncryptedChecksum, BlobFragments, RequestRetransmit, RejectRetransmit, EchoRequest, EchoResponse, Flow, ServerSwitch, TimeSync, Disconnect, …). Each row says what the flag means, who sets it, and what acdream must do on receive.
2. Optional-header fields — every variable-length section (RequestRetransmit list, RejectRetransmit list, AckSequence, ConnectRequest payload, LoginRequest payload, CICMD, TimeSync, EchoRequest/EchoResponse times, Flow). Each row defines the byte layout and our parse/build status.
3. GameMessage opcodes — every top-level opcode the client sees (0xF658 CharacterList, 0xF745 CreateObject, 0xF74C UpdateMotion, 0xF7B0 GameEvent envelope, 0xF7DE TurbineChat, 0xEA60 AdminEnvirons, …) and every top-level opcode the client sends (0xF7C8 CharacterEnterWorldRequest, 0xF657 CharacterEnterWorld, 0xF61C MoveToState, 0xF61B JumpAction, 0xF753 AutonomousPosition, 0xF7E4 DddInterrogationResponse, …).
4. GameEvent sub-opcodes — every entry in GameEventType.cs (94 currently named; ~70+ currently unhandled). Each row identifies the parsing target plus the acdream wiring status.
5. GameAction sub-opcodes — every typed game-action ID (Talk, Tell, Channel, Use, UseWithTarget, MoveToObject, JumpAbsolute, CastSpell, Appraise, Identify, AttackTargetMelee/Missile, Allegiance ops, Inventory ops, Social ops, Skill/Attribute raise, Train, …).

Each row has these columns:

| Code | Direction | Name | Named-retail symbol or address | Holtburger | ACE | acdream today | Phase M target | Notes |

Cell values for "Holtburger" / "ACE" / "acdream today":

• P — parses inbound
• B — builds outbound
• PB — both
• W — wired (parser/builder + dispatched to typed event consumed somewhere)
• – — not implemented
• N/A — not applicable for this side (e.g., a server-only message in ACE column)

"Phase M target" cell values:

• PB+W — must parse, build (if outbound), wire to a typed event by phase end
• PB — must parse, build (if outbound), no wiring required
• P+W — inbound only, must parse and dispatch typed event
• –defer:<phase> — explicitly deferred to a named gameplay phase
• –skip:<reason> — out of scope, with justification

2.2 Inbound parser obligations

For every in-scope inbound opcode:

• A typed C# record represents the message. Fields are named, typed, and ordered to match the wire layout (so a future reader can map field-to-byte without re-reading the parser).
• The parser is a static method on the record (public static MyMessage Parse(ref BinaryReader r)), throws InvalidOperationException on malformed input with a message containing the opcode and offset.
• A round-trip test exists if the opcode is also outbound. A golden-vector test always exists with at least one specific captured wire encoding.
• The parser dispatches to a typed event on IGameProtocol (event Action<MyMessage> OnMyMessage). If wiring to game state is deferred, the matrix row says –defer:<phase> and the typed event still exists — gameplay-phase wiring is then a one-line subscription.

2.3 Outbound builder obligations

For every in-scope outbound opcode:

• A typed C# record represents the message.
• A Build(ref BinaryWriter w) instance method writes the wire encoding.
• A golden-vector test pins at least one specific wire encoding.
• The high-level entry point lives on IGameProtocol (Send(MyAction act) or Send(MyMessage msg)).
• WorldSession exposes a behavior-friendly wrapper (SendTalk(string text) rather than _protocol.Send(new TalkMessage { … })) only for opcodes the user-facing app currently triggers. Less-used outbound builders stay on IGameProtocol directly until a gameplay phase needs the convenience wrapper.

2.4 Three test fixture sources

• Golden vectors. Hand-computed bytes for representative messages. Source: named retail decomp (extract via tools/pdb-extract/), holtburger captures, or by manual trace. Stored in tests/AcDream.Net.Tests/Fixtures/Golden/<opcode>.bin plus a sibling .json describing the fields.
• Live capture replay. A captured session log (raw datagrams + timestamps) replayed offline against the new stack. Captures come from running acdream itself with a ACDREAM_PCAP=1 env-var that dumps every datagram to disk. The first capture is recorded once Phase M.7's runtime is in place; subsequent captures replace it as features land.
• Live ACE smoke. Per-sub-phase, a live dotnet run against 127.0.0.1:9000 that exercises the relevant features. Final M.8 smoke covers login → walk → chat → combat action → teleport → reconnect → logout end-to-end.

2.5 Acceptance for an in-scope opcode

An opcode is "done" for Phase M when:

1. Its matrix row is filled completely.
2. The typed message struct exists and matches the documented byte layout.
3. The parser and/or builder exist and pass round-trip tests where applicable.
4. At least one golden-vector test pins a canonical encoding.
5. The typed event is exposed on IGameProtocol (inbound) or the high-level send method exists (outbound).
6. The matrix row's acdream today column is updated to match Phase M target.

The opcode-class agents working on the matrix produce the per-row data. Phase M.6 implementation work is then "for each row in the matrix where target ≠ today, write the code and tests."

---

3. Three-layer architecture

3.1 Layer overview

┌─────────────────────────────────────────────────────────────┐
│  WorldSession (behavior layer — not part of Phase M's       │
│  protocol library; consumes IGameProtocol)                  │
└────────────────────────────┬────────────────────────────────┘
                             │ subscribes to typed events,
                             │ calls Send(IGameMessage|IGameAction)
┌────────────────────────────▼────────────────────────────────┐
│  IGameProtocol — typed message routing                      │
│    • opcode dispatch table                                  │
│    • GameAction sequence counter                            │
│    • per-message typed events                               │
│    • outbound: typed message → bytes via builder            │
└────────────────────────────┬────────────────────────────────┘
                             │ delivers fully-assembled GameMessage
                             │ payloads; receives outbound payloads
┌────────────────────────────▼────────────────────────────────┐
│  IReliableSession — wire correctness                        │
│    • PacketCodec (header + optional + body framing, CRC,    │
│      ISAAC c2s/s2c, fragment header layout)                 │
│    • inbound ordering buffer + RequestRetransmit issuing    │
│    • outbound packet cache + retransmit on server request   │
│    • ACK queue + piggyback                                  │
│    • EchoRequest reply, TimeSync forwarding                 │
│    • port-switch state machine                              │
│    • fragment assembly (inbound) + splitting (outbound)     │
└────────────────────────────┬────────────────────────────────┘
                             │ INetTransport.Send(bytes, endpoint)
                             │ INetTransport.TryReceive(out bytes, out endpoint)
┌────────────────────────────▼────────────────────────────────┐
│  INetTransport — UDP only                                   │
│    • Send / TryReceive / Close                              │
│    • no protocol knowledge                                  │
│    • UdpNetTransport (prod) / MockTransport (test)          │
└─────────────────────────────────────────────────────────────┘

Hard rules on direction:

• Higher layers know about lower layers; lower layers do not know about higher layers.
• IGameProtocol does not call into INetTransport; it must go through IReliableSession.
• WorldSession does not directly construct UDP packets, ISAAC streams, or fragment headers.
• A unit test for any layer can mock the layer below it.

3.2 INetTransport

public interface INetTransport : IDisposable
{
    /// <summary>
    /// Send a single UDP datagram to the given endpoint. Synchronous.
    /// Returns the number of bytes sent (always == datagram.Length on
    /// success). Throws on socket error.
    /// </summary>
    int Send(ReadOnlySpan<byte> datagram, IPEndPoint remote);

    /// <summary>
    /// Non-blocking receive. Returns false if no datagram is available.
    /// On true, datagram contains the bytes (caller must not retain
    /// the returned span past the next call) and remote contains the
    /// source endpoint.
    /// </summary>
    bool TryReceive(out ReadOnlySpan<byte> datagram, out IPEndPoint remote);

    /// <summary>
    /// Local endpoint we are bound to (after construction).
    /// </summary>
    IPEndPoint LocalEndpoint { get; }
}

Concrete implementations:

• UdpNetTransport — wraps UdpClient + Socket. Sets a 2 MiB recv buffer (matches holtburger). Bound to 0.0.0.0:0 by default; constructor accepts an explicit local endpoint for tests that need port reproducibility.
• MockTransport — in-memory channel with two queues: outbound (datagrams the SUT sent) and inbound (datagrams the test wants the SUT to receive). Tests assert against outbound, inject into inbound. No threads, no async, no time.

Forbidden in INetTransport:

• Any knowledge of PacketHeader, PacketHeaderFlags, ISAAC, fragments, GameMessages.
• Dispatching to event handlers (it returns bytes; routing is the next layer up).
• Owning a recv loop. The recv loop lives in IReliableSession.Tick() or its async equivalent.

3.3 IReliableSession

This is the largest layer. It owns the wire.

public interface IReliableSession : IDisposable
{
    /// <summary>Drive the recv loop once. Call from the host loop or a
    /// dedicated network thread. Drains all available inbound datagrams,
    /// fires events for completed GameMessages, flushes pending ACKs and
    /// retransmits, and emits time-sync updates.</summary>
    void Tick();

    /// <summary>Send a GameMessage payload. The reliable session
    /// allocates a sequence number, encodes the header, computes the
    /// CRC (encrypted if flags require), splits into fragments if the
    /// payload exceeds the single-fragment limit, and ships via
    /// INetTransport.</summary>
    void SendGameMessage(ReadOnlySpan<byte> payload);

    /// <summary>Send a control packet (handshake, disconnect, echo response).
    /// Bypasses the GameMessage path; caller supplies the optional-header
    /// content directly.</summary>
    void SendControl(PacketHeaderFlags flags, ReadOnlySpan<byte> optionalContent);

    /// <summary>Begin the handshake. Drives LoginRequest →
    /// ConnectRequest → ConnectResponse → CharacterList ready, then
    /// transitions to "ready for EnterWorld" state.</summary>
    void BeginHandshake(string account, string password);

    /// <summary>Advance from CharacterSelection to InWorld. Sends
    /// CharacterEnterWorldRequest; waits for ServerReady; sends
    /// CharacterEnterWorld.</summary>
    void EnterWorld(uint characterGuid, string account);

    /// <summary>Disconnect cleanly. Sends Disconnect packet with
    /// client_id, then flushes and closes the transport.</summary>
    void Disconnect();

    // Events surfaced upward:
    event Action<ReadOnlySpan<byte>> OnGameMessageReceived;     // payload only
    event Action<double>             OnTimeSync;                // server time
    event Action<HandshakeState>     OnHandshakeStateChanged;
    event Action<DisconnectReason>   OnDisconnected;
    event Action<EchoStats>          OnEchoStatsUpdated;        // optional, dev-mode
}

Concrete implementation: ReliableSession. Composes seven sub-components:

1. PacketCodec — pure functions: encode, decode, CRC, fragment header pack/parse. Stateless except for the ISAAC streams it borrows.
2. IsaacStreamPair — owns IsaacRandom c2s, s2c plus a shared "search-and-stash" implementation for out-of-order encrypted-checksum recovery (port from holtburger crypto.rs:73-93).
3. InboundOrderingBuffer — BTreeMap<uint, BufferedPacket>-equivalent (SortedDictionary<uint, BufferedPacket> works in C#). Tracks last_server_seq, gaps, and feeds RequestRetransmit when gaps exceed the rate-limit threshold (1 second, max 115 seq IDs in a 256-seq window — match holtburger constants).
4. OutboundPacketCache — LRU dictionary (max=512) of recently-sent packets keyed by sequence. On server-issued RequestRetransmit, looks up + re-encrypts with current ISAAC + RETRANSMISSION flag. Uses Iteration field correctly.
5. AckQueue — pending-ack list. IReliableSession.Tick flushes via piggyback on the next outbound data packet; if no data goes out within the idle threshold, sends a standalone ACK packet. Piggybacks are automatic on every SendGameMessage.
6. FragmentAssembler — inbound: keyed by (sequence, fragmentId), with TTL eviction (default 30s) for orphaned partials. Outbound: splits payloads >448 bytes into multiple fragments with consistent id/count/index/queue per holtburger and ACE conventions.
7. HandshakeMachine — state machine: Idle → LoginSent → ConnectRequestReceived → ConnectResponseQueued (with 200ms deferred send, non-blocking) → PortPending → PortConfirmed → Ready → EnterWorldSent → InWorld. Each transition is logged with timestamps for diagnostic replay.

Forbidden in IReliableSession:

• Knowing the structure of GameMessage payloads beyond "they are bytes."
• Dispatching to typed events for specific opcodes.
• Calling into WorldSession or game state.

3.4 IGameProtocol

public interface IGameProtocol : IDisposable
{
    /// <summary>Send a typed game action (0xF7B1 envelope, bumps the
    /// per-action sequence counter). The implementation builds the
    /// payload and hands it to IReliableSession.SendGameMessage.</summary>
    void Send(IGameAction action);

    /// <summary>Send a non-action GameMessage (e.g., 0xF657
    /// CharacterEnterWorld, 0xF7C8 CharacterEnterWorldRequest, 0xF7E4
    /// DddInterrogationResponse, 0xF753 AutonomousPosition,
    /// 0xF61C MoveToState).</summary>
    void Send(IGameMessage message);

    // Inbound typed events (one per in-scope opcode):
    event Action<CharacterListMessage>   OnCharacterList;
    event Action<CreateObjectMessage>    OnCreateObject;
    event Action<UpdateMotionMessage>    OnUpdateMotion;
    event Action<UpdatePositionMessage>  OnUpdatePosition;
    event Action<DddInterrogationMessage> OnDddInterrogation;
    event Action<PlayerCreateMessage>    OnPlayerCreate;
    event Action<PlayerTeleportMessage>  OnPlayerTeleport;
    event Action<TurbineChatMessage>     OnTurbineChat;
    // ...one per opcode in the matrix...

    // GameEvent sub-opcode events (one per sub-opcode):
    event Action<ChannelBroadcastEvent>  OnChannelBroadcast;
    event Action<TellEvent>              OnTell;
    event Action<UpdateHealthEvent>      OnUpdateHealth;
    // ...one per sub-opcode in the matrix...

    // Unknown / unhandled:
    event Action<UnknownMessage> OnUnknownMessage;  // includes opcode, raw bytes, telemetry
}

The dispatch table is generated from the opcode matrix at build time (or maintained by hand from the matrix; this is a M.6 sub-decision). Every in-scope opcode has its own typed event; unknown opcodes go to OnUnknownMessage with full byte payload so devtools can render them.

Forbidden in IGameProtocol:

• Direct UDP I/O.
• ISAAC, CRC, fragment work.
• Holding onto game state (Characters, current player guid, login state — those live in WorldSession).

3.5 WorldSession (the behavior consumer — not protocol library)

After Phase M, WorldSession is a thin layer:

public sealed class WorldSession : IDisposable
{
    private readonly IGameProtocol _protocol;
    private readonly IReliableSession _reliable;

    // High-level state
    public CharacterListEntry[] Characters { get; private set; }
    public CharacterListEntry? CurrentCharacter { get; private set; }
    public uint? PlayerGuid { get; private set; }

    // High-level commands (convenience wrappers around _protocol.Send)
    public void Login(string account, string password) { ... }
    public void EnterWorld(int characterIndex) { ... }
    public void SendTalk(string text) { ... }
    public void SendTell(string target, string text) { ... }
    public void SendMove(MoveToState moveState) { ... }

    // Subscribes to _protocol events in the constructor; routes them
    // to public events GameWindow / plugins consume.
    public event Action<CreateObjectMessage> OnCreateObject;
    public event Action<UpdateMotionMessage> OnUpdateMotion;
    // ...etc, mirroring _protocol.On... but at the WorldSession surface
    //    so callers don't reach into the protocol layer directly.
}

Target line count after migration: 200–400 LOC vs the current 1213 LOC.

3.6 Layer dependencies and project structure

New project: src/AcDream.Net/.

• AcDream.Net.Transport namespace — INetTransport, UdpNetTransport, MockTransport.
• AcDream.Net.Reliable namespace — IReliableSession, ReliableSession, sub-components (PacketCodec, IsaacStreamPair, InboundOrderingBuffer, OutboundPacketCache, AckQueue, FragmentAssembler, HandshakeMachine), plus PacketHeader, PacketHeaderFlags, PacketHeaderOptional, MessageFragment (moved here from AcDream.Core.Net.Packets).
• AcDream.Net.Protocol namespace — IGameProtocol, GameProtocol, every typed message record, every typed event payload record. Subdivided by class: Protocol/Messages/, Protocol/Events/, Protocol/Actions/.

The existing src/AcDream.Core.Net/ namespace is deleted at end of phase. WorldSession moves to src/AcDream.Core/ (it's behavior, not network plumbing). Any helpers in the old namespace migrate into AcDream.Net.* if still needed; otherwise they're deleted.

Project references:

• AcDream.Net references AcDream.Core (for IPlatformLogger, shared types).
• AcDream.Core references AcDream.Net (for the interfaces — WorldSession needs IGameProtocol, IReliableSession).

This implies one logical cycle that's broken by interface-only references: AcDream.Net only references AcDream.Core's types that don't transitively depend on network code (i.e., logging + result types). If the cycle resists clean breaking, the fallback is a third project AcDream.Net.Abstractions for the interfaces, with AcDream.Net.Implementation and AcDream.Core both depending on it.

3.7 What stays out of the architecture (and where it goes)

• Auth / GLS ticket flow — currently absent. If Phase M needs to support GLS-ticketed login (real retail server flow, not just account/password against ACE), it lives in AcDream.Net.Reliable.HandshakeMachine as an additional pre-LoginRequest stage. For now, ACE only accepts account/password, so this is documented as a non-goal until a real-server phase.
• Plugin packet introspection — surface lives on WorldSession (or a separate dev-tool API), not in the protocol library. Exposing raw fragments to plugins is risky; we expose typed events.
• Capture/replay tooling — lives in tools/network-conformance-replay/, depends on AcDream.Net but not vice-versa.

---

4. Migration strategy

4.1 Worktree branch model

Phase M ships entirely on a long-lived feature branch off main:

• Branch name: claude/phase-m-network-stack
• Worktree path: .claude/worktrees/phase-m-network-stack/ (per existing repo convention)
• All sub-phase commits land on this branch.
• main is untouched until M.8 acceptance gates close.
• Live-ACE testing of the new stack happens by dotnet run from the worktree.
• Live-ACE testing of the old stack continues to happen from main.

4.2 Branch lifetime and rebase cadence

• Estimated lifetime: 6–8 weeks (per cost estimate in §8).

• Rebase cadence: weekly minimum, plus an immediate rebase whenever any of the following lands on main:

  • Touches src/AcDream.Core.Net/, src/AcDream.App/Input/PlayerMovementController.cs, or any networking-adjacent code
  • Updates references/holtburger/ (we re-pull and re-baseline our research)
  • Updates docs/research/named-retail/ (new symbols may invalidate matrix rows)
  • Modifies the roadmap in any way that changes Phase M scope

• Conflict resolution policy:

  • Wire-format conflicts (main lands a fix to MoveToState while we're rewriting it): we adopt the main fix into the new stack, file an issue to verify the same behavior is reproduced post-port.
  • Test conflicts (main adds a test that exercises the old WorldSession): the test moves to test the new stack via the same call site after migration; if the call site is gone, the test is rewritten against the new equivalent.
  • Build conflicts: standard rebase resolution.

• Frequency check: if rebase frequency exceeds 2× per week or rebase work consistently exceeds 30 minutes, the branch is too stale. Pause feature work, catch up, then resume.

4.3 What ships on the branch vs in separate commits to main

• All Phase M code: branch only.
• All Phase M tests: branch only.
• Roadmap updates (ongoing status, not the final "shipped" entry): cherry-pick to main as the phase progresses, so other agents see status.
• Research notes (e.g., new opcode-matrix updates, new findings against ACE/holtburger): land directly on main since they're useful to other phases independent of M.
• The opcode matrix doc itself: lives on main from the start (it's reference data, not protected by the migration).

4.4 Final merge: M.8 ship gate

When M.8 closes:

1. Branch is rebased one final time against current main.
2. Full dotnet build + dotnet test green on the branch.
3. Live-ACE smoke run from the worktree by user: login → walk → chat → combat → portal → logout.
4. Old src/AcDream.Core.Net/ deleted in a final branch commit (NOT before — this is the load-bearing flip).
5. Branch merged to main as a single --no-ff merge commit, message names every sub-phase shipped.
6. Roadmap entry for Phase M moves to "shipped" in the same merge.
7. Memory crib written summarizing the architecture for future sessions.

4.5 Rollback path

If post-merge live ACE breaks unexpectedly, the rollback is:

• git revert the merge commit on main
• File a bug with the live-ACE failure mode
• Cherry-pick the fix onto a new branch off the reverted main
• Re-merge

Since the merge is a single commit, revert is mechanical. The 6–8 weeks of work isn't lost — it's reachable via the original branch tip + the revert undoing the merge.

4.6 Work-in-flight protocol

During Phase M, other agents may want to work on other features. The protocol:

• Other agents work off main as usual.
• They are NOT permitted to touch src/AcDream.Core.Net/ or any file the spec lists as Phase-M-owned.
• If they need to add a new outbound message (e.g., a new gameplay phase needs a new opcode), they file an issue tagged phase-m-followup and we incorporate post-merge.
• The Phase M branch is the only place network changes happen until M.8 closes.

This is enforced by convention, not tooling. The Phase M agent (or human equivalent) communicates in commits + roadmap updates about what's locked.

---

5. Sub-phase definitions of done

Each sub-phase has: entry criteria, exit criteria, conformance test gates, and an hour estimate.

5.1 M.1 — Audit & parity map

Entry: Phase M kickoff. references/holtburger/ is at known commit (629695a as of 2026-05-10).

Exit:

• Opcode matrix at docs/research/2026-05-10-phase-m-opcode-matrix.md is filled to ≥95% completeness across all five sections (transport flags, optional headers, GameMessages, GameEvents, GameActions).
• For every row marked –skip:<reason>, the reason is documented and ratified by spec review.
• For every row marked –defer:<phase>, the deferred phase exists in the roadmap.
• A meta-section at the top of the matrix lists totals: "in-scope opcodes: N", "currently-implemented: M", "Phase M target delta: N-M".

Conformance gates:

• Spot-check 10 randomly-selected rows by hand against all three sources (holtburger / ACE / named retail). Discrepancies block exit.

Hour estimate: 16 hours.

Notes: the holtburger study at docs/research/2026-05-10-holtburger-network-stack-study.md is a partial M.1 deliverable. M.1 completion includes building the formal matrix table from that study + per-opcode source citation.

5.2 M.2 — Layer extraction (skeleton)

Entry: M.1 exit gates green.

Exit:

• New project src/AcDream.Net/ exists with three namespaces (Transport / Reliable / Protocol).
• All three interfaces (INetTransport, IReliableSession, IGameProtocol) compile with their full signatures from §3.
• MockTransport and UdpNetTransport implement INetTransport with passing unit tests.
• Stub implementations of IReliableSession and IGameProtocol exist (throw NotImplementedException on member calls; pass interface compliance tests via the mock).
• The new project compiles. The old src/AcDream.Core.Net/ is unchanged and still works.

Conformance gates:

• dotnet build green.
• dotnet test green for any tests in tests/AcDream.Net.Tests/ (which at this point covers only MockTransport and UdpNetTransport).

Hour estimate: 40 hours.

5.3 M.3 — Reliability core

Entry: M.2 exit gates green.

Exit:

• IReliableSession's ReliableSession implementation is functionally complete: codec, ISAAC pair with search-and-stash, inbound ordering buffer, outbound packet cache, retransmit (both directions), Iteration field handling, RequestRetransmit issuing on gaps with rate-limit, RejectRetransmit handling.
• Sub-component unit tests pass.
• An integration test connects to a MockTransport, simulates an entire ACE session (login → walk → disconnect) with synthetic loss/reorder, verifies state.
• Holtburger study items 1.4 (port-switch race) and 1.7 (retransmit machinery) and ISAAC search-mode (item 6) are landed in this sub-phase.

Conformance gates:

• 100% of unit tests pass.
• Integration test with synthetic 5% packet loss: 100% of GameMessages are eventually delivered; no false positives in retransmit requests.
• Integration test with synthetic 10% reordering: 100% of GameMessages are delivered in correct order; ISAAC search-mode keys are correctly stashed and consumed.

Hour estimate: 40 hours.

5.4 M.4 — ACK and control-packet policy

Entry: M.3 exit gates green.

Exit:

• ACK queue with piggyback works: every outbound SendGameMessage on IReliableSession carries the latest server seq automatically; standalone ACKs flush only when no data goes out within an idle threshold.
• EchoRequest handling: inbound EchoRequest triggers an outbound EchoResponse with mirrored time field.
• Disconnect packet carries client_id (study item 5).
• LoginComplete is sent on every PlayerTeleport and on first PlayerCreate (study item 1.2 — but the dispatch happens at the protocol layer, M.6, not here; M.4 ensures the underlying control-packet send path is correct).
• Idle ping/timeout: 1 Hz net tick, 15s timeout.

Conformance gates:

• ACK piggyback test: send a series of GameMessages, verify each carries the most recent server seq.
• EchoResponse test: receive synthetic EchoRequest, verify EchoResponse goes out within 1 frame with correct time.
• Idle timeout test: don't send anything for 15s, verify keepalive fires and timeout doesn't trigger.

Hour estimate: 16 hours.

5.5 M.5 — Fragment and payload completeness

Entry: M.4 exit gates green.

Exit:

• Inbound fragment assembly with TTL eviction (default 30s) for orphaned partials.
• Outbound multi-fragment splitting for payloads >448 bytes. Handles correct id / count / index / queue per fragment.
• Round-trip tests for: single-fragment, 2-fragment, 5-fragment payloads.

Conformance gates:

• Round-trip test with a 2KB payload: 5 fragments, all assembled correctly on receive.
• TTL test: orphan a fragment, verify it's evicted at 30s.
• Capture from holtburger or ACE of a real multi-fragment packet (e.g., long appraise text), our fragment assembler reproduces the same field values byte-perfect.

Hour estimate: 24 hours.

5.6 M.6 — Typed protocol surface

Entry: M.5 exit gates green. Opcode matrix complete (M.1 exit + any deltas from M.2-M.5).

Exit:

• For every opcode marked PB+W, PB, or P+W in the matrix:
  • Typed message struct exists in AcDream.Net.Protocol.Messages, Events, or Actions.
  • Parser/builder exists.
  • Typed event exists on IGameProtocol for inbound opcodes.
  • Round-trip test passes if applicable.
  • Golden-vector test pins at least one canonical encoding.
• The dispatch table in GameProtocol routes inbound bytes to the correct typed event.
• Unknown opcodes route to OnUnknownMessage with full byte payload.

Conformance gates:

• 100% of in-scope opcodes have green tests.
• A "round-trip every opcode" meta-test exists that, given a list of golden-vector samples, encodes + decodes each and asserts bit-for-bit equivalence.
• The MoveToState wire-format audit (study items 1.1.a-e) lands as part of M.6 — i.e., the new typed MoveToStateMessage builder produces wire output matching holtburger's common.rs:122-186 encoding.

Hour estimate: 80 hours.

Note: This is the largest sub-phase. M.6 is parallelizable via agent dispatch — one agent per opcode class (transport flags, GameMessages, GameEvents, GameActions). Estimated single-developer time is 80h; with effective agent dispatch on the implementation, calendar time may compress to 3-5 days.

5.7 M.7 — Runtime loop and diagnostics

Entry: M.6 exit gates green.

Exit:

• The new stack drives a recv loop that drains all available inbound, fires events, flushes pending ACKs/retransmits/ECHO replies, all within a single Tick().
• Decode/order/reassembly is moved out of the render tick into either (a) the same render-tick Tick() call or (b) a dedicated network thread, depending on M.7's internal decision (logged in the sub-phase commit).
• Byte counters: per-direction, per-opcode, exposed via IGameProtocol.GetTelemetry().
• Packet capture: ACDREAM_PCAP=1 env-var dumps every datagram to disk in a parseable format.
• Replay tool: tools/network-conformance-replay/ reads a capture, replays it against the new stack, asserts no decode errors and matching event sequence.
• Dev-panel diagnostics: a debug overlay shows current handshake state, ACK depth, retransmit queue depth, byte counters.

Conformance gates:

• A 5-minute live ACE session captures a clean replay; replay against the new stack: zero decode errors.
• The render thread's per-frame budget for network work is < 0.5ms median (measured via existing perf instrumentation).

Hour estimate: 16 hours.

5.8 M.8 — Conformance tests and live validation

Entry: M.7 exit gates green.

Exit:

• All tests/AcDream.Net.Tests/ tests green: unit, round-trip, golden-vector, integration with synthetic loss/reorder, replay-against-capture.
• Live ACE smoke: login → walk to lifestone → chat in /general → engage NPC for combat (one attack) → portal recall → logout. User-confirmed visually + via decode-error counter (must be 0).
• The WorldSession shrinkage is complete: pre-migration ~1213 LOC, post-migration ≤400 LOC.
• The src/AcDream.Core.Net/ namespace is deleted.
• Memory crib written: memory/project_phase_m_network.md summarizing layer architecture, key gotchas discovered during implementation, location of opcode matrix.
• Roadmap updated: Phase M moves from "PLANNED" to "shipped" with merge commit reference.

Conformance gates:

• All M.1–M.7 exit gates remain green.
• Final live ACE smoke green.

Hour estimate: 24 hours.

5.9 Total

Sub-phase	Hours	Cumulative
M.1 — Audit & matrix	16	16
M.2 — Layer extraction	40	56
M.3 — Reliability core	40	96
M.4 — ACK + control	16	112
M.5 — Fragments	24	136
M.6 — Typed protocol	80	216
M.7 — Runtime + diagnostics	16	232
M.8 — Tests + live val	24	256

Total: 256 hours ≈ 32 working days ≈ 6.4 weeks single-developer.

Realistic with subagent parallelization on M.6 (typed-message implementation) and M.1 (matrix population): 4-6 weeks calendar time.

---

6. Conformance test plan

6.1 Test surfaces per layer

Layer	Test surface	Backing project
Transport	Mock + Udp behavior, recv-buffer sizing, error paths	tests/AcDream.Net.Tests/Transport/
Reliable	Codec round-trip, CRC encrypted+unencrypted, ISAAC search edge cases, ordering buffer scenarios, retransmit cycles, ACK piggyback, Echo, port-switch state machine, fragment assembly + splitting	tests/AcDream.Net.Tests/Reliable/
Protocol	Per-opcode round-trip + golden-vector + unknown-opcode telemetry	tests/AcDream.Net.Tests/Protocol/
End-to-end	Replay-against-capture, live-ACE smoke	tests/AcDream.Net.Tests/Replay/ + tools/network-conformance-replay/

6.2 Golden-vector library structure

tests/AcDream.Net.Tests/Fixtures/Golden/
├── Transport/
│   ├── login_request.bin
│   ├── connect_request.bin
│   ├── ack_only.bin
│   ├── echo_request.bin
│   └── ...
├── Messages/
│   ├── 0xF658_character_list.bin
│   ├── 0xF61C_movetostate_run_forward.bin
│   ├── 0xF753_autonomous_position.bin
│   └── ...
├── Events/
│   ├── 0x0147_channel_broadcast.bin
│   ├── 0x02BD_tell.bin
│   └── ...
└── manifests/
    └── all-golden.json   # (filename, opcode, decoded fields, source citation)

Each .bin has a sibling .json with the decoded fields and source attribution (holtburger capture / named retail trace / ACE-generated).

6.3 Live capture replay

tools/network-conformance-replay/ is a small console app:

• Reads a .pcap-like capture from disk (binary format defined as part of M.7).
• For each datagram, hands bytes to a fresh ReliableSession + GameProtocol.
• Asserts: no decode errors, every typed event fires in the expected order (event order is part of the capture metadata), final session state matches the capture's recorded final state.
• Output: PASS/FAIL with detailed first-failure diff.

6.4 Live ACE smoke flows

Two tiers:

• Per-sub-phase smoke (lightweight, automated where possible):

  • M.3: handshake completes; CharacterList received; clean disconnect.
  • M.4: 60-second idle session with ECHO traffic flowing both ways; 0 disconnects.
  • M.5: a multi-fragment payload from ACE (e.g., long appraise text) parses correctly.
  • M.6: every opcode the live session naturally produces (login → walk → chat → portal) parses to its typed event.

• M.8 final smoke (manual, user-driven):

  • Account login: user enters credentials, picks +Acdream, enters world.
  • Walk: WASD around Holtburg for 30s; observe local + retail-observer view (via parallel retail client) for blippy movement.
  • Chat: /general "hello", /tell to a name, /a (allegiance), /f (fellowship).
  • Combat: target a guard, swing once, observe damage notification + animation.
  • Portal recall: cast Portal Recall, watch teleport.
  • Logout: clean disconnect, verify ACE shows session ended.
  • Decode-error counter must be 0 throughout.

6.5 What's not tested at this layer

• Game-state correctness: that's per-feature in gameplay phases.
• Rendering correctness: that's the existing renderer test surface.
• Plugin behavior: separate test surface.

---

7. Risk register

#	Risk	Probability	Impact	Mitigation
1	Branch drift — main moves faster than expected, rebase work overwhelms.	Medium	High (could double phase calendar time)	Weekly rebase minimum + watchpoints on key files. Pause and catch up if conflict effort exceeds 30min/week.
2	Opcode ambiguity — three sources (holtburger / ACE / named retail) disagree on a field layout.	Medium	Medium (delays the affected M.6 row)	Per-row triage: cross-check against live ACE traffic if available; file a research note documenting disagreement; pick the source with strongest evidence; revisit if a real-server-deploy phase invalidates the choice.
3	ISAAC stream desync — search-mode port has a subtle bug that corrupts the keystream.	Low	Critical (silent corruption looks like ACE incompat)	Parallel-run old + new ISAAC for 1 week in dev mode; log every divergence; smoke-test with synthetic out-of-order injection.
4	Live ACE incompat — new stack works in unit tests but real ACE rejects something subtle.	Medium	High (blocks M.8)	Per-sub-phase live smoke (not just final). Catches incompats early.
5	Dead-builder integration drift — Phase B.4 surface (Use/UseWithTarget/PickUp) was built without wiring; we may rebuild without verifying the wiring works.	Medium	Medium (fixes one bug, introduces another)	Every typed builder must have a golden-vector test. The matrix row's "Phase M target" includes "verified against live ACE" for any opcode previously dead-built.
6	Iteration field — current code always writes 0; if retail uses non-zero iteration on retransmits in a way ACE validates, we get rejected.	Low	Medium (breaks retransmit specifically)	M.3's retransmit test exercises iteration values 0, 1, 2; live-ACE smoke with synthetic loss to trigger real retransmits.
7	Project structure refactor breaks downstream code — moving WorldSession or deleting AcDream.Core.Net shifts a namespace many files reference.	High	Low (compile errors are immediate)	M.8 deletion is the last commit; entire branch compiles up to that point; deletion + namespace fix lands in one commit, single rebuild.
8	Threading model regression — if M.7 introduces a network thread, render-thread races appear.	Medium	High (intermittent crashes)	Default to keeping single-threaded model; threading is opt-in via a flag for one test session before becoming default.
9	Test fixture rot — golden vectors capture a 2026-05 ACE version; future ACE versions diverge.	Low	Low (fixtures still valid for retail-conformance baseline)	Golden vectors are pinned to retail behavior, not ACE-specific. Live capture replay is from acdream itself (most reproducible).
10	Calendar overrun — 6.4 weeks expands to 12+ weeks.	Medium	Medium (delays Phase F+ gameplay phases)	Mid-phase checkpoint at M.4 close (week 3 in plan). If hours-spent ≥ 1.5× estimate, scope-cut M.6 to "matrix-deferred opcodes only, batch the long tail to M.6.b post-merge."

---

8. Cost estimate

8.1 Summary

Total estimate: 256 hours ≈ 6.4 working weeks single-developer.

With effective subagent dispatch (especially on M.1 matrix population and M.6 typed-message implementation), realistic calendar compression to 4–6 weeks.

8.2 Cost breakdown by sub-phase (repeating for visibility)

Sub-phase	Hours	Calendar weeks	Subagent-friendly?
M.1 — Audit & matrix	16	0.4	Yes (per-class agents)
M.2 — Layer extraction	40	1.0	Limited (architecture-driven, single voice)
M.3 — Reliability core	40	1.0	Limited (ISAAC + ordering buffer interact)
M.4 — ACK + control	16	0.4	Limited
M.5 — Fragments	24	0.6	Limited
M.6 — Typed protocol	80	2.0	Yes (per-opcode-class agents)
M.7 — Runtime + diagnostics	16	0.4	Limited
M.8 — Tests + live val	24	0.6	Limited (live val needs human)
Total	256	6.4

8.3 Critical path

M.1 → M.2 → M.3 → M.4 → M.5 → M.6 → M.7 → M.8
        (mostly sequential within a single-developer flow)

M.1 can partially overlap M.2 (matrix work continues while skeleton lands). M.3 / M.4 / M.5 are conceptually parallel within the reliable layer, but practically sequenced because they share state. M.6 is the parallelization cliff — agents work on different opcode classes simultaneously. M.7 / M.8 are sequential.

8.4 Resource assumptions

• One primary developer driving the architecture and integration.
• Subagent dispatch budget: liberal (acdream's sustained pattern is to use Sonnet agents heavily for bounded chunks; per CLAUDE.md "Subagent policy").
• Live ACE on 127.0.0.1:9000 available throughout for smoke tests.
• User available for M.8 final visual gate (the only step that genuinely needs human eyes).

8.5 What buys schedule slack

If budget compresses (e.g., 4 weeks max), the following are scope-cuts in order:

1. Long-tail GameEvent sub-opcodes (House*, Trade*, Book*, Vendor*, Barber*, Allegiance updates, ContractTracker*) — 30+ rows that gameplay phases will need eventually but not for M.8 acceptance. Move to a M.6.b follow-up.
2. Outbound multi-fragment splitting (M.5 second half) — defer until a gameplay phase needs >448-byte outbound payload.
3. M.7 dev-panel diagnostics — keep the byte counters and capture, drop the visual overlay.
4. M.8 replay harness — keep the smoke gate, drop the automated replay testing (move to follow-up).

These cuts get total down to ~150–180 hours / 4 weeks if necessary. The architecture is preserved; the long-tail completeness regresses to "covers everything observed in live ACE during normal play, not the long tail."

---

Status & next steps

Spec status as of 2026-05-10: Sections 1–8 written. Awaiting:

1. Opcode matrix construction (M.1's main deliverable). Dispatch agents: one per opcode class. Output: docs/research/2026-05-10-phase-m-opcode-matrix.md.
2. Roadmap update. Phase M entry shrinks to a one-paragraph summary + status table + pointer to this spec. M.0 sub-lane folds into M.3 / M.4 / M.6 (no longer ships separately).

When implementation starts: create the worktree, branch off main, begin M.1 matrix completion → M.2 skeleton.