acdream/docs/plans/2026-07-29-network-transport-campaign.md

Campaign N — Retail-Faithful Network Transport

Status: ACTIVE (approved 2026-07-29). Slices N0–N6.

acdream cannot survive a single lost UDP packet in either direction. This campaign ports retail's reliable-transport mechanism (the Sept 2013 named decomp is the oracle; ACE and holtburger are cross-checks) so live-server sessions survive real packet loss. Root-cause record: docs/ISSUES.md #260.

Read this before touching src/AcDream.Core.Net/.

---

1. Why (the two fatal bugs)

1. Outbound (the #260 wedge): no sent-packet cache, no resend. ACE NAKs a client-sequence gap with RequestRetransmit; we parse the list and consume it nowhere. One lost C2S datagram → ACE buffers everything after the gap forever → all actions void, position updates void → ACE stops streaming new areas (the #256 "invisible portals") while the session stays alive on cleartext acks. Reproduced twice on Coldeve 2026-07-29 with the ACDREAM_PROBE_NET probe (artifacts/coldeve-probe-20260729/, local).
2. Inbound (found at design time): PacketCodec.TryDecodeBorrowed consumes the inbound ISAAC word BEFORE comparing. One lost S2C packet shifts the keystream permanently; every later encrypted packet fails checksum and burns another word. Inbound silent forever.

Loopback ACE never drops packets — every historical gate was structurally blind to both bugs. Slice N5's loss gate removes that blindness permanently.

2. The retail mechanism (port target)

Owners: inbound = ReceiverData (per-connection, in SharedNet::receivers_); outbound = RecipientData + ClientFlowQueue + SentPacketStore (under PacketController). Both pumped once per frame from Client::UseTime @ 0x00411C40 (receive: SharedNet::UseTime @ 0x00542450; send: PacketController::UseTime @ 0x005410D0).

2.1 Outbound resend

• Every reliable packet (fragments aboard) is cached AFTER a successful send (FlowQueue::TransmitNewPackets @ 0x00547A60 → SentPacketStore:: AddSentPacket @ 0x0054AB00), with disposable optional headers stripped first (NetPacket::RemoveDisposableOptionalHeaders @ 0x00549510). Under our standalone-control design (§4) reliable packets never carry optional headers, so the strip is a provable no-op — assert optionalLength == 0 in SentPacketStore.Add.
• Server RequestRetransmit (0x1000) → merge-insert ids wrap-safe sorted (FlowQueue::EnqueueAcks @ 0x005488E0); ids[0] doubles as an implicit cumulative ack (RecipientData::ProcessNaks @ 0x00547010). Ids no longer cached → retail answers RejectRetransmit; we drop silently (row TS-57).
• Resend (FlowQueue::TransmitAcks @ 0x005485B0 / DequeueAck @ 0x005472F0): re-emit with a REBUILT 20-byte header — flags Retransmission|EncryptedChecksum (=3; |BlobFragments =7 with fragments), Time = the CURRENT interval id, Sequence/DataSize verbatim, checksum = fresh header hash + stored sealed checksum. Body bytes untouched; the original ISAAC key is reused (CryptoSystem:: EncryptData @ 0x0065FF40 takes the key as an optional in/out — null draws, non-null reuses). Never a new keystream word.
• AckSequence (0x4000) inbound → wrap-safe max watermark → prune strictly older (SentPacketStore::Flush @ 0x0054ACD0, pops from FIFO head while seqNum < watermark). No timer-based resend exists — resend only on explicit NAK.
• Sequence allocation: highestIDSent_ starts 1; ++; wrap 0xFFFFFFFF → 1 (never 0). Unsequenced packets reuse the current watermark, cleartext, no ISAAC word.
• Pump order per frame (FlowQueue::Empty @ 0x00548A20): NAKs → retransmits → new packets.

2.2 Inbound sequencing + ISAAC discipline

No transport reorder buffer — packets process on arrival (SharedNet::ProcessPacket @ 0x00544790 → ProcessNewSeqNum @ 0x00544690):

1. SeqIDSanityCheck @ 0x00543A20: drop if seq is newer than highestIDReceived_ + 0x7FFF (wrap-safe).
2. Encrypted + not newer than the watermark = duplicate/late arrival: remove from the NAK set — hit → decrypt with the parked pre-drawn key; miss → drop silently at zero keystream cost.
3. Newer than the watermark → gap walk (ProcessNewestSeqNum @ 0x00541930): for each missing id (skip 0) AddNakked(id, null) — pre-draws one inbound ISAAC word per missing id, in sequence order, BEFORE the arriving packet's own key, parking it beside the id (ReceiverData::AddNakked @ 0x00549240, idempotent). Cleartext packets walk to seq+1 (they borrow an already-delivered sequence). Then watermark = seq.
4. Encrypted packets then decrypt with the parked key or the next drawn word.
5. Parse failure on a sequenced packet → re-park its consumed key (AddNakked(seq, &key)), so the retransmission decodes.
6. Inbound RejectRetransmit (SharedNet::HandleEmptyAck @ 0x005448F0) → remove ids from the NAK set, silent abandonment.

This keeps the inbound keystream aligned to SEQUENCE order, not arrival order — the invariant the whole inbound port hangs on.

2.3 Ack/NAK sweep

Per-frame per-connection sweep (ClientNet::ProcessConnection @ 0x00545450), mutually exclusive on ONE shared timestamp (ReceiverData::timeStamp_):

• NAK set non-empty → RequestRetransmit: ≤114 ids ascending, cleartext, 0.6 s gate (SharedNet::EnqueueNaks @ 0x00543BD0, ReceiverData::GetNaks @ 0x005490C0), and NO ack this sweep.
• Else → one cumulative AckSequence carrying highestIDReceived_, unsequenced + cleartext, 2.0 s gate (SharedNet::EnqueuePak @ 0x00543B10 — the ONLY 0x4000 construction site in the binary; retail never acks per-packet).

2.4 Keepalive/time/flow (mostly deferred — §5)

TimeSync (8-byte double) + EchoRequest (4-byte float) every ~3 s (ClientFlowQueue::IncrementLocalInterval @ 0x00547F10); inbound TimeSync adopts the server clock; Flow report sent on remote-interval advance; inbound Flow ignored, no throttle (WireRoomLeft is a folded return 1 — verified); dead link at 140 s without inbound data (self-stall guarded) with referral auto-reconnect; 0.5 s interval clock feeds ProtoHeader::interval_ (our PacketHeader.Time).

2.5 Constants

464 max payload after the 20-byte header · 448 fragment payload · 114 NAK list cap · +0x7FFF sanity window · 0.5 s interval tick · 2.0 s ack gate · 0.6 s NAK gate · ~3 s TimeSync/Echo · 140 s dead link · 10 s disconnect drain · 0xBADD70DD checksum placeholder · sequence init 1, wrap 0xFFFFFFFF→1 · 0.333 s ConnectResponse handshake resend.

3. ACE constraints (Coldeve runs ACE; references in the MAIN repo checkout)

Constraint	Source	Consequence
NAKs client gaps at expected+2, 1 s limit, arrival-driven ONLY	NetworkSession.cs:351-363	after one C2S loss two more sends must arrive before ACE NAKs; a quiet client is never NAKed
Crypto search window 256 keys; re-key orphans permanently	CryptoSystem.cs:30-49	NEVER re-key a resend; NEVER resend an already-accepted packet
Watermark advances on ANY packet with flags ≠ exactly AckSequence	NetworkSession.cs:474-476	standalone unsequenced control packets other than exact-AckSequence/exact-RequestRetransmit can skip a REAL packet forever (the self-induced wedge)
Honours only CLEARTEXT NAKs; encrypted NAKs silently ignored	NetworkSession.cs:283-284	no NAK piggybacking; NAKs never refresh ACE's 60 s timeout
Ack-only dedup exemption requires flags == AckSequence exactly	NetworkSession.cs:342-343	equality check, not HasFlag
ACE never proactively resends S2C; S2C cache prunes at 120 s	NetworkSession.cs:675-708, :251-262	the client MUST NAK or inbound stalls; old NAKs get RejectRetransmit
C2S fragment sequence strictly contiguous or dispatch stalls silently	NetworkSession.cs:532-543	packet-level retransmission heals it automatically
ACE sends TimeSync/20 s, cumulative ack/2 s, EchoResponse on request; NO disconnect/error packets ever	NetworkSession.cs:207-216, grep	every transport death is silence
Rejects AckSequence/TimeSync/EchoRequest/Flow during AuthLoginRequest	Session.cs:101-102	no sweep before negotiation completes
Inbound Header.Time ignored	grep	our interval field is cosmetic against ACE — safe

4. Design

src/AcDream.Core.Net/Transport/ — retail's split adapted to one login + one world connection on one frame thread. TransportClock (injectable monotonic source + 0.5 s interval counter), SequenceMath (IsNewer(a,b) => unchecked((int)(a-b)) > 0), SentPacketStore (FIFO of rented wire buffers), OutboundFlowQueue (outbound ISAAC, HighestIdSent, fragment seq, pending resends, NAK/ack consumption), InboundSequenceTracker (inbound ISAAC, HighestIdReceived, NAK set SortedDictionary<uint,uint> seq→parked key), AckNakScheduler (ONE shared timestamp; per sweep NAK xor ack), ReliableTransport (composition + Sweep()), TransportStats (unconditional counters; printing probe-gated).

Cache entry: { Sequence, rented Buffer (header+body), BodyLength, SealedChecksum (= payloadHash ^ isaacKey — retail NetPacket::checksum_), IsaacKey, HasFragments }. Resend rewrites ONLY the header per §2.1; PacketCodec.FinalizeInPlace gains an overload returning (isaacKeyUsed, sealedChecksum); the old signature forwards.

Inbound: codec splits pure-parse (TryParseBorrowed — no keystream) from VerifyChecksum(header, headerHash, payloadHash, uint? isaacKey); the tracker owns every key decision (retail's own factoring — EncryptData's optional key parameter). TryDecodeBorrowed(datagram, IsaacRandom?) is deleted; owned TryDecode stays (test-only).

WorldSession keeps its entire ~60-method public surface; the two SendGameMessage overloads delegate to the transport; GameActionCapture is untouched. Sweep() runs at the end of Tick() (after the budget break) AND inside the blocking handshake pump loops (the EnterWorld flood arrives before Tick ever runs), gated on _transportNegotiated. Cache is unbounded like retail (ACE acks every 2 s; steady state is tens of entries); cache=N in [net-tick] is the watchdog, not a silent cap.

5. Scope deferrals (divergence-register rows, filed in-slice)

• TS-57 no outbound RejectRetransmit — ACE no-ops it; the standalone unsequenced form trips the watermark hole (§3 row 3).
• AP-125 standalone control packets only, no CoalesceData piggyback — piggybacked NAKs become encrypted, which ACE ignores.
• TS-58 no TimeSync/EchoRequest keepalive — standalone-unsafe against the watermark hole; our 2 s ack already refreshes ACE's 60 s timeout.
• TS-59 no Flow report — ACE parses and has no handler.
• AD-49 blob-layer ephemeral ordering stamps not ported — provably a no-op against ACE (fragment Id is constant 0x80000000 and the stamp table keys on ACE's per-message-unique fragment sequence, so FragIsObsoleteEmphemeral @ 0x0054A450 can never fire).
• TS-60 no 140 s dead-link/referral auto-reconnect — LinkStatus exposes the input; reconnect is Runtime's, its own campaign.
• TS-61 UDP send-failure burns the sequence+key (retail retries from the queue head) — effectively unreachable.
• AP-126 one monotonic clock for all gates (retail's cur/local split immaterial to the gates we port).
• Multi-fragment outbound (>448 B) stays unimplemented (nothing sends >448; existing behavior, keep its row current).

6. Slices

#	Slice	Size	Parallel?	Review	Gate
N0	ACE-behaviour test double + virtual clock + lossy link	M	yes (test-only)	Opus	unit
N1	Outbound cache + resend on NAK (the #260 fix)	L	no	Fable	unit + local ACE lifecycle
N2	Inbound sequence-aligned ISAAC + NAK set	L	no	Fable	unit + lifecycle + nine-stop
N3	AckNakScheduler + 2.0 s cumulative ack	M	no	Opus	unit + lifecycle + nine-stop
N4	Client NAK emission + RejectRetransmit consumption	M	no	Opus	unit + local ACE + loss gate
N5	Observability + LossyTransportDecorator + connected loss gate	M	partly	Opus	unit + loss gate
N6	Optional: ConnectResponse 0.333 s retransmit; assembler TTL	S	yes	Opus	unit + local ACE

N1–N5 all touch WorldSession.cs: strictly sequential, ONE agent at a time. Implementers Sonnet; a redo escalates to Opus; every diff reviewed by the Review-column model before commit. Key per-slice test specs are in the approved plan (~/.claude/plans/mac-os-is-not-robust-sphinx.md) and travel verbatim in each implementer prompt.

7. Landmines (verbatim in every implementer prompt)

1. Resends are NOT byte-identical — flags gain Retransmission, Time advances, the header hash MUST be recomputed; checksum = new header hash
   • stored sealed checksum. A verbatim resend fails ACE's CRC silently.
2. Never draw a new ISAAC word on a resend — one re-key permanently orphans a key in ACE's 256-entry window.
3. Never resend unrequested — a duplicate of an accepted packet burns up to the whole window.
4. The inbound gap-walk draws parked keys BEFORE the arriving packet's own key, in sequence order — reversed, the stream is off by the gap size forever.
5. Ack flags are an EQUALITY check (Flags == AckSequence), never HasFlag.
6. NAKs are cleartext (Flags == RequestRetransmit exactly) or ACE ignores them; NAKs do not refresh ACE's timeout.
7. The 0.6 s NAK gate and 2.0 s ack gate share ONE timestamp — a NAK delays the next ack and vice versa; never both in one sweep.
8. The sweep must also run inside the blocking handshake pump loops — the EnterWorld CreateObject flood precedes the first Tick().
9. ACE/holtburger reference sources live in the MAIN repo checkout (C:\Users\erikn\source\repos\acdream\references\), not in worktrees.

8. Verification ladder

1. Per slice: dotnet build + full dotnet test -c Release + slice gate + one commit (retail symbol+address citations) + model review + revert SHA recorded here.
2. N1–N5: tools/run-connected-world-lifecycle-gate.ps1 vs local ACE; N2/N3 add the canonical nine-stop route.
3. From N4: tools/run-connected-loss-gate.ps1 at ACDREAM_NET_DROP_PCT=2 vs local ACE — passes only with NON-ZERO resend/NAK counters.
4. Final: one user Coldeve session past the 15-minute wedge horizon with portal churn, after N5. [net-tick] nak-in/s becomes the first direct measure of real C2S loss the project has had.
5. Rollback: one commit per slice, git revert; no runtime kill switch.

9. Slice ledger

Slice	Status	Commit	Notes
N0	complete	7e9134b4 + e3958610	ACE-behaviour double + virtual clock + lossy link; the review fix-up added the Session.CheckState inbound gate, faithful SendBundle coalescing/splitting, two-phase termination, an ACE-loose C2S fragment parse, and ACE's MessageBuffer edge cases. 687 Core.Net tests green. N1 folded in the re-review's ProcessFragment two-branch split (existing-buffer checks Complete; new-buffer parks without checking — the zero-count buffer stays parked).
N1	complete	43e60a69	Outbound sent-packet cache + resend on NAK (Transport/: TransportClock, SequenceMath, SentPacketStore, OutboundFlowQueue, ReliableTransport, TransportStats); PacketCodec.FinalizeInPlace sealed-checksum overload; WorldSession sweep in Tick + both handshake pump loops; TS-57 filed, TS-27 narrowed to inbound-only. Fable review PASS. Advisories: fresh sends keep Time=0 (byte-identical wire; retail stamps the interval on every packet and ACE ignores the field — N3 folds the retail stamp in with the cadence work); the stale-NAK single-redundant-resend window is shared with retail (pending prunes at flush, after transmit — same frame order as RecipientData::UseTime).
N2	complete	46d209d0 — Fable review PASS (tracker/codec-split/admission verified pre-commit against retail rules; gates lifecycle+nine-stop both PASS)	Inbound sequence-aligned ISAAC + NAK set (Transport/InboundSequenceTracker: watermark, sanity window, duplicate/parked-key path, sequence-ordered gap-walk pre-draw, verify-failure re-park, RejectRetransmit abandonment — ProcessNewSeqNum @ 0x00544690, ProcessNewestSeqNum @ 0x00541930, SeqIDSanityCheck @ 0x00543A20, AddNakked @ 0x00549240, HandleEmptyAck @ 0x005448F0); PacketCodec split into keystream-free TryParseBorrowed + VerifyChecksum (TryDecodeBorrowed deleted); RejectRetransmit ids exposed on both optional-header decoders; ReliableTransport now owns both keystreams; stats gained InboundDupsDropped/InboundSanityDrops/ChecksumFailures/KeysParked. Watermark init 1 is the AD-50 ACE adaptation (retail zero-init vs ACE's re-prime dance — first encrypted S2C is sequence 2; holtburger api.rs:30 agrees); pinned by the clean-lifecycle conformance test (zero NAKs, min encrypted S2C sequence == 2). 716 Core.Net tests green. N3/N4 handoff note: the interim per-packet reflex ack acks the ARRIVING sequence even while a gap is parked, so ACE prunes the lost id from its S2C cache (AcknowledgeSequence strictly-below) before N4 can NAK it — message-level recovery of a real loss needs N3's retail NAK-xor-ack sweep (§2.3's mutual exclusivity is load-bearing). Also noted for N4: ACE's RejectRetransmit consumes a fresh CLEARTEXT sequence via FlushPackets (no keystream word), so the client's gap walk parks a word for an id that never had one server-side — a real retail-vs-ACE incompatibility to resolve in N4's design (retail never assigns new sequences to cleartext).
N3	complete	0265cc42 — Opus review PASS (advisories folded into N4: transitional-state wording, SharedInit citation, pump-order wording, control-packet Time/Iteration rule, stale budget-break comment)	AckNakScheduler + 2.0 s cumulative ack (Transport/AckNakScheduler): ONE shared timestamp (ReceiverData::timeStamp_ @ +0x10) arbitrating NAK-xor-ack per sweep (ClientNet::ProcessConnection @ 0x00545450); the ack is one cleartext exact-flags AckSequence carrying the tracker's highestIDReceived_ behind the >= 2.0 s gate (SharedNet::EnqueuePak @ 0x00543B10 — the binary's only 0x4000 construction site), armed at connection birth (ReceiverData::Init @ 0x00548EF0). The Phase 4.9 per-packet reflex ack and WorldSession.SendAck are DELETED; the [net-tick] acks/s probe now reads Stats.AcksSent. Sweep order per FlowQueue::Empty @ 0x00548A20: interval clock, NAK/ack arbitration, pending resends, prune. N3 transitional state (closed by N4): a non-empty NAK set suppressed the ack and emitted NOTHING — the exposure was real even on loopback, just low-probability: one receive-buffer drop parks an id, every later sweep takes the silent NAK branch, acks stop (witness: [net-tick] acks/s=0), and ACE disconnects the quiet session at its 60 s timeout. N4 completed the branch. N1 advisory retired (fold-in): fresh reliable sends now stamp Header.Time = the current interval id (FlowQueue::TransmitNewPackets @ 0x00547A60, header build at 0x00547A84); ACE ignores inbound Header.Time, so the wire is unaffected. New WorldSession.TransportClockSource seam drives the gate on virtual time. 723 Core.Net tests green (keepalive property proven: a quiet session's 2 s acks refresh ACE's 60 s deadline across a 120 s virtual horizon; storm collapse: a 50-packet flood → ONE ack; the model accepts the reused-sequence ack without advancing its watermark). Connected lifecycle + canonical nine-stop gates PASS.
N4	complete	852a59e3 — Opus review PASS (reclaim invariant attacked from five angles, held; NAK fidelity verified to the x87 masks; AP-125 filed + F1 false-arithmetic wording + F5 ordinal sentinel fixed in the acceptance commit; F3 Iteration-on-fresh-sends folds into N5)	Client NAK emission + RejectRetransmit consumption. AckNakScheduler completes the NAK branch: one cleartext exact-flags RequestRetransmit per sweep behind the STRICT 0.6 s gate on the ONE shared timestamp (SharedNet::EnqueueNaks @ 0x00543BD0 — the 0x41-mask x87 test at 0x00543C03 proceeds only on strictly-greater, contrast the ack's >=), body u32 count + ids ascending capped at 114 (ReceiverData::GetNaks @ 0x005490C0, cap 0x72), borrowed sequence, never an ack in a NAK sweep. Control-header rule decided for BOTH emissions: Time = interval id, Iteration = session iteration, per the shared retail header build (FlowQueue::TransmitNewPackets @ 0x00547A60 @ 0x00547A84); ACE reads neither. THE design piece: the AD-51 reclaimed-word pool in InboundSequenceTracker closes the ACE cleartext-reject keystream hazard the N2 ledger row recorded — ACE's RejectRetransmit consumes a fresh cleartext sequence with NO keystream word, so the gap walk mis-parks a word for it and the whole inbound stream runs one word ahead. On a VALIDATED cleartext reject (WorldSession calls OnCleartextRejectSequence post-checksum), the tracker removes the mis-park, bubble-shifts every later-drawn parked word down one position (per-word draw ordinals; ascending id ⇔ ascending draw order), and pools the excess for the next fresh draws, consumed lowest-draw-order-first — exact for any number of interleaved rejects in any arrival order (a plain FIFO is NOT: reject-after-higher-arrival crosses the parked chain, and dual out-of-order rejects pool out of draw order — both pinned by tests). Reject BODY ids keep N2's discard (word-bearing server-side, consumed-in-place). N3 advisories all folded: honest transitional wording (above), ReceiverData::SharedInit @ 0x00548EF0 (from Init @ 0x00548FA0) citation, FlowQueue::Empty pump-order comment (TransmitNaks → TransmitAcks → TransmitNewPackets, interval increment LAST @ 0x00548A9D; our clock-first order is cosmetic vs ACE), the Time/Iteration rule, and the stale WorldSession budget-break comment. Gate arithmetic hardened: gate ticks now round (0.6 has no exact double; truncation opened the strict gate AT the boundary). 737 Core.Net tests green, including: strict-gate boundary, shared-timestamp both directions, NAK-suppresses-ack, full wire-shape + 114-cap pins, model-served retransmission round trip, five tracker reclaim proofs, the 130-s virtual prune → fresh-sequence reject system test (victim abandoned, later traffic decodes, pool drains to zero), 10 s long-loss (NAKs on the gate cadence, zero acks, heal inside the window), and the capstone soak: 2% seeded bidirectional loss × 10,000 messages → zero message loss both ways, ACE crypto headroom 256 at convergence with a ≥250 no-erosion floor mid-flight, NAK set / reclaim pool / pending resends / ACE out-of-order buffer all zero, cache at the single watermark entry (retail Flush prunes STRICTLY below the ack). Soak notes: ACE never NAKs a quiet client (§3 row 1), so convergence keeps a C2S trickle flowing — a real idle-client tail loss heals only on the next action, an ACE constraint outside N4's scope.
N5	pending	—
N6	pending	—