From 46d209d05333544d7f0d472014cbd937aa856147 Mon Sep 17 00:00:00 2001 From: Erik Date: Wed, 29 Jul 2026 13:10:20 +0200 Subject: [PATCH] feat(net): N2 - inbound sequence-aligned ISAAC + NAK set Campaign N Slice N2 (docs/plans/2026-07-29-network-transport-campaign.md S2.2) - the second fatal #260 fix: the inbound keystream now aligns to SEQUENCE order instead of arrival order. One lost S2C datagram no longer desyncs the inbound cipher permanently - the missing id's pre-drawn key parks in the NAK set, later packets keep decoding, and the retransmission decodes with the parked key. New src/AcDream.Core.Net/Transport/InboundSequenceTracker.cs - retail's ReceiverData inbound half, ported rule for rule: - Sanity window: drop when seq is wrap-safe newer than highestIDReceived_ + 0x7FFF (SharedNet::SeqIDSanityCheck @ 0x00543A20; the boundary itself is accepted). - Duplicate/late arrival (encrypted, at/below the watermark): NAK-set hit -> decrypt with the PARKED pre-drawn key; miss -> silent drop at ZERO keystream cost (SharedNet::ProcessNewSeqNum @ 0x00544690, the AVL::Remove branch) - the dup-word-burn and double-dispatch bugs close together. - Gap walk (SharedNet::ProcessNewestSeqNum @ 0x00541930): one inbound ISAAC word per missing id, drawn IN SEQUENCE ORDER BEFORE the arriving packet's own key (landmine #4), parked beside the id (ReceiverData::AddNakked @ 0x00549240, idempotent; id 0 skipped per retail's `if (esi_1 != 0)`). Cleartext walks to seq+1 - the borrowed id itself gets NAKed, so the real encrypted packet at that id can still decode later. - Verify-failure re-park: a sequenced encrypted checksum failure parks the consumed key back beside its id so the retransmission decodes (SharedNet::ProcessPacket @ 0x00544790 tail, AddNakked(seq, &key)). - Inbound RejectRetransmit -> silent NAK-set abandonment; parked keys discarded, alignment holds because the words were already drawn (SharedNet::HandleEmptyAck @ 0x005448F0). - NAK set = SortedDictionary seq -> parked key; ascending raw-uint enumeration matches retail's AVL walk for N4's <=114-id NAK emission (ReceiverData::GetNaks @ 0x005490C0). PacketCodec split (campaign S4, retail's own factoring - the key is an optional in/out of ReceiverData::Decrypt): TryParseBorrowed is the pure parse + checksum-summand computation with NO keystream access anywhere; VerifyChecksum(header, headerHash, payloadHash, uint? key) compares the additive cleartext form (null) or headerHash + (key ^ payloadHash). TryDecodeBorrowed(datagram, IsaacRandom?) - the consume-before-compare site that WAS the bug - is deleted; the owned TryDecode stays (test-only). RejectRetransmit ids are now exposed on both decoders (borrowed RejectRetransmitBytes/Count like the Request pair; owned RejectRetransmits list); the bytes were always inside the hashed span, so parse-hash coverage is unchanged. WorldSession: ProcessDatagram head is now parse -> sequence-0 split (cleartext seq-0 = handshake/control, verified additively and processed as before; encrypted seq-0 dropped before any keystream access, like retail's ProcessPacket) -> tracker.Admit -> VerifyChecksum with the admission key -> failure re-park -> unchanged flag handling, N1 transport consumption, reflex ack, and fragment loop. The RejectRetransmit flag routes to the tracker beside the N1 NAK/ack consumption. The handshake Connect loop moved to parse + cleartext-verify (no tracker exists before ISAAC seeding; the ConnectRequest is cleartext seq 0). ReliableTransport now takes both Isaacs and exposes Inbound; the session's _inboundIsaac field is deleted. No production caller constructed the N1 ctor outside WorldSession, so no compatibility shape was kept. TransportStats gains InboundDupsDropped, InboundSanityDrops, ChecksumFailures, KeysParked (unconditional, like the N1 counters). Watermark init = 1 is an ACE adaptation, register row AD-50 (watermark INIT only, not a mechanism change; AD-49 stays reserved for the campaign S5 blob-layer deferral): retail zero-inits ReceiverData, but ACE never emits S2C sequence 1 - PacketSequence starts unprimed at uint.MaxValue, the cleartext ConnectRequest takes NextValue 0, and the first ENCRYPTED flush re-primes CurrentValue to 1 so the first encrypted sequenced packet is 2 (ACE NetworkSession.cs:716-717 resolving to UIntSequence(startingValue: 1), Sequence/UIntSequence.cs:9-13,30-41). A zero-init watermark would gap-walk the permanent id-1 hole: one spurious NAK, the first pre-drawn word mis-assigned to id 1, and the keystream off by one from the first encrypted packet onward. holtburger seeds the same value (crates/holtburger-session/src/session/api.rs:30, last_server_seq: 1), mirroring ACE's own C2S-side lastReceivedPacketSequence = 1 (NetworkSession.cs:57). The N0 model's dance is pinned by the clean-lifecycle conformance test: min encrypted S2C sequence == 2, zero NAKs, zero spurious drops. Tests (+14; Core.Net 702 -> 716): the decisive gap test (10,11,13,14 - 13 and 14 decode with fresh words while 12's key parks with KeysParked=1/NakCount=1, the late 12 decodes with the parked key, 15 takes the next fresh word - impossible pre-N2), zero-cost duplicate drop (shadow ISAAC position unchanged), re-park -> byte-identical retransmission decode, the cleartext borrowed-id rule, cleartext at the watermark (no NAK/key/watermark change), sanity boundary +0x7FFF accepted / +0x8000 dropped wrap-safe, skip-id-0 across the 32-bit wrap with ascending NAK enumeration, RejectRetransmit abandonment with alignment held, warm zero-alloc Admit; plus four real-WorldSession conformance runs against the N0 ACE double: clean lifecycle (zero NAKs at every stage), S2C loss of one packet of a Count=2 fragment set (later packets STILL decode - the N2 win; late byte-identical redelivery completes the split message intact), duplicate delivery dropped BEFORE dispatch, and the seq-0 tracker bypass. N3/N4 handoff notes are recorded in the campaign S9 N2 row: the interim per-packet reflex ack acks the arriving sequence even while a gap is parked (ACE prunes the lost id from its S2C cache before N4 could NAK it - message recovery needs N3's retail NAK-xor-ack sweep), and ACE's RejectRetransmit consumes a fresh CLEARTEXT sequence with no keystream word, an ACE-vs-retail wrinkle N4's design must resolve. Gates: dotnet build green; AcDream.Core.Net.Tests 716/716; full-solution Release 9,732 passed / 5 skipped / 0 failed; connected world-lifecycle gate vs local ACE RESULT=PASS (zero failures, one pre-existing expected world-edge landblock-miss warning); canonical nine-stop connected route RESULT=PASS. Co-Authored-By: Claude Opus 4.8 --- .../retail-divergence-register.md | 3 +- .../2026-07-29-network-transport-campaign.md | 2 +- .../Packets/BorrowedPacket.cs | 11 +- src/AcDream.Core.Net/Packets/PacketCodec.cs | 144 ++-- .../Packets/PacketHeaderOptional.cs | 12 +- .../Transport/InboundSequenceTracker.cs | 257 +++++++ .../Transport/ReliableTransport.cs | 13 +- .../Transport/TransportStats.cs | 20 + src/AcDream.Core.Net/WorldSession.cs | 164 ++++- .../Packets/BorrowedPacketCodecTests.cs | 233 ++++-- .../Transport/FakeAceTransport.cs | 16 + .../Transport/InboundSequenceTrackerTests.cs | 664 ++++++++++++++++++ 12 files changed, 1359 insertions(+), 180 deletions(-) create mode 100644 src/AcDream.Core.Net/Transport/InboundSequenceTracker.cs create mode 100644 tests/AcDream.Core.Net.Tests/Transport/InboundSequenceTrackerTests.cs diff --git a/docs/architecture/retail-divergence-register.md b/docs/architecture/retail-divergence-register.md index fed01aed..ae536563 100644 --- a/docs/architecture/retail-divergence-register.md +++ b/docs/architecture/retail-divergence-register.md @@ -62,13 +62,14 @@ accepted-divergence entries (#96, #49, #50). --- -## 2. Adaptation (AD) — 41 rows (AD-47 and AD-48 filed 2026-07-29 at Campaign V slice V11 — the MSAA sample-position and present-pacing rows the campaign's risk register scheduled for the GL deletion; AD-11 retired 2026-07-23 — exact low-bit ItemUses predicate; AD-31 retired 2026-07-15 — the DAT-authored portal-space viewport replaces the black transit cover) +## 2. Adaptation (AD) — 42 rows (AD-50 filed 2026-07-29 at Campaign N slice N2 — the inbound-watermark ACE init; AD-49 stays reserved for Campaign N §5's blob-layer ordering deferral, filed when its slice lands; AD-47 and AD-48 filed 2026-07-29 at Campaign V slice V11 — the MSAA sample-position and present-pacing rows the campaign's risk register scheduled for the GL deletion; AD-11 retired 2026-07-23 — exact low-bit ItemUses predicate; AD-31 retired 2026-07-15 — the DAT-authored portal-space viewport replaces the black transit cover) | # | Divergence | Where (file:line) | Why it is safe / justified | Risk if assumption breaks | Retail oracle | |---|---|---|---|---|---| | AD-46 | **LIVE. Reframed at Campaign V slice V11 (2026-07-29), when GL was deleted and the comparison that discovered this row ceased to exist.** Dense alpha-blended distant scenery (the treeline) may read slightly denser than retail's, because the anisotropic TAP PATTERN is implementation-defined and acdream's Vulkan driver does not tap identically to retail's D3D9 one. Both request the same sampler state — trilinear, clamp-and-repeat, the device's maximum anisotropy. **What changed at V11 is only the left-hand side of the comparison**: this was measured GL-vs-Vulkan (~15% of the pixels in the band), and it is now a Vulkan-vs-retail question against the D3D oracle in the last column. The measurement below is retained as the evidence that the residual is a tap pattern and not a bug, even though one of its two arms no longer exists. | `src/AcDream.App/Rendering/Wb/WorldTextureArray.cs` (`RhiWorldTextureArray.WorldArrayAnisotropy`); measured in plan §5.5.19, reframed §5.5.24 | Not assumed — narrowed by measurement while both backends still existed, on an offline capture with no session, no entities and both clocks pinned. Anisotropy 1 → 41,509 differing pixels in the tree band; anisotropy 16 (GL's value, and retail's `m_D3DCaps.MaxAnisotropy`) → 22,266, and the rest of the frame fell to 497 px of 563,200, i.e. 8.8e-04, inside the campaign's 0.001 threshold. The residual was not a sub-pixel shift (an integer shift search found none), not a sharpness change (high-frequency energy matched within 5%), and not depth precision (forcing Vulkan's window-depth range to GL's compressed [0.5, 1] moved it by 3%). Monotone improvement toward GL's own anisotropy with no knob left is what made it a driver property rather than a bug. | Distant foliage shimmers or reads denser than retail's. The class is confined to alpha-blended dense overlap: opaque terrain, roofs, walls, water, statics, the character and the whole retained UI are inside threshold. **Now unfalsifiable by self-differential** — with GL gone, the only way to retire this row is a side-by-side against the retail client, not against another acdream backend. | `RenderDeviceD3D::SetDefaultD3DStates @ 0x005a3800`, whose `SetSamplerState(stage, 0xA /* D3DSAMP_MAXANISOTROPY */, m_D3DCaps.MaxAnisotropy)` at `0x005a4230` is the value acdream requests | | AD-47 | **Filed at Campaign V slice V11 (2026-07-29); the campaign's risk register scheduled this row here.** Multisample resolve sample POSITIONS are unspecified by both the Vulkan and D3D9 specifications, so acdream's MSAA-on silhouette edges do not match retail's pixel-for-pixel even at the same sample count. acdream's strict pixel gates therefore run with MSAA forced OFF on every arm, and MSAA-on gets only a relaxed visual smoke. | `src/AcDream.App/RuntimeOptions.cs` (`ACDREAM_MSAA_SAMPLES`); forced to 0 in `tools/run-offline-pixel-gate.ps1` | Measured, not assumed: plan §5.5.16 compared two backends at 4x and found **8.83% of the frame differing — 81,359 px of 921,600 — essentially all of it hugging foliage and silhouette edges**, which is ninety-fold over the 0.001 gate threshold. That is two implementations' sample patterns, not a renderer divergence, which is why forcing MSAA off is what makes the remaining difference attributable rather than a threshold relaxation. | Edge quality on thin geometry (fence rails, foliage, distant railings) differs from retail at the sub-pixel level whenever MSAA is on, which is the ordinary player configuration. Because the gates run MSAA off, **a real regression confined to the multisample path would not be caught by them** — that is the actual exposure this row records. | D3D9 `D3DRS_MULTISAMPLEANTIALIAS` / `D3DMULTISAMPLE_TYPE` as set by `RenderDeviceD3D::SetDefaultD3DStates @ 0x005a3800`; retail's sample pattern is the driver's, exactly as ours is | | AD-48 | **Filed at Campaign V slice V11 (2026-07-29).** Presentation is paced by the Vulkan swapchain present mode (FIFO, i.e. VSync) or by a refresh-rate software pacer when uncapped, rather than by retail's D3D9 `Present` with its own frame-rate limiter. Frame delivery cadence, and therefore input-to-photon latency, is a property of our present path rather than a port of retail's. | `src/AcDream.App/RuntimeOptions.cs:98-100`; `src/AcDream.App/Rendering/Gpu/Vk/VulkanSwapchain.cs` | Retail's limiter and ours both bound the frame rate to the display; the simulation is fixed-step and clock-driven, so gameplay timing does not ride on presentation cadence. The uncapped path exists for measurement and is not the shipping default. | A pacing mismatch shows up as judder or input latency that differs from retail's feel without any visual difference in a captured frame — invisible to every pixel gate by construction. Issue **#235** (the capped/RDP jump-presentation cadence alias) is the known live instance of this class. | D3D9 `IDirect3DDevice9::Present`; retail's frame limiter in `RenderDeviceD3D` | +| AD-50 | **Filed at Campaign N slice N2 (2026-07-29).** The inbound sequence tracker's watermark (`highestIDReceived_`) initializes to **1**, not retail's zero-init of `ReceiverData`. Watermark INIT only — every mechanism (sanity window, duplicate/parked-key path, gap walk, re-park, RejectRetransmit abandonment) is the verbatim retail port. | `src/AcDream.Core.Net/Transport/InboundSequenceTracker.cs` (`AceInitialWatermark`) | ACE never emits S2C sequence 1: its `PacketSequence` starts unprimed at `uint.MaxValue`, the cleartext ConnectRequest takes NextValue 0, and the first ENCRYPTED flush re-primes CurrentValue to 1 so the first encrypted sequenced packet is 2 (ACE NetworkSession.cs:716-717 + Sequence/UIntSequence.cs:9-13,30-41; pinned by the N0 double and the N2 clean-lifecycle conformance test asserting min encrypted S2C sequence == 2 with zero NAKs). A zero-init watermark would gap-walk the permanent id-1 hole: one spurious NAK, the first pre-drawn word mis-assigned to id 1, and the keystream off by one from the very first encrypted packet. holtburger seeds the same value (crates/holtburger-session/src/session/api.rs:30, `last_server_seq: 1`), mirroring ACE's own C2S-side `lastReceivedPacketSequence = 1` (NetworkSession.cs:57). | Against a hypothetical server that DOES emit sequence 1 as its first encrypted packet (retail's own numbering), init-1 would classify it "not newer" and drop it as a duplicate — the mirror-image wedge. Only ACE-family servers exist for this client today. | `ReceiverData` zero-init (construction inside `SharedNet`; `highestIDReceived_` starts 0); `SharedNet::ProcessNewestSeqNum @ 0x00541930` (the walk that would mis-NAK id 1) | | AD-38 | Outgoing teleport viewports retire when retail's quantized animation level exceeds the last captured visible level 1022 (index 96), suppressing levels 1023/1024 up to 20.2 ms before retail's literal `elapsed >= 1.0` state edge. Incoming fades retain the exact timer. | `src/AcDream.Core/World/TeleportAnimSequencer.cs` (`OutgoingViewportReachedTerminalProjection`) | An uncapped 2000 FPS pass can publish the finite tunnel at levels 1023/1024 even though the paired 2013 retail capture switches viewports after 1022. The table-level cutover preserves the captured visible viewport ordering without throttling the application. | Exit sound, viewport replacement, and logout tunnel entry can occur at most two easing-table quanta (about 20.2 ms) earlier than retail's logical timer. | `UIGlobals::GetAnimLevel @ 0x004EE540`; `gmSmartBoxUI::UseTime @ 0x004D6E30`; paired retail/acdream captures documented in `docs/research/2026-07-15-retail-portal-space-pseudocode.md` | | AD-1 | Lost-cell machinery replaced by recoverable outdoor demote (**#107** safety net) + outdoor-restore `max(terrainZ, z)` under-terrain lift; retail goes `GotoLostCell` | `src/AcDream.Core/Physics/PhysicsEngine.cs:553` (+ :808) | acdream has no lost-cell state machine; outdoor landcell is the recoverable equivalent; the #107 auto-entry hold should make the demote branch unreachable | Gap in the hold → player committed to outdoor terrain inside/under a building (fake-grounded spawn, fall-through); a legit below-heightmap server restore is silently lifted — upward warp vs server | `GotoLostCell` pc:283418; `SetPositionInternal` 0x00515bd0, pc:283892-283945 | | AD-2 | Async readiness gates replace retail's synchronous destination cell load. **#229 refinement (2026-07-20):** login and F751 portal-space exit now share `WorldRevealReadinessBarrier`, so neither path can expose the normal viewport until the same render-publication, composite-texture, and collision domains converge. A hydratable indoor claim requires its owning Near-tier static/EnvCell mesh set, destination composites, and exact EnvCell physics (`IsSpawnCellReady`); an outdoor claim requires those render domains plus terrain/collision residency for the required Near ring. Hard-recenter generations and tier-aware completion application prevent stale overlapping loads/unloads or Far/Near jobs from opening or erasing the gate; mesh upload remains separate from balanced landblock ownership. Claims beyond NumCells still take the loud unhydratable-placement path. `RuntimeWorldTransitState` owns the shared reveal generation, accepted readiness, transit correlation, and exact generation/cell-scoped host-acknowledgement suffix. `WorldRevealCoordinator` is a graphical adapter holding only App resource receipts; normalized Runtime checkpoints observe ownership without defining another readiness path. **Slice E3 refinement (2026-07-24):** the same generation now publishes an immediate `WorldGenerationQuiescence` edge: old-world drawing/spatial queries, simulation/effect clocks, reconciliation, targeting, and 3-D audio stop while retained physical teardown advances through metered cursors and destination network/UI/streaming/readiness remain live. **Slice E4 refinement (2026-07-24):** accepted render/physics/static publication may span update frames through retained exact cursors, but reveal still consumes only the completed spatial/render-ready generation; building and EnvCell snapshots remain invisible until complete and the final spatial identity swap stays observer-atomic. **Slice E5 refinement (2026-07-24):** the reveal generation owns one exact destination reservation across every typed budget dimension. Stale completion cannot consume or clear its replacement, and hydratable incomplete content is never force-revealed; portal transit retains the DAT tunnel and centered retail wait cue until readiness converges. The hold→materialize→regain-control lifecycle remains owned by `TeleportAnimSequencer`. | `src/AcDream.Runtime/World/RuntimeWorldTransitState.cs`; `src/AcDream.App/Streaming/WorldRevealCoordinator.cs`; `src/AcDream.App/Streaming/WorldGenerationQuiescence.cs`; `src/AcDream.App/Streaming/WorldRevealReadinessBarrier.cs`; `src/AcDream.App/Streaming/StreamingOriginRecenterCoordinator.cs`; `src/AcDream.App/Streaming/LandblockPresentationPipeline.cs`; `src/AcDream.App/Streaming/StreamingController.cs`; `src/AcDream.App/Rendering/PortalTunnelPresentation.cs`; `src/AcDream.App/UI/PortalWaitNoticeController.cs`; `src/AcDream.App/Streaming/GpuWorldState.cs` (`IsRenderReady`); `src/AcDream.App/Rendering/Wb/LandblockSpawnAdapter.cs`; `src/AcDream.Core/Physics/PhysicsEngine.cs` (`IsSpawnCellReady`, `IsNeighborhoodTerrainResident`) | This is the asynchronous equivalent of retail leaving `SmartBox::position_update_complete` false while `CellManager::blocking_for_cells` is set: neither initial login nor portal arrival may reveal or continue simulating an old/partial collision world, a terrain-only Far shell, or a published-but-not-drawable GPU landblock. Indoor does not require a terrain heightmap, only the owning render landblock and exact EnvCell. | Gate opens early → grey/untextured first login or portal reveal, free-fall, wrong-cell rooting, missing scenery, or a still-active old generation; predicate never satisfies (streamer/DAT/upload failure) → login remains behind the world render gate, while portal transit remains in the authored tunnel and presents the centered wait cue after five seconds. | `SmartBox::UseTime` 0x00455410; `gmSmartBoxUI::UseTime` 0x004D6E30; `gmSmartBoxUI::EndTeleportAnimation` 0x004D65A0 | diff --git a/docs/plans/2026-07-29-network-transport-campaign.md b/docs/plans/2026-07-29-network-transport-campaign.md index 34abcd56..07ced8db 100644 --- a/docs/plans/2026-07-29-network-transport-campaign.md +++ b/docs/plans/2026-07-29-network-transport-campaign.md @@ -253,7 +253,7 @@ verbatim in each implementer prompt. |---|---|---|---| | N0 | complete | `7e9134b4` + `e3958610` | ACE-behaviour double + virtual clock + lossy link; the review fix-up added the `Session.CheckState` inbound gate, faithful `SendBundle` coalescing/splitting, two-phase termination, an ACE-loose C2S fragment parse, and ACE's MessageBuffer edge cases. 687 Core.Net tests green. N1 folded in the re-review's `ProcessFragment` two-branch split (existing-buffer checks Complete; new-buffer parks without checking — the zero-count buffer stays parked). | | N1 | complete | `43e60a69` | Outbound sent-packet cache + resend on NAK (`Transport/`: `TransportClock`, `SequenceMath`, `SentPacketStore`, `OutboundFlowQueue`, `ReliableTransport`, `TransportStats`); `PacketCodec.FinalizeInPlace` sealed-checksum overload; `WorldSession` sweep in Tick + both handshake pump loops; TS-57 filed, TS-27 narrowed to inbound-only. **Fable review PASS.** Advisories: fresh sends keep `Time=0` (byte-identical wire; retail stamps the interval on every packet and ACE ignores the field — N3 folds the retail stamp in with the cadence work); the stale-NAK single-redundant-resend window is shared with retail (pending prunes at flush, after transmit — same frame order as `RecipientData::UseTime`). | -| N2 | pending | — | | +| N2 | complete | SHA recorded at N3 kickoff | Inbound sequence-aligned ISAAC + NAK set (`Transport/InboundSequenceTracker`: watermark, sanity window, duplicate/parked-key path, sequence-ordered gap-walk pre-draw, verify-failure re-park, RejectRetransmit abandonment — `ProcessNewSeqNum @ 0x00544690`, `ProcessNewestSeqNum @ 0x00541930`, `SeqIDSanityCheck @ 0x00543A20`, `AddNakked @ 0x00549240`, `HandleEmptyAck @ 0x005448F0`); `PacketCodec` split into keystream-free `TryParseBorrowed` + `VerifyChecksum` (`TryDecodeBorrowed` deleted); `RejectRetransmit` ids exposed on both optional-header decoders; `ReliableTransport` now owns both keystreams; stats gained `InboundDupsDropped`/`InboundSanityDrops`/`ChecksumFailures`/`KeysParked`. Watermark init 1 is the AD-50 ACE adaptation (retail zero-init vs ACE's re-prime dance — first encrypted S2C is sequence 2; holtburger api.rs:30 agrees); pinned by the clean-lifecycle conformance test (zero NAKs, min encrypted S2C sequence == 2). 716 Core.Net tests green. **N3/N4 handoff note:** the interim per-packet reflex ack acks the ARRIVING sequence even while a gap is parked, so ACE prunes the lost id from its S2C cache (`AcknowledgeSequence` strictly-below) before N4 can NAK it — message-level recovery of a real loss needs N3's retail NAK-xor-ack sweep (§2.3's mutual exclusivity is load-bearing). Also noted for N4: ACE's `RejectRetransmit` consumes a fresh CLEARTEXT sequence via FlushPackets (no keystream word), so the client's gap walk parks a word for an id that never had one server-side — a real retail-vs-ACE incompatibility to resolve in N4's design (retail never assigns new sequences to cleartext). | | N3 | pending | — | | | N4 | pending | — | | | N5 | pending | — | | diff --git a/src/AcDream.Core.Net/Packets/BorrowedPacket.cs b/src/AcDream.Core.Net/Packets/BorrowedPacket.cs index b369b7d6..12c0ca02 100644 --- a/src/AcDream.Core.Net/Packets/BorrowedPacket.cs +++ b/src/AcDream.Core.Net/Packets/BorrowedPacket.cs @@ -28,14 +28,9 @@ internal readonly record struct BorrowedOptionalHeader( uint ConnectRequestClientSeed, ReadOnlyMemory RawBytes, ReadOnlyMemory RetransmitRequestBytes, - int RetransmitRequestCount); - -internal readonly record struct BorrowedPacketDecodeResult( - BorrowedPacket Packet, - PacketCodec.DecodeError Error) -{ - public bool IsOk => Error == PacketCodec.DecodeError.None; -} + int RetransmitRequestCount, + ReadOnlyMemory RejectRetransmitBytes, + int RejectRetransmitCount); internal readonly record struct BorrowedMessageFragment( MessageFragmentHeader Header, diff --git a/src/AcDream.Core.Net/Packets/PacketCodec.cs b/src/AcDream.Core.Net/Packets/PacketCodec.cs index 9033dc8b..04a8c0f0 100644 --- a/src/AcDream.Core.Net/Packets/PacketCodec.cs +++ b/src/AcDream.Core.Net/Packets/PacketCodec.cs @@ -116,29 +116,41 @@ public static class PacketCodec } /// - /// Decode and verify one packet without materializing packet, optional- - /// header, fragment-list, body, or fragment-payload objects. Returned - /// memory borrows and must not outlive it. + /// Parse one packet without materializing packet, optional-header, + /// fragment-list, body, or fragment-payload objects, and WITHOUT + /// touching any keystream: checksum verification is the separate + /// step, because the key decision belongs + /// to the inbound sequence tracker (campaign doc §2.2 — retail's own + /// factoring: ReceiverData::Decrypt takes the key as an optional + /// in/out). Returned memory borrows and + /// must not outlive it. On success and + /// carry the two checksum summands + /// (payload = optional-section hash + Σ fragment hashes). /// - internal static BorrowedPacketDecodeResult TryDecodeBorrowed( + internal static bool TryParseBorrowed( ReadOnlyMemory datagram, - IsaacRandom? inboundIsaac) + out BorrowedPacket packet, + out uint headerHash, + out uint payloadHash, + out DecodeError error) { + packet = default; + headerHash = 0; + payloadHash = 0; + ReadOnlySpan wire = datagram.Span; if (wire.Length < PacketHeader.Size) { - return new BorrowedPacketDecodeResult( - default, - DecodeError.TooShort); + error = DecodeError.TooShort; + return false; } PacketHeader header = PacketHeader.Unpack(wire); int bodyLength = header.DataSize; if (wire.Length - PacketHeader.Size < bodyLength) { - return new BorrowedPacketDecodeResult( - default, - DecodeError.HeaderSizeExceedsBuffer); + error = DecodeError.HeaderSizeExceedsBuffer; + return false; } ReadOnlyMemory body = datagram.Slice( @@ -150,9 +162,8 @@ public static class PacketCodec out BorrowedOptionalHeader optional, out int optionalConsumed)) { - return new BorrowedPacketDecodeResult( - default, - DecodeError.InvalidOptionalHeader); + error = DecodeError.InvalidOptionalHeader; + return false; } ReadOnlyMemory fragmentBytes = @@ -171,9 +182,8 @@ public static class PacketCodec out int payloadLength, out int consumed)) { - return new BorrowedPacketDecodeResult( - default, - DecodeError.InvalidFragment); + error = DecodeError.InvalidFragment; + return false; } fragmentHash += @@ -190,47 +200,39 @@ public static class PacketCodec } } - uint headerHash = header.CalculateHeaderHash32(); - uint optionalHash = Hash32.Calculate( - optional.RawBytes.Span); - uint payloadHash = optionalHash + fragmentHash; - if (header.HasFlag( - PacketHeaderFlags.EncryptedChecksum)) - { - if (inboundIsaac is null) - { - return new BorrowedPacketDecodeResult( - default, - DecodeError.ChecksumMismatch); - } - - uint expectedKey = - (header.Checksum - headerHash) ^ payloadHash; - uint isaacKey = inboundIsaac.Next(); - if (expectedKey != isaacKey) - { - return new BorrowedPacketDecodeResult( - default, - DecodeError.ChecksumMismatch); - } - } - else if (header.Checksum != headerHash + payloadHash) - { - return new BorrowedPacketDecodeResult( - default, - DecodeError.ChecksumMismatch); - } - - return new BorrowedPacketDecodeResult( - new BorrowedPacket( - header, - optional, - body, - fragmentBytes, - fragmentCount), - DecodeError.None); + headerHash = header.CalculateHeaderHash32(); + payloadHash = + Hash32.Calculate(optional.RawBytes.Span) + fragmentHash; + packet = new BorrowedPacket( + header, + optional, + body, + fragmentBytes, + fragmentCount); + error = DecodeError.None; + return true; } + /// + /// Verify a parsed packet's checksum from the + /// summands. + /// null → the additive cleartext form + /// (checksum == headerHash + payloadHash); non-null → the + /// encrypted form + /// (checksum == headerHash + (key ^ payloadHash)). The caller + /// (the inbound sequence tracker via WorldSession) owns every + /// key decision — parked vs freshly drawn — so this function never + /// touches a keystream. + /// + internal static bool VerifyChecksum( + in PacketHeader header, + uint headerHash, + uint payloadHash, + uint? isaacKey) => + isaacKey is uint key + ? header.Checksum == headerHash + (key ^ payloadHash) + : header.Checksum == headerHash + payloadHash; + private static bool TryParseBorrowedOptional( ReadOnlyMemory bodyMemory, PacketHeaderFlags flags, @@ -251,6 +253,8 @@ public static class PacketCodec uint connectRequestClientSeed = 0; int retransmitOffset = 0; int retransmitCount = 0; + int rejectOffset = 0; + int rejectCount = 0; if (HasFlag(flags, PacketHeaderFlags.ServerSwitch) && !Take(body, ref position, 8)) @@ -288,7 +292,13 @@ public static class PacketCodec { return Invalid(out optional, out consumed); } - position += checked((int)count * 4); + + // N2: expose the abandoned ids (SharedNet::HandleEmptyAck + // @ 0x005448F0 consumes them). The bytes were always inside the + // hashed span; only the borrowed view is new. + rejectOffset = position; + rejectCount = checked((int)count); + position += rejectCount * 4; } if (HasFlag(flags, PacketHeaderFlags.AckSequence)) @@ -318,7 +328,9 @@ public static class PacketCodec connectRequestServerSeed, connectRequestClientSeed, retransmitOffset, - retransmitCount); + retransmitCount, + rejectOffset, + rejectCount); consumed = position; return true; } @@ -418,7 +430,9 @@ public static class PacketCodec connectRequestServerSeed, connectRequestClientSeed, retransmitOffset, - retransmitCount); + retransmitCount, + rejectOffset, + rejectCount); consumed = position; return true; } @@ -437,7 +451,9 @@ public static class PacketCodec uint connectRequestServerSeed, uint connectRequestClientSeed, int retransmitOffset, - int retransmitCount) => + int retransmitCount, + int rejectOffset, + int rejectCount) => new( ackSequence, timeSync, @@ -455,7 +471,13 @@ public static class PacketCodec : body.Slice( retransmitOffset, retransmitCount * 4), - retransmitCount); + retransmitCount, + rejectCount == 0 + ? ReadOnlyMemory.Empty + : body.Slice( + rejectOffset, + rejectCount * 4), + rejectCount); private static bool Invalid( out BorrowedOptionalHeader optional, diff --git a/src/AcDream.Core.Net/Packets/PacketHeaderOptional.cs b/src/AcDream.Core.Net/Packets/PacketHeaderOptional.cs index 464001f1..8c651cfb 100644 --- a/src/AcDream.Core.Net/Packets/PacketHeaderOptional.cs +++ b/src/AcDream.Core.Net/Packets/PacketHeaderOptional.cs @@ -38,6 +38,10 @@ public sealed class PacketHeaderOptional public uint AckSequence { get; private set; } public IReadOnlyList RetransmitRequests { get; private set; } = Array.Empty(); + /// N2: sequence ids the server refuses to retransmit + /// (RejectRetransmit 0x2000) — the client abandons the matching + /// NAK-set entries (SharedNet::HandleEmptyAck @ 0x005448F0). + public IReadOnlyList RejectRetransmits { get; private set; } = Array.Empty(); public double TimeSync { get; private set; } public float EchoRequestClientTime { get; private set; } public uint FlowBytes { get; private set; } @@ -94,7 +98,13 @@ public sealed class PacketHeaderOptional if (!Take(body, ref pos, 4)) return -1; uint count = BinaryPrimitives.ReadUInt32LittleEndian(body.Slice(pos - 4)); if (count > 1024 || body.Length - pos < (int)count * 4) return -1; - pos += (int)count * 4; // consume without storing + var rejected = new uint[count]; + for (int i = 0; i < count; i++) + { + rejected[i] = BinaryPrimitives.ReadUInt32LittleEndian(body.Slice(pos)); + pos += 4; + } + RejectRetransmits = rejected; } if (HasFlag(flags, PacketHeaderFlags.AckSequence)) diff --git a/src/AcDream.Core.Net/Transport/InboundSequenceTracker.cs b/src/AcDream.Core.Net/Transport/InboundSequenceTracker.cs new file mode 100644 index 00000000..c5bea03b --- /dev/null +++ b/src/AcDream.Core.Net/Transport/InboundSequenceTracker.cs @@ -0,0 +1,257 @@ +using System.Buffers.Binary; +using AcDream.Core.Net.Cryptography; + +namespace AcDream.Core.Net.Transport; + +/// +/// The inbound half of retail's reliable transport (ReceiverData +/// under SharedNet::receivers_): owns the inbound ISAAC keystream, +/// the received-sequence watermark (highestIDReceived_), and the NAK +/// set (m_SeqIDsWeNAKed — retail keeps an AVL of sequence → parked +/// keystream word; a gives the +/// same raw-uint in-order enumeration N4's NAK emission needs). +/// +/// +/// The invariant the whole inbound port hangs on (campaign doc §2.2): the +/// inbound keystream is aligned to SEQUENCE order, not arrival order. There +/// is no transport reorder buffer — packets process on arrival +/// (SharedNet::ProcessPacket @ 0x00544790 → +/// ProcessNewSeqNum @ 0x00544690); when a gap opens, one keystream +/// word per missing id is pre-drawn IN SEQUENCE ORDER and parked beside the +/// id, BEFORE the arriving packet's own key (landmine #4 — reversed, the +/// stream is off by the gap size forever). +/// +/// +/// Ported rules, per packet: +/// +/// Sanity (SharedNet::SeqIDSanityCheck @ 0x00543A20): drop when +/// the sequence is wrap-safe newer than highestIDReceived_ + 0x7FFF. +/// Encrypted AND not newer than the watermark = duplicate/late +/// arrival: remove the sequence from the NAK set — hit → decrypt with the +/// PARKED pre-drawn key; miss → drop silently at ZERO keystream cost +/// (ProcessNewSeqNum @ 0x00544690, the AVL::Remove branch). +/// Newer than the watermark → gap walk +/// (SharedNet::ProcessNewestSeqNum @ 0x00541930): walk end = +/// encrypted ? seq : seq + 1 (a cleartext packet borrows an +/// already-delivered sequence, so the borrowed id itself gets NAKed — the +/// real encrypted packet at that id may still be in flight); for each id +/// (skipping id 0) AddNakked(id, null) pre-draws one word; then the +/// watermark becomes seq. +/// An encrypted packet's own verify key: the parked key when step 2 +/// found one, else the NEXT drawn word (ReceiverData::Decrypt's +/// optional in/out key — the same factoring +/// CryptoSystem::EncryptData @ 0x0065FF40 uses outbound). +/// Checksum-verify FAILURE on a sequenced encrypted packet → +/// re-parks the consumed key +/// (ProcessPacket @ 0x00544790 tail: AddNakked(seq, &key)) +/// so the retransmission decodes. +/// Inbound RejectRetransmit +/// (SharedNet::HandleEmptyAck @ 0x005448F0) → +/// removes the ids — silent abandonment; +/// the parked keys are discarded and alignment holds because the words were +/// already drawn. +/// +/// +/// +/// Single-threaded by design (the ISAAC keystream is order-sensitive), like +/// the rest of the transport: every member runs on the session's frame +/// thread. +/// +/// +internal sealed class InboundSequenceTracker +{ + /// + /// ACE ADAPTATION (watermark INIT only, not a mechanism change; register + /// AD-50): retail zero-initializes ReceiverData (so + /// highestIDReceived_ starts 0), but ACE never emits S2C sequence + /// 1 — its PacketSequence starts unprimed at + /// uint.MaxValue, the cleartext ConnectRequest takes NextValue 0, + /// and the first ENCRYPTED flush re-primes CurrentValue to 1 so the + /// first encrypted sequenced packet is 2 (ACE + /// NetworkSession.cs:716-717 + Sequence/UIntSequence.cs:9-13,30-41, + /// verified against the N0 double). A zero-init watermark would walk the + /// permanent id-1 hole into a spurious NAK, mis-assign the pre-drawn + /// word to id 1, and desync the keystream on the very first encrypted + /// packet. holtburger seeds the same value for the same reason + /// (crates/holtburger-session/src/session/api.rs:30, + /// last_server_seq: 1), mirroring ACE's own C2S-side + /// lastReceivedPacketSequence = 1 (NetworkSession.cs:57). + /// + internal const uint AceInitialWatermark = 1; + + /// Retail's acceptance horizon above the watermark + /// (SeqIDSanityCheck @ 0x00543A20). + internal const uint SanityWindow = 0x7FFF; + + private readonly IsaacRandom _inboundIsaac; + private readonly TransportStats _stats; + + /// Retail m_SeqIDsWeNAKed: missing sequence → the + /// pre-drawn keystream word parked for it. + private readonly SortedDictionary _nakSet = new(); + + /// Retail highestIDReceived_ — the newest sequence ever + /// admitted (NOT "highest fully processed"; the gap walk advances it + /// past holes). + public uint HighestIdReceived { get; private set; } + + /// Missing ids currently carrying a parked key — the value + /// N4's RequestRetransmit emission drains. + public int NakCount => _nakSet.Count; + + public InboundSequenceTracker( + IsaacRandom inboundIsaac, + TransportStats stats, + uint initialWatermark = AceInitialWatermark) + { + ArgumentNullException.ThrowIfNull(inboundIsaac); + ArgumentNullException.ThrowIfNull(stats); + _inboundIsaac = inboundIsaac; + _stats = stats; + HighestIdReceived = initialWatermark; + } + + /// + /// The verdict for one arriving sequenced packet. — + /// discard without touching the checksum. Otherwise verify with + /// : the keystream word for encrypted packets, + /// null for the additive cleartext form. + /// + public readonly record struct Admission(bool Drop, uint? VerifyKey) + { + public static Admission Dropped => new(true, null); + + public static Admission Process(uint? verifyKey) => + new(false, verifyKey); + } + + /// + /// Steps 1–4 above for one arriving packet with a non-zero sequence. + /// Sequence-0 packets never reach the tracker: retail's + /// ProcessPacket @ 0x00544790 routes cleartext seq-0 + /// (handshake/control) around ProcessNewSeqNum entirely and drops + /// encrypted seq-0 — the caller owns that split. + /// + public Admission Admit(uint sequence, bool encrypted) + { + // Step 1 — SeqIDSanityCheck @ 0x00543A20: wrap-safe horizon. + // highest + 0x7FFF itself is accepted; one past it is dropped. + if (SequenceMath.IsNewer( + sequence, + unchecked(HighestIdReceived + SanityWindow))) + { + _stats.InboundSanityDrops++; + return Admission.Dropped; + } + + bool newer = SequenceMath.IsNewer(sequence, HighestIdReceived); + + // Step 2 — duplicate/late arrival (encrypted, at or below the + // watermark): a NAK-set hit hands back the parked pre-drawn key; a + // miss is a duplicate of an already-decoded packet and drops at + // zero keystream cost. Cleartext packets skip this entirely + // (retail reprocesses cleartext dups; they never touch the wheel). + uint? parkedKey = null; + if (encrypted && !newer) + { + if (!_nakSet.Remove(sequence, out uint parked)) + { + _stats.InboundDupsDropped++; + return Admission.Dropped; + } + + parkedKey = parked; + } + + // Step 3 — gap walk (ProcessNewestSeqNum @ 0x00541930): pre-draw + // one word per missing id IN SEQUENCE ORDER, BEFORE the arriving + // packet's own key (landmine #4). Cleartext walks one past its own + // sequence — the borrowed-id rule — and id 0 is skipped (retail's + // `if (esi_1 != 0)`). + if (newer) + { + uint end = encrypted ? sequence : unchecked(sequence + 1u); + for (uint id = unchecked(HighestIdReceived + 1u); + id != end; + id = unchecked(id + 1u)) + { + if (id != 0) + AddNakked(id); + } + + HighestIdReceived = sequence; + } + + if (!encrypted) + return Admission.Process(null); + + // Step 4 — the packet's own key: parked when step 2 found one, + // else the next fresh word. + return Admission.Process(parkedKey ?? _inboundIsaac.Next()); + } + + /// + /// Step 5 — checksum-verify failure on a sequenced encrypted packet: + /// park the consumed key back beside its sequence + /// (ProcessPacket @ 0x00544790 tail, + /// AddNakked(seq, &key)) so the byte-identical retransmission + /// decodes with the same word. Idempotent like retail's AddNakked. + /// + public void ReparkKey(uint sequence, uint key) + { + if (_nakSet.ContainsKey(sequence)) + return; + + _nakSet.Add(sequence, key); + _stats.KeysParked++; + } + + /// + /// Step 6 — inbound RejectRetransmit + /// (SharedNet::HandleEmptyAck @ 0x005448F0): the server no longer + /// has these ids; abandon them silently. The parked keys are discarded — + /// alignment holds because the words were already drawn in sequence + /// order. is the borrowed optional header's + /// raw little-endian u32 id list. + /// + public void OnRejectRetransmit(ReadOnlySpan idBytes, int count) + { + if (count <= 0 || idBytes.Length < count * 4) + return; + + for (int i = 0; i < count; i++) + { + _nakSet.Remove( + BinaryPrimitives.ReadUInt32LittleEndian( + idBytes.Slice(i * 4))); + } + } + + /// + /// Copy the NAKed ids in ascending raw-uint order — the same in-order + /// enumeration retail's AVL yields (ReceiverData::GetNaks + /// @ 0x005490C0 walks it ascending for the ≤114-id NAK list). N4 + /// adds the cap; this is the simple full copy. + /// + public void CopyNakkedSequencesAscending(List destination) + { + ArgumentNullException.ThrowIfNull(destination); + destination.Clear(); + foreach (uint sequence in _nakSet.Keys) + destination.Add(sequence); + } + + /// + /// ReceiverData::AddNakked @ 0x00549240 with a null key pointer: + /// idempotent; a missing entry pre-draws ONE inbound keystream word + /// (CryptoSystem::GetNextCryptoSeed) and parks it beside the id. + /// + private void AddNakked(uint sequence) + { + if (_nakSet.ContainsKey(sequence)) + return; + + _nakSet.Add(sequence, _inboundIsaac.Next()); + _stats.KeysParked++; + } +} diff --git a/src/AcDream.Core.Net/Transport/ReliableTransport.cs b/src/AcDream.Core.Net/Transport/ReliableTransport.cs index 268f9a3d..b323f99d 100644 --- a/src/AcDream.Core.Net/Transport/ReliableTransport.cs +++ b/src/AcDream.Core.Net/Transport/ReliableTransport.cs @@ -5,9 +5,9 @@ namespace AcDream.Core.Net.Transport; /// /// Composition root for the session's reliable transport (campaign doc §4): -/// one , the outbound flow queue (N1), and the -/// unconditional counters. The inbound sequence tracker joins in N2 and the -/// AckNakScheduler in N3/N4 — N1 deliberately leaves ack behavior in +/// one , the outbound flow queue (N1), the +/// inbound sequence tracker (N2), and the unconditional counters. The +/// AckNakScheduler joins in N3/N4 — until then ack behavior stays in /// WorldSession untouched. /// /// @@ -27,10 +27,16 @@ internal sealed class ReliableTransport : IDisposable public OutboundFlowQueue Outbound { get; } + /// N2: the inbound sequence tracker — inbound ISAAC, + /// highestIDReceived_, and the NAK set. Born beside the outbound + /// queue at ISAAC-seeding time so both keystreams share one owner. + public InboundSequenceTracker Inbound { get; } + public TransportStats Stats { get; } public ReliableTransport( IsaacRandom outboundIsaac, + IsaacRandom inboundIsaac, ushort sessionClientId, DatagramSendDelegate send, TransportClock? clock = null, @@ -45,6 +51,7 @@ internal sealed class ReliableTransport : IDisposable Stats, send, pool); + Inbound = new InboundSequenceTracker(inboundIsaac, Stats); Stats.CacheDepthSource = () => Outbound.CacheDepth; } diff --git a/src/AcDream.Core.Net/Transport/TransportStats.cs b/src/AcDream.Core.Net/Transport/TransportStats.cs index e28f6c6a..6e3eb639 100644 --- a/src/AcDream.Core.Net/Transport/TransportStats.cs +++ b/src/AcDream.Core.Net/Transport/TransportStats.cs @@ -27,6 +27,26 @@ internal sealed class TransportStats /// (explicit acks plus the NAK ids[0] implicit ack). public long AcksConsumed; + /// N2: inbound duplicates of already-decoded packets, dropped + /// at zero keystream cost (encrypted, at/below the watermark, no parked + /// key — ProcessNewSeqNum @ 0x00544690). + public long InboundDupsDropped; + + /// N2: inbound packets past the wrap-safe + /// watermark + 0x7FFF horizon + /// (SeqIDSanityCheck @ 0x00543A20). + public long InboundSanityDrops; + + /// N2: inbound packets whose checksum failed verification + /// after admission (a sequenced encrypted failure also re-parks its + /// consumed key for the retransmission). + public long ChecksumFailures; + + /// N2: inbound keystream words parked in the NAK set — one per + /// gap-walked missing id (ReceiverData::AddNakked @ 0x00549240 + /// pre-draw) plus one per checksum-failure re-park. + public long KeysParked; + /// Live sent-packet cache depth — the N5 watchdog value /// (cache=N in [net-tick]; the cache is unbounded like /// retail's, so depth is the health signal, not a cap). diff --git a/src/AcDream.Core.Net/WorldSession.cs b/src/AcDream.Core.Net/WorldSession.cs index 51d1e591..c8a82045 100644 --- a/src/AcDream.Core.Net/WorldSession.cs +++ b/src/AcDream.Core.Net/WorldSession.cs @@ -673,17 +673,16 @@ public sealed class WorldSession : IDisposable Environment.GetEnvironmentVariable("ACDREAM_DUMP_APPEARANCE") == "1"; private readonly System.Collections.Generic.HashSet _seenUnhandledOpcodes = new(); - private IsaacRandom? _inboundIsaac; private ushort _sessionClientId; private ushort _sessionIteration; private bool _transportNegotiated; /// - /// Campaign N Slice N1: the reliable outbound transport — outbound - /// ISAAC, packet/fragment sequences, sent-packet cache, resend on NAK. - /// Constructed at ISAAC-seeding time in ; null - /// before negotiation (reliable sends are impossible then anyway — the - /// keystream does not exist yet). + /// Campaign N Slices N1+N2: the reliable transport — both ISAAC + /// keystreams, packet/fragment sequences, sent-packet cache, resend on + /// NAK (outbound), and the sequence-aligned inbound tracker + NAK set + /// (inbound). Constructed at ISAAC-seeding time in ; + /// null before negotiation (neither keystream exists yet). /// private ReliableTransport? _transport; @@ -827,22 +826,39 @@ public sealed class WorldSession : IDisposable PooledInboundDatagram datagram = received.Value; try { - BorrowedPacketDecodeResult dec = - PacketCodec.TryDecodeBorrowed( + // N2: pure parse + cleartext verify. No tracker exists + // before the ISAAC seeds do, and the ConnectRequest is a + // cleartext sequence-0 handshake packet; anything encrypted + // here is undecodable and skipped, matching the pre-N2 + // null-keystream behavior. + bool parsedOk = PacketCodec.TryParseBorrowed( datagram.Memory, - inboundIsaac: null); - if (dec.IsOk - && dec.Packet.Header.HasFlag( + out BorrowedPacket parsed, + out uint headerHash, + out uint payloadHash, + out _); + PacketHeader parsedHeader = parsed.Header; + if (parsedOk + && !parsedHeader.HasFlag( + PacketHeaderFlags.EncryptedChecksum) + && PacketCodec.VerifyChecksum( + in parsedHeader, + headerHash, + payloadHash, + isaacKey: null) + && parsedHeader.HasFlag( PacketHeaderFlags.ConnectRequest)) { - connectRequest = dec.Packet.Optional with + connectRequest = parsed.Optional with { RawBytes = ReadOnlyMemory.Empty, RetransmitRequestBytes = ReadOnlyMemory.Empty, + RejectRetransmitBytes = + ReadOnlyMemory.Empty, }; connectRequestIteration = - dec.Packet.Header.Iteration; + parsedHeader.Iteration; connectRequestReceived = true; } } @@ -869,20 +885,22 @@ public sealed class WorldSession : IDisposable BinaryPrimitives.WriteUInt32LittleEndian(serverSeedBytes, opt.ConnectRequestServerSeed); byte[] clientSeedBytes = new byte[4]; BinaryPrimitives.WriteUInt32LittleEndian(clientSeedBytes, opt.ConnectRequestClientSeed); - _inboundIsaac = new IsaacRandom(serverSeedBytes); _sessionClientId = (ushort)opt.ConnectRequestClientId; // SharedNet::SendOptionalHeader @ 0x00543160 copies this ReceiverData // generation into connection-level control packets, including the // final disconnect. ACE currently emits iteration 1. _sessionIteration = connectRequestIteration; - // N1: the reliable transport is born at ISAAC-seeding time, owning - // the outbound keystream + packet/fragment sequences the session - // used to hold directly. highestIDSent starts 1 (the ConnectResponse - // below carries sequence 1), so the first reliable packet after the - // handshake keeps packet sequence 2 and fragment sequence 1 — - // byte-identical to the pre-N1 wire behavior. + // N1+N2: the reliable transport is born at ISAAC-seeding time, + // owning BOTH keystreams. Outbound: highestIDSent starts 1 (the + // ConnectResponse below carries sequence 1), so the first reliable + // packet after the handshake keeps packet sequence 2 and fragment + // sequence 1 — byte-identical to the pre-N1 wire behavior. Inbound: + // the tracker owns the server keystream, the received watermark, + // and the NAK set (campaign §2.2); its watermark starts 1 (see + // InboundSequenceTracker.AceInitialWatermark). _transport = new ReliableTransport( new IsaacRandom(clientSeedBytes), + new IsaacRandom(serverSeedBytes), _sessionClientId, datagram => _net.Send(datagram)); _transportNegotiated = true; @@ -1292,19 +1310,87 @@ public sealed class WorldSession : IDisposable List? opcodesOut = null, bool dispatchWorldEvents = true) { - BorrowedPacketDecodeResult dec = - PacketCodec.TryDecodeBorrowed( + if (!PacketCodec.TryParseBorrowed( bytes, - _inboundIsaac); - if (!dec.IsOk) return; + out BorrowedPacket packet, + out uint headerHash, + out uint payloadHash, + out _)) + { + return; + } + + PacketHeader serverHeader = packet.Header; + bool encrypted = serverHeader.HasFlag( + PacketHeaderFlags.EncryptedChecksum); + + // N2: retail's inbound admission split (SharedNet::ProcessPacket + // @ 0x00544790 → ProcessNewSeqNum @ 0x00544690). Sequence-0 packets + // bypass the tracker entirely: cleartext seq-0 is handshake/control + // (verified additively, processed as before); encrypted seq-0 does + // not exist on the wire and drops before any keystream is touched. + if (serverHeader.Sequence == 0) + { + if (encrypted + || !PacketCodec.VerifyChecksum( + in serverHeader, + headerHash, + payloadHash, + isaacKey: null)) + { + return; + } + } + else if (_transport is { } inboundTransport) + { + InboundSequenceTracker.Admission admission = + inboundTransport.Inbound.Admit( + serverHeader.Sequence, + encrypted); + if (admission.Drop) + return; + + if (!PacketCodec.VerifyChecksum( + in serverHeader, + headerHash, + payloadHash, + admission.VerifyKey)) + { + inboundTransport.Stats.ChecksumFailures++; + // Verify failure on a sequenced encrypted packet re-parks + // the consumed key so the retransmission decodes + // (ProcessPacket @ 0x00544790 tail — campaign §2.2 step 5). + if (encrypted) + { + inboundTransport.Inbound.ReparkKey( + serverHeader.Sequence, + admission.VerifyKey!.Value); + } + + return; + } + } + else + { + // Sequenced traffic before negotiation — the pre-N2 behavior of + // a null inbound keystream: encrypted cannot verify; cleartext + // verifies additively. + if (encrypted + || !PacketCodec.VerifyChecksum( + in serverHeader, + headerHash, + payloadHash, + isaacKey: null)) + { + return; + } + } // Retail LinkStatusHolder::OnHeartbeat @ 0x004113D0 updates its - // last-heard clock only for valid server traffic. Record at decode + // last-heard clock only for valid server traffic. Record at checksum // acceptance, before any heavy render-thread message handling. Volatile.Write(ref _lastInboundPacketTicks, Stopwatch.GetTimestamp()); - PacketHeader serverHeader = dec.Packet.Header; - // N1: consume the transport control surfaces FIRST, before the // reflex ack below (which still fires unchanged this slice; the // AckNakScheduler replaces it in N3). @@ -1315,17 +1401,29 @@ public sealed class WorldSession : IDisposable // implicit cumulative ack (RecipientData::ProcessNaks // @ 0x00547010). The resends go out on the next sweep. if ((serverHeader.Flags & PacketHeaderFlags.RequestRetransmit) != 0 - && dec.Packet.Optional.RetransmitRequestCount > 0) + && packet.Optional.RetransmitRequestCount > 0) { transport.Outbound.OnRetransmitRequest( - dec.Packet.Optional.RetransmitRequestBytes.Span, - dec.Packet.Optional.RetransmitRequestCount); + packet.Optional.RetransmitRequestBytes.Span, + packet.Optional.RetransmitRequestCount); + } + + // N2: inbound RejectRetransmit (0x2000) — the server abandoned + // these ids; drop them from the NAK set, discarding the parked + // keys (SharedNet::HandleEmptyAck @ 0x005448F0). Alignment + // holds: the words were already drawn in sequence order. + if ((serverHeader.Flags & PacketHeaderFlags.RejectRetransmit) != 0 + && packet.Optional.RejectRetransmitCount > 0) + { + transport.Inbound.OnRejectRetransmit( + packet.Optional.RejectRetransmitBytes.Span, + packet.Optional.RejectRetransmitCount); } // Cumulative ack (AckSequence 0x4000): wrap-safe max into the // watermark; the cache prunes strictly below it on the sweep. if ((serverHeader.Flags & PacketHeaderFlags.AckSequence) != 0) - transport.Outbound.OnAckSequence(dec.Packet.Optional.AckSequence); + transport.Outbound.OnAckSequence(packet.Optional.AckSequence); } // Phase 4.9: send an ACK_SEQUENCE control packet for every received @@ -1346,7 +1444,7 @@ public sealed class WorldSession : IDisposable // periodically — no explicit opcode, just the header flag. if ((serverHeader.Flags & PacketHeaderFlags.TimeSync) != 0) { - double t = dec.Packet.Optional.TimeSync; + double t = packet.Optional.TimeSync; if (t > 0) { LastServerTimeTicks = t; @@ -1355,7 +1453,7 @@ public sealed class WorldSession : IDisposable } foreach (BorrowedMessageFragment frag - in dec.Packet.Fragments) + in packet.Fragments) { if (!_assembler.TryIngest( frag, diff --git a/tests/AcDream.Core.Net.Tests/Packets/BorrowedPacketCodecTests.cs b/tests/AcDream.Core.Net.Tests/Packets/BorrowedPacketCodecTests.cs index 4aeec1c4..5ea2aef3 100644 --- a/tests/AcDream.Core.Net.Tests/Packets/BorrowedPacketCodecTests.cs +++ b/tests/AcDream.Core.Net.Tests/Packets/BorrowedPacketCodecTests.cs @@ -4,10 +4,16 @@ using AcDream.Core.Net.Packets; namespace AcDream.Core.Net.Tests.Packets; +/// +/// The borrowed decode path, post-N2 split: TryParseBorrowed is the +/// pure parse + hash computation (no keystream anywhere) and +/// VerifyChecksum is the separate additive/encrypted comparison the +/// inbound sequence tracker keys. +/// public class BorrowedPacketCodecTests { [Fact] - public void TryDecodeBorrowed_AllOptionalFieldsAndFragments_MatchesOwnedDecoder() + public void TryParseBorrowed_AllOptionalFieldsAndFragments_MatchesOwnedDecoder() { const PacketHeaderFlags flags = PacketHeaderFlags.ServerSwitch @@ -78,43 +84,68 @@ public class BorrowedPacketCodecTests PacketCodec.PacketDecodeResult owned = PacketCodec.TryDecode(datagram, inboundIsaac: null); - BorrowedPacketDecodeResult borrowed = - PacketCodec.TryDecodeBorrowed(datagram, inboundIsaac: null); + bool parsedOk = PacketCodec.TryParseBorrowed( + datagram, + out BorrowedPacket borrowed, + out uint headerHash, + out uint payloadHash, + out PacketCodec.DecodeError error); + Assert.True(parsedOk); + Assert.Equal(PacketCodec.DecodeError.None, error); AssertEquivalent(owned, borrowed); - Assert.Equal(serverTime, borrowed.Packet.Optional.ConnectRequestServerTime); - Assert.Equal(cookie, borrowed.Packet.Optional.ConnectRequestCookie); - Assert.Equal(clientId, borrowed.Packet.Optional.ConnectRequestClientId); - Assert.Equal(serverSeed, borrowed.Packet.Optional.ConnectRequestServerSeed); - Assert.Equal(clientSeed, borrowed.Packet.Optional.ConnectRequestClientSeed); - Assert.Equal(timeSync, borrowed.Packet.Optional.TimeSync); - Assert.Equal(echoTime, borrowed.Packet.Optional.EchoRequestClientTime); - Assert.Equal(0xA0A1A2A3u, borrowed.Packet.Optional.FlowBytes); - Assert.Equal(0xB0B1, borrowed.Packet.Optional.FlowInterval); + AssertCleartextChecksumHolds(borrowed, headerHash, payloadHash); + Assert.Equal(serverTime, borrowed.Optional.ConnectRequestServerTime); + Assert.Equal(cookie, borrowed.Optional.ConnectRequestCookie); + Assert.Equal(clientId, borrowed.Optional.ConnectRequestClientId); + Assert.Equal(serverSeed, borrowed.Optional.ConnectRequestServerSeed); + Assert.Equal(clientSeed, borrowed.Optional.ConnectRequestClientSeed); + Assert.Equal(timeSync, borrowed.Optional.TimeSync); + Assert.Equal(echoTime, borrowed.Optional.EchoRequestClientTime); + Assert.Equal(0xA0A1A2A3u, borrowed.Optional.FlowBytes); + Assert.Equal(0xB0B1, borrowed.Optional.FlowInterval); Assert.Equal( new uint[] { 0x10111213, 0x20212223 }, - ReadRetransmits(borrowed.Packet.Optional)); + ReadIds( + borrowed.Optional.RetransmitRequestBytes, + borrowed.Optional.RetransmitRequestCount)); + + // N2: the RejectRetransmit ids are exposed on both decoders (they + // were always inside the hashed span; only the views are new). + Assert.Equal( + new uint[] { 0x30313233 }, + ReadIds( + borrowed.Optional.RejectRetransmitBytes, + borrowed.Optional.RejectRetransmitCount)); + Assert.Equal( + new uint[] { 0x30313233 }, + owned.Packet!.Optional.RejectRetransmits); } [Fact] - public void TryDecodeBorrowed_LoginPayload_MatchesOwnedDecoder() + public void TryParseBorrowed_LoginPayload_MatchesOwnedDecoder() { byte[] payload = LoginRequest.Build("borrowed", "packet", 47); byte[] datagram = Encode(PacketHeaderFlags.LoginRequest, payload); PacketCodec.PacketDecodeResult owned = PacketCodec.TryDecode(datagram, inboundIsaac: null); - BorrowedPacketDecodeResult borrowed = - PacketCodec.TryDecodeBorrowed(datagram, inboundIsaac: null); + Assert.True(PacketCodec.TryParseBorrowed( + datagram, + out BorrowedPacket borrowed, + out uint headerHash, + out uint payloadHash, + out _)); AssertEquivalent(owned, borrowed); - Assert.Equal(payload, borrowed.Packet.Optional.RawBytes.ToArray()); - Assert.Equal(payload, borrowed.Packet.Body.ToArray()); - Assert.Equal(0, borrowed.Packet.FragmentCount); + AssertCleartextChecksumHolds(borrowed, headerHash, payloadHash); + Assert.Equal(payload, borrowed.Optional.RawBytes.ToArray()); + Assert.Equal(payload, borrowed.Body.ToArray()); + Assert.Equal(0, borrowed.FragmentCount); } [Fact] - public void TryDecodeBorrowed_EncryptedChecksum_MatchesOwnedDecoder() + public void VerifyChecksum_EncryptedForm_MatchesOwnedDecoder() { byte[] body = new byte[4]; BinaryPrimitives.WriteUInt32LittleEndian(body, 0x12345678); @@ -129,28 +160,41 @@ public class BorrowedPacketCodecTests PacketCodec.TryDecode( datagram, new IsaacRandom(seed)); - BorrowedPacketDecodeResult borrowed = - PacketCodec.TryDecodeBorrowed( - datagram, - new IsaacRandom(seed)); - + Assert.True(owned.IsOk); + Assert.True(PacketCodec.TryParseBorrowed( + datagram, + out BorrowedPacket borrowed, + out uint headerHash, + out uint payloadHash, + out _)); AssertEquivalent(owned, borrowed); + + // The parse never touched a keystream: the caller supplies the key. + PacketHeader header = borrowed.Header; + uint key = new IsaacRandom(seed).Next(); + Assert.True(PacketCodec.VerifyChecksum( + in header, headerHash, payloadHash, key)); + Assert.False(PacketCodec.VerifyChecksum( + in header, headerHash, payloadHash, key ^ 1u)); + // The additive cleartext form must not accept an encrypted checksum. + Assert.False(PacketCodec.VerifyChecksum( + in header, headerHash, payloadHash, isaacKey: null)); } [Fact] - public void TryDecodeBorrowed_MalformedPackets_MatchOwnedDecoderErrors() + public void TryParseBorrowed_MalformedPackets_MatchOwnedDecoderErrors() { byte[] shortHeader = new byte[PacketHeader.Size - 1]; - AssertSameError(shortHeader); + AssertSameParseError(shortHeader); var oversized = new byte[PacketHeader.Size + 2]; new PacketHeader { DataSize = 3 }.Pack(oversized); - AssertSameError(oversized); + AssertSameParseError(oversized); byte[] shortOptional = EncodeUnchecked( PacketHeaderFlags.TimeSync, [0x01, 0x02, 0x03, 0x04]); - AssertSameError(shortOptional); + AssertSameParseError(shortOptional); byte[] zeroCount = BuildFragment( sequence: 1, @@ -158,7 +202,7 @@ public class BorrowedPacketCodecTests index: 0, queue: 0, payload: [0x01]); - AssertSameError(EncodeUnchecked( + AssertSameParseError(EncodeUnchecked( PacketHeaderFlags.BlobFragments, zeroCount)); @@ -168,19 +212,34 @@ public class BorrowedPacketCodecTests index: 1, queue: 0, payload: [0x01]); - AssertSameError(EncodeUnchecked( + AssertSameParseError(EncodeUnchecked( PacketHeaderFlags.BlobFragments, invalidIndex)); + // A checksum mismatch is no longer a parse error: the parse + // succeeds and VerifyChecksum reports the failure (the owned + // decoder still folds it into DecodeError.ChecksumMismatch). byte[] wrongChecksum = Encode( PacketHeaderFlags.AckSequence, [1, 2, 3, 4]); wrongChecksum[8] ^= 0x80; - AssertSameError(wrongChecksum); + Assert.Equal( + PacketCodec.DecodeError.ChecksumMismatch, + PacketCodec.TryDecode(wrongChecksum, inboundIsaac: null).Error); + Assert.True(PacketCodec.TryParseBorrowed( + wrongChecksum, + out BorrowedPacket parsed, + out uint headerHash, + out uint payloadHash, + out PacketCodec.DecodeError error)); + Assert.Equal(PacketCodec.DecodeError.None, error); + PacketHeader header = parsed.Header; + Assert.False(PacketCodec.VerifyChecksum( + in header, headerHash, payloadHash, isaacKey: null)); } [Fact] - public void TryDecodeBorrowed_WarmSingleFragmentPath_AllocatesNothing() + public void TryParseBorrowed_WarmSingleFragmentPath_AllocatesNothing() { byte[] fragment = BuildFragment( sequence: 77, @@ -204,13 +263,26 @@ public class BorrowedPacketCodecTests private static int DecodeAndRead(ReadOnlyMemory datagram) { - BorrowedPacketDecodeResult decoded = - PacketCodec.TryDecodeBorrowed( + if (!PacketCodec.TryParseBorrowed( datagram, - inboundIsaac: null); - int checksum = decoded.Packet.FragmentCount; + out BorrowedPacket decoded, + out uint headerHash, + out uint payloadHash, + out _)) + { + return 0; + } + + PacketHeader header = decoded.Header; + if (!PacketCodec.VerifyChecksum( + in header, headerHash, payloadHash, isaacKey: null)) + { + return 0; + } + + int checksum = decoded.FragmentCount; foreach (BorrowedMessageFragment fragment - in decoded.Packet.Fragments) + in decoded.Fragments) { checksum += fragment.Header.TotalSize; checksum += fragment.Payload.Span[0]; @@ -219,62 +291,76 @@ public class BorrowedPacketCodecTests return checksum; } + private static void AssertCleartextChecksumHolds( + in BorrowedPacket borrowed, + uint headerHash, + uint payloadHash) + { + PacketHeader header = borrowed.Header; + Assert.True(PacketCodec.VerifyChecksum( + in header, headerHash, payloadHash, isaacKey: null)); + } + private static void AssertEquivalent( PacketCodec.PacketDecodeResult owned, - BorrowedPacketDecodeResult borrowed) + in BorrowedPacket borrowed) { - Assert.Equal(owned.Error, borrowed.Error); - if (!owned.IsOk) - return; - + Assert.True(owned.IsOk); Packet packet = Assert.IsType(owned.Packet); - Assert.Equal(packet.Header, borrowed.Packet.Header); - Assert.Equal(packet.BodyBytes, borrowed.Packet.Body.ToArray()); + Assert.Equal(packet.Header, borrowed.Header); + Assert.Equal(packet.BodyBytes, borrowed.Body.ToArray()); Assert.Equal( packet.Optional.RawBytes, - borrowed.Packet.Optional.RawBytes.ToArray()); + borrowed.Optional.RawBytes.ToArray()); Assert.Equal( packet.Optional.AckSequence, - borrowed.Packet.Optional.AckSequence); + borrowed.Optional.AckSequence); Assert.Equal( packet.Optional.TimeSync, - borrowed.Packet.Optional.TimeSync); + borrowed.Optional.TimeSync); Assert.Equal( packet.Optional.EchoRequestClientTime, - borrowed.Packet.Optional.EchoRequestClientTime); + borrowed.Optional.EchoRequestClientTime); Assert.Equal( packet.Optional.FlowBytes, - borrowed.Packet.Optional.FlowBytes); + borrowed.Optional.FlowBytes); Assert.Equal( packet.Optional.FlowInterval, - borrowed.Packet.Optional.FlowInterval); + borrowed.Optional.FlowInterval); Assert.Equal( packet.Optional.ConnectRequestServerTime, - borrowed.Packet.Optional.ConnectRequestServerTime); + borrowed.Optional.ConnectRequestServerTime); Assert.Equal( packet.Optional.ConnectRequestCookie, - borrowed.Packet.Optional.ConnectRequestCookie); + borrowed.Optional.ConnectRequestCookie); Assert.Equal( packet.Optional.ConnectRequestClientId, - borrowed.Packet.Optional.ConnectRequestClientId); + borrowed.Optional.ConnectRequestClientId); Assert.Equal( packet.Optional.ConnectRequestServerSeed, - borrowed.Packet.Optional.ConnectRequestServerSeed); + borrowed.Optional.ConnectRequestServerSeed); Assert.Equal( packet.Optional.ConnectRequestClientSeed, - borrowed.Packet.Optional.ConnectRequestClientSeed); + borrowed.Optional.ConnectRequestClientSeed); Assert.Equal( packet.Optional.RetransmitRequests, - ReadRetransmits(borrowed.Packet.Optional)); + ReadIds( + borrowed.Optional.RetransmitRequestBytes, + borrowed.Optional.RetransmitRequestCount)); + Assert.Equal( + packet.Optional.RejectRetransmits, + ReadIds( + borrowed.Optional.RejectRetransmitBytes, + borrowed.Optional.RejectRetransmitCount)); var fragments = new List(); foreach (BorrowedMessageFragment fragment - in borrowed.Packet.Fragments) + in borrowed.Fragments) { fragments.Add(fragment); } - Assert.Equal(packet.Fragments.Count, borrowed.Packet.FragmentCount); + Assert.Equal(packet.Fragments.Count, borrowed.FragmentCount); Assert.Equal(packet.Fragments.Count, fragments.Count); for (int index = 0; index < fragments.Count; index++) { @@ -287,30 +373,33 @@ public class BorrowedPacketCodecTests } } - private static uint[] ReadRetransmits( - BorrowedOptionalHeader optional) + private static uint[] ReadIds( + ReadOnlyMemory idMemory, + int count) { - var retransmits = new uint[optional.RetransmitRequestCount]; - ReadOnlySpan bytes = optional.RetransmitRequestBytes.Span; - for (int index = 0; index < retransmits.Length; index++) + var ids = new uint[count]; + ReadOnlySpan bytes = idMemory.Span; + for (int index = 0; index < ids.Length; index++) { - retransmits[index] = + ids[index] = BinaryPrimitives.ReadUInt32LittleEndian( bytes.Slice(index * 4)); } - return retransmits; + return ids; } - private static void AssertSameError(byte[] datagram) + private static void AssertSameParseError(byte[] datagram) { PacketCodec.PacketDecodeResult owned = PacketCodec.TryDecode(datagram, inboundIsaac: null); - BorrowedPacketDecodeResult borrowed = - PacketCodec.TryDecodeBorrowed( - datagram, - inboundIsaac: null); - Assert.Equal(owned.Error, borrowed.Error); + Assert.False(PacketCodec.TryParseBorrowed( + datagram, + out _, + out _, + out _, + out PacketCodec.DecodeError borrowedError)); + Assert.Equal(owned.Error, borrowedError); } private static byte[] Encode( diff --git a/tests/AcDream.Core.Net.Tests/Transport/FakeAceTransport.cs b/tests/AcDream.Core.Net.Tests/Transport/FakeAceTransport.cs index 3b92b600..2f8356cb 100644 --- a/tests/AcDream.Core.Net.Tests/Transport/FakeAceTransport.cs +++ b/tests/AcDream.Core.Net.Tests/Transport/FakeAceTransport.cs @@ -127,6 +127,22 @@ internal sealed class FakeAceTransport : IWorldSessionTransport } } + /// + /// N2 test hook: deliver raw bytes straight into the client's receive + /// queue, bypassing both the model and the link. Used for late + /// byte-identical redelivery of a dropped S2C datagram (ACE's cached + /// packet shape), duplicate injections, and crafted sequence-0 control + /// packets. + /// + public void InjectServerDatagram(byte[] datagram) + { + lock (_gate) + { + _toClient.Enqueue((byte[])datagram.Clone()); + _deliverable.Release(); + } + } + private void PumpServerLocked() { Model.Update(); diff --git a/tests/AcDream.Core.Net.Tests/Transport/InboundSequenceTrackerTests.cs b/tests/AcDream.Core.Net.Tests/Transport/InboundSequenceTrackerTests.cs new file mode 100644 index 00000000..28c50de7 --- /dev/null +++ b/tests/AcDream.Core.Net.Tests/Transport/InboundSequenceTrackerTests.cs @@ -0,0 +1,664 @@ +using System.Buffers.Binary; +using System.Net; +using AcDream.Core.Net.Cryptography; +using AcDream.Core.Net.Messages; +using AcDream.Core.Net.Packets; +using AcDream.Core.Net.Transport; + +namespace AcDream.Core.Net.Tests.Transport; + +/// +/// Campaign N Slice N2 — inbound sequence-aligned ISAAC + the NAK set. +/// Unit tests pin the tracker against a shadow ISAAC seeded identically +/// (the shadow's draw order IS the sequence order the tracker must keep); +/// conformance tests run a REAL against the N0 +/// ACE-behaviour double and prove the second fatal bug is gone: one lost +/// S2C packet no longer desyncs the inbound keystream. +/// +public sealed class InboundSequenceTrackerTests +{ + private const uint Seed = 0x5EED5EEDu; + + // ===================================================================== + // The decisive unit test — the bug that exists today: pre-N2, the + // arrival-order burn desyncs the keystream at the first gap and every + // later encrypted packet fails checksum forever. + // ===================================================================== + + [Fact] + public void GapWalk_ParksTheMissingKey_LaterPacketsAndLateArrivalDecode() + { + (InboundSequenceTracker tracker, TransportStats stats) = + CreateTracker(initialWatermark: 9); + IsaacRandom shadow = MakeIsaac(Seed); + uint w10 = shadow.Next(); + uint w11 = shadow.Next(); + uint w12 = shadow.Next(); + uint w13 = shadow.Next(); + uint w14 = shadow.Next(); + uint w15 = shadow.Next(); + + // In-order packets draw in sequence order. + Assert.Equal(w10, Admitted(tracker, 10)); + Assert.Equal(w11, Admitted(tracker, 11)); + + // Sequence 12 is lost. 13 arrives: the walk pre-draws 12's word and + // parks it BEFORE 13's own key (landmine #4) — 13 decodes with w13, + // not w12 (impossible pre-N2). + Assert.Equal(w13, Admitted(tracker, 13)); + Assert.Equal(1, tracker.NakCount); + Assert.Equal(1, stats.KeysParked); + Assert.Equal(13u, tracker.HighestIdReceived); + + // 14 keeps flowing on the aligned stream. + Assert.Equal(w14, Admitted(tracker, 14)); + + // The late 12 decodes with the PARKED key at zero fresh cost. + Assert.Equal(w12, Admitted(tracker, 12)); + Assert.Equal(0, tracker.NakCount); + Assert.Equal(14u, tracker.HighestIdReceived); + + // And the next fresh packet takes the next fresh word. + Assert.Equal(w15, Admitted(tracker, 15)); + Assert.Equal(0, stats.InboundDupsDropped); + Assert.Equal(0, stats.InboundSanityDrops); + } + + // ===================================================================== + // Duplicates and re-parks — ProcessNewSeqNum @ 0x00544690 step 2 and + // ProcessPacket @ 0x00544790 step 5 + // ===================================================================== + + [Fact] + public void Duplicate_NeverNakked_DropsAtZeroKeystreamCost() + { + (InboundSequenceTracker tracker, TransportStats stats) = + CreateTracker(initialWatermark: 9); + IsaacRandom shadow = MakeIsaac(Seed); + uint w10 = shadow.Next(); + uint w11 = shadow.Next(); + + Assert.Equal(w10, Admitted(tracker, 10)); + + // A duplicate of the already-decoded 10: dropped BEFORE any + // keystream access (pre-N2 this burned one word and shifted the + // stream). + InboundSequenceTracker.Admission dup = tracker.Admit(10, encrypted: true); + Assert.True(dup.Drop); + Assert.Equal(1, stats.InboundDupsDropped); + + // The shadow position is unchanged: 11 draws the very next word. + Assert.Equal(w11, Admitted(tracker, 11)); + } + + [Fact] + public void ChecksumFailureRepark_TheRetransmissionDecodesWithTheSameKey() + { + (InboundSequenceTracker tracker, TransportStats stats) = + CreateTracker(initialWatermark: 9); + IsaacRandom shadow = MakeIsaac(Seed); + uint w10 = shadow.Next(); + uint w11 = shadow.Next(); + + // 10 arrives corrupt: admission consumed w10, verification failed, + // the session re-parks the consumed key (step 5) and drops. + Assert.Equal(w10, Admitted(tracker, 10)); + tracker.ReparkKey(10, w10); + Assert.Equal(1, tracker.NakCount); + Assert.Equal(1, stats.KeysParked); + + // The byte-identical retransmission decodes with the SAME word. + Assert.Equal(w10, Admitted(tracker, 10)); + Assert.Equal(0, tracker.NakCount); + + // Alignment held throughout. + Assert.Equal(w11, Admitted(tracker, 11)); + } + + // ===================================================================== + // The cleartext walk rules — ProcessNewestSeqNum @ 0x00541930 + // ===================================================================== + + [Fact] + public void Cleartext_AtHighestPlusOne_NaksItsOwnBorrowedId() + { + // The least obvious line of the port: a cleartext packet walks to + // seq + 1, so its OWN sequence gets NAKed — cleartext borrows an + // already-delivered sequence, and the real encrypted packet at that + // id may still be in flight (retail `if ((header_ & 2) == 0) + // seqID_ += 1`). + (InboundSequenceTracker tracker, TransportStats stats) = + CreateTracker(initialWatermark: 9); + IsaacRandom shadow = MakeIsaac(Seed); + uint w10 = shadow.Next(); + uint w11 = shadow.Next(); + + InboundSequenceTracker.Admission cleartext = + tracker.Admit(10, encrypted: false); + Assert.False(cleartext.Drop); + Assert.Null(cleartext.VerifyKey); + Assert.Equal(10u, tracker.HighestIdReceived); + Assert.Equal(1, tracker.NakCount); + Assert.Equal(1, stats.KeysParked); + + // The real encrypted 10 arrives later: parked key, exact word. + Assert.Equal(w10, Admitted(tracker, 10)); + Assert.Equal(0, tracker.NakCount); + Assert.Equal(w11, Admitted(tracker, 11)); + } + + [Fact] + public void Cleartext_AtHighest_NoNak_NoKey_NoWatermarkChange() + { + (InboundSequenceTracker tracker, TransportStats stats) = + CreateTracker(initialWatermark: 9); + IsaacRandom shadow = MakeIsaac(Seed); + uint w10 = shadow.Next(); + uint w11 = shadow.Next(); + + Assert.Equal(w10, Admitted(tracker, 10)); + + // ACE's pure ack reuses the current sequence: not newer → no walk, + // no key, no watermark movement, processed additively. + InboundSequenceTracker.Admission ack = + tracker.Admit(10, encrypted: false); + Assert.False(ack.Drop); + Assert.Null(ack.VerifyKey); + Assert.Equal(10u, tracker.HighestIdReceived); + Assert.Equal(0, tracker.NakCount); + Assert.Equal(0, stats.KeysParked); + + Assert.Equal(w11, Admitted(tracker, 11)); + } + + // ===================================================================== + // Sanity window — SeqIDSanityCheck @ 0x00543A20 + // ===================================================================== + + [Fact] + public void SanityWindow_HighestPlus0x7FFF_Accepted_OnePastIt_Dropped() + { + // The accept boundary (walks the whole window — retail does too). + (InboundSequenceTracker accepted, _) = + CreateTracker(initialWatermark: 100); + InboundSequenceTracker.Admission atBoundary = + accepted.Admit(100u + 0x7FFFu, encrypted: true); + Assert.False(atBoundary.Drop); + Assert.Equal(100u + 0x7FFFu, accepted.HighestIdReceived); + + // One past it: dropped at zero keystream cost. + (InboundSequenceTracker dropped, TransportStats stats) = + CreateTracker(initialWatermark: 100); + IsaacRandom shadow = MakeIsaac(Seed); + uint w101 = shadow.Next(); + InboundSequenceTracker.Admission pastBoundary = + dropped.Admit(100u + 0x8000u, encrypted: true); + Assert.True(pastBoundary.Drop); + Assert.Equal(1, stats.InboundSanityDrops); + Assert.Equal(100u, dropped.HighestIdReceived); + Assert.Equal(0, dropped.NakCount); + Assert.Equal(w101, Admitted(dropped, 101)); + } + + [Fact] + public void SanityWindow_IsWrapSafe() + { + // Watermark near the 32-bit wrap: the horizon lands past 0. + (InboundSequenceTracker tracker, TransportStats stats) = + CreateTracker(initialWatermark: 0xFFFFFF00u); + + // watermark + 0x8000 wraps to 0x7F00 — still one past the horizon, + // still dropped. + InboundSequenceTracker.Admission pastBoundary = + tracker.Admit(unchecked(0xFFFFFF00u + 0x8000u), encrypted: true); + Assert.True(pastBoundary.Drop); + Assert.Equal(1, stats.InboundSanityDrops); + + // An in-window post-wrap sequence is fine (small hop to keep the + // walk short): 0xFFFFFF00 → 3 is newer across the wrap and inside + // the horizon. + InboundSequenceTracker.Admission postWrap = + tracker.Admit(3u, encrypted: true); + Assert.False(postWrap.Drop); + Assert.Equal(3u, tracker.HighestIdReceived); + } + + [Fact] + public void GapWalk_SkipsSequenceZero_AcrossTheWrap() + { + // Retail's walk skips id 0 (`if (esi_1 != 0)`): sequence 0 is the + // handshake id and never carries a keystream word. + (InboundSequenceTracker tracker, _) = + CreateTracker(initialWatermark: 0xFFFFFFFEu); + IsaacRandom shadow = MakeIsaac(Seed); + uint wMax = shadow.Next(); // parked for 0xFFFFFFFF + uint w1 = shadow.Next(); // parked for 1 (0 skipped between) + uint w2 = shadow.Next(); // the arriving packet's own key + + Assert.Equal(w2, Admitted(tracker, 2)); + Assert.Equal(2, tracker.NakCount); + + // Ascending raw-uint order, like retail's AVL enumeration. + var naks = new List(); + tracker.CopyNakkedSequencesAscending(naks); + Assert.Equal(new uint[] { 1u, 0xFFFFFFFFu }, naks); + + // Both parked keys decode their late arrivals. + Assert.Equal(wMax, Admitted(tracker, 0xFFFFFFFFu)); + Assert.Equal(w1, Admitted(tracker, 1)); + Assert.Equal(0, tracker.NakCount); + } + + // ===================================================================== + // RejectRetransmit — HandleEmptyAck @ 0x005448F0 + // ===================================================================== + + [Fact] + public void RejectRetransmit_RemovesIds_AndTheStreamStaysAligned() + { + (InboundSequenceTracker tracker, TransportStats stats) = + CreateTracker(initialWatermark: 9); + IsaacRandom shadow = MakeIsaac(Seed); + uint w10 = shadow.Next(); + _ = shadow.Next(); // w11 — parked, then abandoned + _ = shadow.Next(); // w12 — parked, then abandoned + uint w13 = shadow.Next(); + uint w14 = shadow.Next(); + + Assert.Equal(w10, Admitted(tracker, 10)); + Assert.Equal(w13, Admitted(tracker, 13)); // parks 11 + 12 + Assert.Equal(2, tracker.NakCount); + + // The server answers RejectRetransmit [11, 12]: silent abandonment. + Span ids = stackalloc byte[8]; + BinaryPrimitives.WriteUInt32LittleEndian(ids, 11u); + BinaryPrimitives.WriteUInt32LittleEndian(ids.Slice(4), 12u); + tracker.OnRejectRetransmit(ids, count: 2); + Assert.Equal(0, tracker.NakCount); + + // Alignment holds: the words were already drawn in sequence order. + Assert.Equal(w14, Admitted(tracker, 14)); + + // A very late 11 after abandonment is a plain duplicate now — + // dropped at zero cost. + Assert.True(tracker.Admit(11, encrypted: true).Drop); + Assert.Equal(1, stats.InboundDupsDropped); + } + + // ===================================================================== + // Zero-alloc steady state + // ===================================================================== + + [Fact] + public void WarmAdmit_NoGap_AllocatesNothing() + { + (InboundSequenceTracker tracker, _) = CreateTracker(initialWatermark: 1); + uint sequence = 2; + for (int i = 0; i < 128; i++) + _ = tracker.Admit(sequence++, encrypted: true); + + long before = GC.GetAllocatedBytesForCurrentThread(); + for (int i = 0; i < 1_000; i++) + _ = tracker.Admit(sequence++, encrypted: true); + long allocated = GC.GetAllocatedBytesForCurrentThread() - before; + + Assert.Equal(0, allocated); + } + + // ===================================================================== + // Conformance against the N0 ACE-behaviour double (real WorldSession) + // ===================================================================== + + /// + /// The watermark-init pin: ACE's re-prime dance (cleartext ConnectRequest + /// at sequence 0, first encrypted flush re-primes to 1, first encrypted + /// sequenced packet at 2 — NetworkSession.cs:716-717) must produce ZERO + /// NAKs and zero spurious drops across a clean lifecycle with the + /// tracker's init watermark of 1. + /// + [Fact] + public void CleanLifecycle_ZeroNaks_ZeroSpuriousDrops() + { + var transport = new FakeAceTransport(); + var session = new WorldSession( + new IPEndPoint(IPAddress.Loopback, 9000), + transport); + try + { + session.Connect("testaccount", "testpassword", TimeSpan.FromSeconds(10)); + AssertNoInboundFaults(session); + + session.EnterWorld(0, TimeSpan.FromSeconds(10)); + AssertNoInboundFaults(session); + + // One in-world round trip. + var messages = new List(); + session.ServerMessageReceived += m => messages.Add(m.Message); + transport.Model.EnqueueGameMessage( + BuildServerMessage("clean lifecycle"), + GameMessageGroup.UIQueue); + transport.PumpServer(); + PumpUntil(session, () => messages.Count > 0); + Assert.Equal("clean lifecycle", Assert.Single(messages)); + AssertNoInboundFaults(session); + + // The model's dance, pinned: the first ENCRYPTED sequenced S2C + // packet is sequence 2 (never 1) — the fact the init watermark + // adaptation is built on. + uint minEncrypted = uint.MaxValue; + uint maxSequence = 0; + foreach (byte[] datagram in transport.Model.SentDatagrams) + { + PacketHeader header = PacketHeader.Unpack(datagram); + if (header.HasFlag(PacketHeaderFlags.EncryptedChecksum) + && header.Sequence != 0 + && header.Sequence < minEncrypted) + { + minEncrypted = header.Sequence; + } + + maxSequence = SequenceMath.Max(maxSequence, header.Sequence); + } + + Assert.Equal(2u, minEncrypted); + + // The tracker followed the whole stream: watermark == the + // newest sequence ACE ever emitted, with an empty NAK set. + Assert.Equal( + maxSequence, + session.Transport!.Inbound.HighestIdReceived); + } + finally + { + session.Dispose(); + } + + Assert.Equal(WorldSession.State.Disconnected, session.CurrentState); + Assert.Equal(0, transport.Model.CrcDropCount); + } + + /// + /// The N2 win end-to-end: one lost S2C packet mid-stream (a Count=2 + /// fragment set spanning two packets — the model splits >448 B + /// messages) no longer kills the inbound cipher. Later packets STILL + /// DECODE, the missing id carries a parked key, and ACE's cached-shape + /// late redelivery completes the fragment set intact. + /// + [Fact] + public void S2CLoss_LaterPacketsStillDecode_LateRedeliveryCompletesTheMessage() + { + var transport = new FakeAceTransport(); + var session = new WorldSession( + new IPEndPoint(IPAddress.Loopback, 9000), + transport); + try + { + session.Connect("testaccount", "testpassword", TimeSpan.FromSeconds(10)); + session.EnterWorld(0, TimeSpan.FromSeconds(10)); + + var messages = new List(); + session.ServerMessageReceived += m => messages.Add(m.Message); + + // A >448 B message splits into a Count=2 fragment set spanning + // two S2C packets; the link eats the FIRST of them. + string bigText = new string('x', 600); + transport.Link.DropNext(LinkDirection.ServerToClient); + transport.Model.EnqueueGameMessage( + BuildServerMessage(bigText), + GameMessageGroup.UIQueue); + transport.PumpServer(); + PumpUntil( + session, + () => session.Transport!.Stats.KeysParked > 0); + + // The missing id is parked, the message can't complete yet. + Assert.Equal(1, session.Transport!.Inbound.NakCount); + Assert.Equal(1, session.Transport.Stats.KeysParked); + Assert.Empty(messages); + + // THE WIN: traffic AFTER the loss still decodes (pre-N2 every + // later encrypted packet failed checksum forever). + transport.Model.EnqueueGameMessage( + BuildServerMessage("after the loss"), + GameMessageGroup.UIQueue); + transport.PumpServer(); + PumpUntil(session, () => messages.Count > 0); + Assert.Equal("after the loss", Assert.Single(messages)); + Assert.Equal(0, session.Transport.Stats.ChecksumFailures); + + // Late byte-identical redelivery of the dropped datagram (the + // bytes ACE cached and would resend): decodes with the parked + // key, completes the fragment set, dispatches intact. + var parked = new List(); + session.Transport.Inbound.CopyNakkedSequencesAscending(parked); + uint missingSequence = Assert.Single(parked); + byte[]? droppedDatagram = null; + foreach (byte[] datagram in transport.Model.SentDatagrams) + { + if (PacketHeader.Unpack(datagram).Sequence == missingSequence) + { + droppedDatagram = datagram; + break; + } + } + + Assert.NotNull(droppedDatagram); + transport.InjectServerDatagram(droppedDatagram!); + PumpUntil(session, () => messages.Count > 1); + + Assert.Equal(2, messages.Count); + Assert.Equal(bigText, messages[1]); + Assert.Equal(0, session.Transport.Inbound.NakCount); + Assert.Equal(0, session.Transport.Stats.InboundDupsDropped); + Assert.Equal(0, session.Transport.Stats.ChecksumFailures); + } + finally + { + session.Dispose(); + } + } + + /// + /// The double-dispatch bug closes: a duplicate delivery of an + /// already-processed server packet drops BEFORE dispatch, at zero + /// keystream cost. + /// + [Fact] + public void DuplicateServerPacket_DropsBeforeDispatch() + { + var transport = new FakeAceTransport(); + var session = new WorldSession( + new IPEndPoint(IPAddress.Loopback, 9000), + transport); + try + { + session.Connect("testaccount", "testpassword", TimeSpan.FromSeconds(10)); + session.EnterWorld(0, TimeSpan.FromSeconds(10)); + + var messages = new List(); + session.ServerMessageReceived += m => messages.Add(m.Message); + + transport.Model.EnqueueGameMessage( + BuildServerMessage("once only"), + GameMessageGroup.UIQueue); + transport.PumpServer(); + PumpUntil(session, () => messages.Count > 0); + Assert.Equal("once only", Assert.Single(messages)); + + // Redeliver the exact datagram that carried it. + byte[]? carrier = null; + foreach (byte[] datagram in transport.Model.SentDatagrams) + { + PacketHeader header = PacketHeader.Unpack(datagram); + if (header.HasFlag(PacketHeaderFlags.BlobFragments) + && header.Sequence + == session.Transport!.Inbound.HighestIdReceived) + { + carrier = datagram; + } + } + + Assert.NotNull(carrier); + transport.InjectServerDatagram(carrier!); + PumpUntil( + session, + () => session.Transport!.Stats.InboundDupsDropped > 0); + + // Dropped before dispatch: the message did NOT arrive twice. + Assert.Single(messages); + Assert.Equal(1, session.Transport!.Stats.InboundDupsDropped); + + // And the keystream did not move: fresh traffic still decodes. + transport.Model.EnqueueGameMessage( + BuildServerMessage("still aligned"), + GameMessageGroup.UIQueue); + transport.PumpServer(); + PumpUntil(session, () => messages.Count > 1); + Assert.Equal("still aligned", messages[1]); + Assert.Equal(0, session.Transport.Stats.ChecksumFailures); + } + finally + { + session.Dispose(); + } + } + + /// + /// Sequence-0 packets bypass the tracker entirely: cleartext seq-0 is + /// handshake/control (processed, no watermark/NAK movement); encrypted + /// seq-0 drops before any keystream access. + /// + [Fact] + public void SequenceZero_CleartextBypassesTracker_EncryptedDrops() + { + var transport = new FakeAceTransport(); + var session = new WorldSession( + new IPEndPoint(IPAddress.Loopback, 9000), + transport); + try + { + session.Connect("testaccount", "testpassword", TimeSpan.FromSeconds(10)); + session.EnterWorld(0, TimeSpan.FromSeconds(10)); + + uint watermarkBefore = session.Transport!.Inbound.HighestIdReceived; + long parkedBefore = session.Transport.Stats.KeysParked; + + var serverTimes = new List(); + session.ServerTimeUpdated += t => serverTimes.Add(t); + + // A crafted cleartext seq-0 TimeSync control packet processes + // through the handshake/control path without touching the + // tracker. + byte[] timeSyncBody = new byte[8]; + BinaryPrimitives.WriteInt64LittleEndian( + timeSyncBody, + BitConverter.DoubleToInt64Bits(777.5)); + transport.InjectServerDatagram(PacketCodec.Encode( + new PacketHeader + { + Sequence = 0, + Flags = PacketHeaderFlags.TimeSync, + }, + timeSyncBody, + outboundIsaac: null)); + PumpUntil(session, () => serverTimes.Contains(777.5)); + + Assert.Equal( + watermarkBefore, + session.Transport.Inbound.HighestIdReceived); + Assert.Equal(0, session.Transport.Inbound.NakCount); + Assert.Equal(parkedBefore, session.Transport.Stats.KeysParked); + + // An encrypted seq-0 packet does not exist on retail's wire: + // dropped before any keystream access — later traffic proves + // the wheel never moved. + var throwawayIsaac = MakeIsaac(0xDEADBEEFu); + transport.InjectServerDatagram(PacketCodec.Encode( + new PacketHeader + { + Sequence = 0, + Flags = PacketHeaderFlags.TimeSync + | PacketHeaderFlags.EncryptedChecksum, + }, + timeSyncBody, + throwawayIsaac)); + + var messages = new List(); + session.ServerMessageReceived += m => messages.Add(m.Message); + transport.Model.EnqueueGameMessage( + BuildServerMessage("wheel intact"), + GameMessageGroup.UIQueue); + transport.PumpServer(); + PumpUntil(session, () => messages.Count > 0); + Assert.Equal("wheel intact", Assert.Single(messages)); + Assert.Equal(0, session.Transport.Stats.ChecksumFailures); + } + finally + { + session.Dispose(); + } + } + + // ===================================================================== + // Fixture helpers + // ===================================================================== + + private static (InboundSequenceTracker Tracker, TransportStats Stats) + CreateTracker(uint initialWatermark) + { + var stats = new TransportStats(); + return ( + new InboundSequenceTracker(MakeIsaac(Seed), stats, initialWatermark), + stats); + } + + /// Admit an encrypted packet that must NOT drop; returns the + /// verify key the tracker handed out. + private static uint Admitted(InboundSequenceTracker tracker, uint sequence) + { + InboundSequenceTracker.Admission admission = + tracker.Admit(sequence, encrypted: true); + Assert.False(admission.Drop); + Assert.NotNull(admission.VerifyKey); + return admission.VerifyKey!.Value; + } + + private static void AssertNoInboundFaults(WorldSession session) + { + AcDream.Core.Net.Transport.ReliableTransport? transport = session.Transport; + Assert.NotNull(transport); + Assert.Equal(0, transport!.Inbound.NakCount); + Assert.Equal(0, transport.Stats.KeysParked); + Assert.Equal(0, transport.Stats.InboundDupsDropped); + Assert.Equal(0, transport.Stats.InboundSanityDrops); + Assert.Equal(0, transport.Stats.ChecksumFailures); + } + + private static void PumpUntil(WorldSession session, Func condition) + { + DateTime deadline = DateTime.UtcNow.AddSeconds(10); + while (!condition() && DateTime.UtcNow < deadline) + { + session.Tick(); + Thread.Sleep(5); + } + + Assert.True(condition(), "condition not reached before the deadline"); + } + + private static byte[] BuildServerMessage(string text) + { + var writer = new PacketWriter(64 + text.Length); + writer.WriteUInt32(ServerMessage.Opcode); // 0xF7E0 + writer.WriteString16L(text); + writer.WriteUInt32(1); // ChatMessageType + return writer.ToArray(); + } + + private static IsaacRandom MakeIsaac(uint seed) + { + Span seedBytes = stackalloc byte[4]; + BinaryPrimitives.WriteUInt32LittleEndian(seedBytes, seed); + return new IsaacRandom(seedBytes); + } +}