docs: movement deep dive — AC2D + holtburger cross-reference

Exhaustive analysis of two working AC clients revealing three critical findings that reshape acdream's movement system: 1. Server-authoritative Z: neither AC2D nor holtburger computes local terrain Z for the player. AC2D sends keys, receives position. Holtburger dead-reckons for smoothing but the server overrides. 2. Terrain split formula mismatch: AC2D and ACViewer's render path use 0x0CCAC033-based FSplitNESW; WorldBuilder (our source) uses a different 214614067-based physics formula. Our terrain mesh triangulation doesn't match the real AC client's, causing Z mismatches on slopes. 3. Movement deduplication: MoveToState sent once per state change, not per frame. AutonomousPosition heartbeat every 1 second. Also adds AC2D to CLAUDE.md reference repos section. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 21:52:12 +02:00 · 2026-04-12 21:52:12 +02:00 · 5cd776914a
commit 5cd776914a
parent 29fe0c0714
2 changed files with 343 additions and 0 deletions
--- a/CLAUDE.md
+++ b/CLAUDE.md
@ -182,6 +182,16 @@ these, ideally all four:
  the message-builder layer. ACE shows what the server expects;
  holtburger shows what a real client actually sends.

+- **`references/AC2D/`** — **C++ AC client emulator.** Oldest reference,
+  fixed-function OpenGL, but has the **real AC terrain split formula**
+  (`FSplitNESW` with constants `0x0CCAC033`, `0x421BE3BD`, `0x6C1AC587`,
+  `0x519B8F25`) which differs from WorldBuilder's physics-path formula.
+  Also has the complete `0xF61C` movement packet format with flag bits
+  and the `stMoveInfo` sequence counters. Key lesson from AC2D: it does
+  NOT do client-side terrain Z — it sends movement keys to the server
+  and uses the server's authoritative Z. See
+  `docs/research/2026-04-12-movement-deep-dive.md` for the full analysis.
+
 Pattern: when you encounter an unknown behavior, grep all four for the
 relevant term, read each hit, and compose a multi-source understanding
 BEFORE writing acdream code. A single reference can be misleading; the
--- a/docs/research/2026-04-12-movement-deep-dive.md
+++ b/docs/research/2026-04-12-movement-deep-dive.md
@ -0,0 +1,333 @@
+# Movement System Deep Dive — AC2D + holtburger Cross-Reference
+
+## Executive Summary
+
+After exhaustive analysis of two working AC clients (AC2D in C++, holtburger
+in Rust) plus ACViewer's dual-formula terrain code, three critical findings
+reshape how acdream should handle movement:
+
+1. **The server is authoritative for Z.** Neither AC2D nor holtburger does
+   local terrain Z-snapping for the player character. AC2D sends movement
+   keys to the server and receives authoritative positions back. Holtburger
+   does client-side dead-reckoning for smooth interpolation but the
+   server's `UpdatePosition` overrides it.
+
+2. **AC has TWO different terrain split formulas.** ACViewer's codebase
+   reveals a render-path formula (`0x0CCAC033`) and a physics-path formula
+   (`214614067 / 1813693831`). acdream uses the physics formula for both
+   rendering AND Z-sampling, which means our visual terrain doesn't match
+   the real AC terrain mesh, causing Z mismatches.
+
+3. **Movement is a state machine, not a position pump.** Both clients send
+   "I'm now walking forward" (MoveToState, once) and then periodic position
+   heartbeats (AutonomousPosition, every 1s). They do NOT send a new
+   MoveToState every frame. The server controls the motion; the client
+   reports its state changes.
+
+---
+
+## Part 1: Movement Architecture Comparison
+
+### AC2D (C++ — simple, server-authoritative)
+
+```
+WASD pressed → bAnimUpdate = true
+  ↓
+Per-frame: if bAnimUpdate:
+  1. Compute iFB, iStrafe, iTurn from key states
+  2. Call SendAnimUpdate(iFB, iStrafe, iTurn, !bShift)
+  3. Call SetMoveVelocities(iFB*3.0, iStrafe*-1.0, iTurn*1.5)
+  ↓
+SendAnimUpdate builds 0xF61C packet:
+  - Flags bitmask (0x04=forward, 0x20=strafe, 0x100=turn, 0x01=running)
+  - Motion commands (0x45000005=forward, etc.)
+  - Full stLocation (32 bytes: landblock + XYZ + quaternion)
+  - stMoveInfo (8 bytes: numLogins, moveCount, numPortals, numOverride)
+  ↓
+Server processes movement, echoes back:
+  - 0xF748 (position update) with authoritative position
+  ↓
+Client: CalcPosition.CalcFromLocation(&server_location)
+```
+
+**Key insight:** AC2D does local prediction (SetMoveVelocities + per-frame
+UpdatePosition), but the server's F748 response OVERRIDES the local
+prediction. The client never computes terrain Z — it uses whatever Z the
+server sends.
+
+**Local prediction velocities:**
+- Forward/backward: `iFB * 3.0` AC units/sec (before /240 render division)
+- Strafe: `iStrafe * -1.0`
+- Turn: `iTurn * 1.5` (radians/sec multiplied by PI/2)
+
+**ACK timing:** periodic every 2000ms (not per-packet).
+
+### holtburger (Rust — sophisticated, dead-reckoning)
+
+```
+PlayerDriveIntent queued (Manual, Autonomous, or Stop)
+  ↓
+MovementSystem.tick() every 30ms:
+  1. Expire timed drives
+  2. Ingest queued intents
+  3. Match active drive:
+     - Manual(state)     → send MoveToState (deduped, only on state change)
+     - Autonomous(delta) → compute wire state, send MoveToState
+     - Stop              → send stop MoveToState
+  4. Every 1 second: send AutonomousPosition heartbeat
+  ↓
+SpatialPhysics.solve() every 30ms:
+  - For local player: advance position by desired_world_delta
+  - For entities: integrate velocity + omega (dead reckoning)
+  - Landblock transitions via coordinate math (no portal detection)
+  ↓
+Server sends UpdatePosition → hard override of local position
+Server sends AutonomousPosition → sync sequence counters
+```
+
+**Key insight:** holtburger does client-side position advancement (the
+SpatialPhysics solver) but it's purely dead-reckoning — linear
+extrapolation of velocity, no terrain sampling, no collision. The server
+is still authoritative. The client advances locally for smooth rendering
+and sends heartbeats so the server knows the client's view of its position.
+
+**MoveToState deduplication:** only sent when `current_intent != last_sent_intent`.
+Not sent every frame. Not sent every tick.
+
+**AutonomousPosition heartbeat:** every 1 second (constant
+`AUTONOMOUS_POSITION_HEARTBEAT_INTERVAL`).
+
+**ACK timing:** per-packet (every received packet with sequence > 0 gets
+an ACK queued and piggybacked on the next outbound packet).
+
+---
+
+## Part 2: Critical Protocol Details
+
+### Sequence Counters (from holtburger)
+
+Every MoveToState and AutonomousPosition carries 4 sequence counters:
+```
+instance_sequence:       u16  // object instantiation epoch
+server_control_sequence: u16  // server motion control epoch
+teleport_sequence:       u16  // teleport epoch
+force_position_sequence: u16  // rubber-band epoch
+```
+These are initialized from the server's CreateObject/PlayerCreate message
+and echoed back in every outbound movement packet. The server uses them
+to detect stale/reordered packets. If a packet arrives with an old
+teleport_sequence (from before the last teleport), the server ignores it.
+
+acdream currently sends 0 for all four counters. This is likely why ACE
+accepts our movement grudgingly — it can't detect staleness, so it accepts
+everything, but the server-side validation may be looser than intended.
+
+### RawMotionState Wire Format (from holtburger)
+
+The 0xF61C (MoveToState) payload has a complex packed format:
+```
+packed_flags: u32
+  bits 0-10:  flag bitmask (which optional fields follow)
+  bits 11-31: command list length
+
+Optional fields (each conditional on a flag bit):
+  [bit 0] current_hold_key: u32  (1=None, 2=Run)
+  [bit 1] current_style: u32     (MotionStance, e.g. 0x8000003D)
+  [bit 2] forward_command: u32
+  [bit 3] forward_hold_key: u32
+  [bit 4] forward_speed: f32
+  [bit 5] sidestep_command: u32
+  [bit 6] sidestep_hold_key: u32
+  [bit 7] sidestep_speed: f32
+  [bit 8] turn_command: u32
+  [bit 9] turn_hold_key: u32
+  [bit 10] turn_speed: f32
+
+Command items array (8 bytes each):
+  command: u16
+  packed_sequence: u16
+  speed: f32
+```
+
+### Speed Constants (from holtburger)
+
+| Movement | Speed |
+|----------|-------|
+| Walk forward | base_walk_forward_velocity from MotionTable |
+| Run forward | base_run_forward_velocity * run_rate_scalar (default 4.5) |
+| Backstep | 1.0 (fixed) |
+| Strafe | 1.0 (fixed) |
+| Turn (walk) | 1.0 rad/s |
+| Turn (run) | 1.5 rad/s |
+
+### Heading Convention
+
+AC heading: 0 = facing west (-X), PI/2 = facing north (+Y).
+Velocity from heading: `(-cos(heading) * speed, sin(heading) * speed, 0)`.
+This is DIFFERENT from our current convention where 0 = facing east (+X).
+
+---
+
+## Part 3: Terrain Split Formula Discrepancy
+
+### The Two Formulas
+
+**Render formula** (AC2D + ACViewer render path):
+```cpp
+DWORD dw = x * y * 0x0CCAC033 - x * 0x421BE3BD + y * 0x6C1AC587 - 0x519B8F25;
+bool splitNESW = (dw & 0x80000000) != 0;
+```
+Where x, y are GLOBAL cell coordinates: `(blockX * 8 + cellX, blockY * 8 + cellY)`.
+
+**Physics formula** (ACViewer physics path + WorldBuilder):
+```csharp
+uint seedA = globalX * 214614067;
+uint seedB = globalY * 1109124029;
+float splitDir = (seedA + 1813693831 - seedB - 1369149221) * 2.3283064e-10f;
+bool splitSEtoNW = splitDir >= 0.5f;
+```
+
+**acdream currently uses:** the physics formula (from WorldBuilder).
+
+**Impact:** The render mesh has triangles split along different diagonals
+than the physics mesh in some cells. When we sample terrain Z via bilinear
+interpolation (which doesn't account for split direction at all), the result
+can be above or below the actual triangle surface.
+
+### The Correct Fix
+
+For acdream, we should use the RENDER formula for the terrain mesh (since
+that's what the player sees) and match it in the physics Z sampling. This
+means:
+1. Replace `CalculateSplitDirection` with the AC2D formula for rendering.
+2. Make `TerrainSurface.SampleZ` triangle-aware: given the split direction,
+   determine which triangle the query point falls in, then do barycentric
+   interpolation within that triangle (not bilinear across the whole cell).
+
+---
+
+## Part 4: What acdream Should Change
+
+### Movement System Redesign
+
+**Current (broken):**
+- Send MoveToState every frame when state changes
+- Local physics engine computes terrain Z via bilinear interpolation
+- No AutonomousPosition heartbeat
+- No sequence counters in movement packets
+- Heading convention mismatch (0 = east, should be 0 = west)
+
+**Proposed (matching holtburger's pattern):**
+1. Send MoveToState ONCE when motion state changes (start walking, stop,
+   turn, change gait). Deduplicate — if the intent hasn't changed, don't
+   send.
+2. Send AutonomousPosition every 1 second while moving. Include the
+   server-echoed Z (not locally computed Z).
+3. Track and include the 4 sequence counters in every movement packet.
+4. Use server-sent Z from the last UpdatePosition/AutonomousPosition echo
+   as the authoritative ground truth. Client-side Z is cosmetic only
+   (for smooth interpolation between server updates).
+5. Fix heading convention: 0 = west, PI/2 = north.
+6. Keep local prediction for smooth rendering but let server override.
+
+### Terrain Rendering Fix
+
+1. Replace `CalculateSplitDirection` with the AC2D render formula.
+2. Update `TerrainSurface.SampleZ` to do per-triangle barycentric Z
+   interpolation using the render split direction.
+
+### ACK Pattern Fix
+
+Current: per-packet ACK (correct, matches holtburger).
+Also add: 5-second keepalive ping (from holtburger) to prevent idle timeout.
+
+---
+
+## Part 5: Implementation Priority
+
+1. **Fix terrain split formula** — use AC2D's 0x0CCAC033 render formula.
+   This alone may fix the slope Z clipping because the terrain mesh and
+   Z sampler will agree on triangle boundaries.
+
+2. **Triangle-aware Z sampling** — replace bilinear interpolation with
+   barycentric interpolation within the correct triangle. This eliminates
+   the fundamental mismatch between the visual terrain and the physics Z.
+
+3. **Server-authoritative Z** — stop fighting the local Z computation.
+   Use the server's Z for the player's authoritative position and only
+   use local Z for cosmetic smoothing between server updates.
+
+4. **MoveToState deduplication** — send once per state change, not per
+   frame.
+
+5. **AutonomousPosition heartbeat** — every 1 second.
+
+6. **Sequence counters** — extract from CreateObject/UpdatePosition and
+   echo back in movement packets.
+
+7. **Heading convention** — fix to AC standard (0 = west).
+
+8. **Keepalive ping** — every 5 seconds of idle.
+
+---
+
+## Appendix A: Wire Format Reference
+
+### 0xF61C — MoveToState (AC2D format, simpler)
+```
+DWORD opcode = 0xF61C
+DWORD flags
+  [flag-dependent motion commands and speeds]
+stLocation (32 bytes)
+stMoveInfo (8 bytes): numLogins, moveCount, numPortals, numOverride
+DWORD 1  (unknown constant)
+```
+
+### 0xF753 — AutonomousPosition
+```
+DWORD opcode = 0xF753
+stLocation (32 bytes)
+stMoveInfo (8 bytes)
+DWORD 1
+```
+
+### stLocation (32 bytes)
+```
+DWORD landblock   (XXYY + cell in low 16 bits)
+float x, y, z     (local coords within landblock)
+float w, a, b, c  (quaternion: w=cos, a=x, b=y, c=z)
+```
+
+### stMoveInfo (8 bytes)
+```
+WORD numLogins      (character login count)
+WORD moveCount      (animation sequence from last F74C)
+WORD numPortals     (portal transition count)
+WORD numOverride    (force position count)
+```
+
+---
+
+## Appendix B: AC2D FSplitNESW (the CORRECT render formula)
+
+```cpp
+bool FSplitNESW(DWORD x, DWORD y) {
+    DWORD dw = x * y * 0x0CCAC033 - x * 0x421BE3BD + y * 0x6C1AC587 - 0x519B8F25;
+    return (dw & 0x80000000) != 0;
+}
+// x = blockX * 8 + cellX (global cell X)
+// y = blockY * 8 + cellY (global cell Y)
+// true = NE/SW split, false = NW/SE split
+```
+
+## Appendix C: holtburger Heading Convention
+
+```rust
+// heading = 0 → facing west (-X)
+// heading = PI/2 → facing north (+Y)
+// heading = PI → facing east (+X)
+// heading = 3*PI/2 → facing south (-Y)
+fn planar_velocity_for_heading(heading: f32, speed: f32) -> Vector3 {
+    Vector3::new(-heading.cos() * speed, heading.sin() * speed, 0.0)
+}
+```