docs(workflow): align CLAUDE.md + memory + roadmap with named-retail foundation

CLAUDE.md edits (6 surgical ranges): - Goal section: introduce named-retail/ as primary; old chunks remain as fallback for chunk-by-chunk address-range navigation. - Workflow renamed to "grep named -> decompile -> verify -> port" with a new STEP 0 GREP NAMED FIRST. Decompile demoted to a fallback (Step 1) for the rare obfuscated/packed minority that pseudo-C lacks. - Function-map citation updated to point at symbols.json + the cross-port hand-curated table. - "Do not guess" rule strengthened: PDB has the answer for almost everything; guessing is now negligence. - Phase completion checklist accepts named symbols + addresses. - Reference hierarchy table gets a new top row pointing at docs/research/named-retail/ as the primary oracle for any AC-specific algorithm — beats every other reference. memory/project_named_decompilation.md (new): evergreen crib-sheet with file inventory, grep examples, hard rules. Pattern matches project_ui_architecture.md. memory/project_retail_research_index.md: updated preamble to point named-retail/ as first stop; older slices remain useful for pseudocode + C# port sketches. memory/project_collision_port.md: rewrote the "Decompiled ground truth" section to put named-retail/ first, chunks second. The "DECOMPILE FIRST" mandate becomes "GREP NAMED FIRST, then DECOMPILE FALLBACK". docs/architecture/acdream-architecture.md: Guiding Principle text updated to introduce named-retail as the primary decomp source. docs/plans/2026-04-11-roadmap.md: new Phase R block — Retail research infrastructure. R.1 (corpus, shipped a9a01d8), R.2 (pdb-extract, shipped 69d884a), R.3 (actestclient vendored, shipped a9a01d8). All marked SHIPPED 2026-04-25. Auto-loaded MEMORY.md index updated with a new entry pointing at project_named_decompilation.md so post-compaction sessions inherit the workflow change automatically. Acceptance verified: - grep -c "named-retail" CLAUDE.md = 9 (>= 3 required) - grep -c "named-retail" MEMORY.md = 1 - dotnet build green (docs-only commit, but verified) Foundation phases A + B + C all landed. Next: Phase D files ISSUES #8/#9/#11 + closes #10 (KillerNotification orphan parser). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 17:36:53 +02:00 · 2026-04-25 17:36:53 +02:00 · 0a429a980c
commit 0a429a980c
parent 69d884a3d6
6 changed files with 192 additions and 23 deletions
--- a/CLAUDE.md
+++ b/CLAUDE.md
@ -8,10 +8,17 @@ with a plugin API the original never had.

 **The code is modern. The behavior is retail.**

-Every AC-specific algorithm is ported from the decompiled retail client
-(`docs/research/decompiled/`, 22,225 functions, 688K lines of C). The code
-around those algorithms is modern C# with clean architecture. The plugin API
-exposes game state through well-defined interfaces.
+Every AC-specific algorithm is ported from the **named retail decomp**
+at `docs/research/named-retail/` — Sept 2013 EoR build PDB (18,366
+named functions, 5,371 named struct/class types) + Binary Ninja
+pseudo-C with 99.6% function-name recovery + verbatim retail header
+struct definitions. The older Ghidra `FUN_xxx` chunks under
+`docs/research/decompiled/` (22,225 functions, 688K lines) remain a
+fallback for chunk-by-chunk address-range navigation. **Grep
+`named-retail/acclient_2013_pseudo_c.txt` by `class::method` BEFORE
+decompiling fresh.** The code around those algorithms is modern C#
+with clean architecture. The plugin API exposes game state through
+well-defined interfaces.

 **Architecture:** `docs/architecture/acdream-architecture.md` is the
 single source of truth for how the client is structured. All work must
@ -84,19 +91,33 @@ a phase just landed, and move to the next todo item.
 always yes — keep going.** The single exception is visual verification;
 otherwise, act.

-## Development workflow: decompile → verify → port
+## Development workflow: grep named → decompile → verify → port

 **This is the mandatory workflow for implementing ANY AC-specific behavior.**
 The triangle-boundary Z bug cost 5 failed fix attempts from guessing.
 The animation frame-swap bug cost 4 failed attempts. Every time we
 checked the decompiled code first, we got it right on the first try.
+**Now we have named retail symbols too — Step 0 cuts most lookups
+from 30 minutes to 5 seconds.**

 ### For each new feature or bug fix:

-1. **DECOMPILE FIRST.** Before writing any AC-specific code, find the
-   matching function in the decompiled client (`docs/research/decompiled/`)
-   or decompile a new region using `tools/decompile_acclient.py`. Use
-   the function map at `docs/research/acclient_function_map.md` to find
+0. **GREP NAMED FIRST.** Before any decompilation work, search
+   `docs/research/named-retail/acclient_2013_pseudo_c.txt` by
+   `class::method` name. 99.6% of functions have real names from the
+   Sept 2013 EoR build PDB. `docs/research/named-retail/acclient.h`
+   has every retail struct verbatim. `docs/research/named-retail/symbols.json`
+   is greppable by name or address (regenerate via
+   `py tools/pdb-extract/pdb_extract.py refs/acclient.pdb`). Only fall
+   back to Step 1 below if the named pseudo-C lacks a function (rare —
+   covers only the obfuscated/packed minority).
+
+1. **DECOMPILE FIRST (fallback).** Only when grep-named-first returned
+   nothing. Find the matching function in the older Ghidra chunks at
+   `docs/research/decompiled/` or decompile a new region using
+   `tools/decompile_acclient.py`. Use the function map at
+   `docs/research/acclient_function_map.md` (cross-port index) +
+   `docs/research/named-retail/symbols.json` (raw PDB names) to find
   known functions. If the function isn't mapped yet, search by
   characteristic constants (motion commands, magic numbers, string
   literals).
@ -132,7 +153,11 @@ checked the decompiled code first, we got it right on the first try.
 ### What NOT to do:

 - **Do not guess** at AC-specific algorithms, formulas, constants, wire
-  formats, or coordinate conventions. Ever.
+  formats, or coordinate conventions. Ever. **The named retail decomp
+  has the answer for almost everything; guessing is no longer a
+  recoverable error, it's negligence.** If you can't find it in
+  `docs/research/named-retail/`, file a research note and ASK before
+  writing.
 - **Do not "fix" the decompiled code.** If the retail client does
  something that looks wrong, it's probably right. Verify before
  changing.
@ -149,7 +174,8 @@ checked the decompiled code first, we got it right on the first try.
 Before marking any phase as done:

 - [ ] Every AC-specific algorithm has a decompiled reference cited in
-      comments (function address + chunk file)
+      comments (named symbol + address from `named-retail/symbols.json`,
+      OR function address + chunk file from older `decompiled/` chunks)
 - [ ] Conformance tests exist for the critical paths
 - [ ] The code was cross-referenced against at least 2 reference repos
 - [ ] `dotnet build` green, `dotnet test` green
@ -430,6 +456,7 @@ decompiled client code and would have fixed it in minutes.

 | Domain | Primary Oracle | Secondary | Notes |
 |--------|---------------|-----------|-------|
+| **Any AC-specific algorithm** | **`docs/research/named-retail/`** (PDB-named decomp + verbatim retail header structs from Sept 2013 EoR build) | the existing references below | The retail client itself, fully named. 18,366 functions + 5,371 struct types + 1.4 M lines of pseudo-C in one searchable tree. Beats every other reference for "what does the real client do." |
 | **Terrain** (split direction, height sampling, palCode, vertex position, normals) | **ACME `ClientReference.cs`** — decompiled retail client with exact offsets | ACME `TerrainGeometryGenerator.cs` (matches the mesh index buffer) | WorldBuilder original is SUPERSEDED for terrain algorithms. AC2D confirms the same formula. |
 | **Terrain blending** (texture atlas, alpha masks, road overlays) | **ACME `LandSurfaceManager.cs`** | WorldBuilder original `LandSurfaceManager.cs` (same code, less tested) | Both use the same TexMerge pipeline. ACME has conformance tests. |
 | **GfxObj / Setup rendering** (mesh extraction, multi-part assembly, ObjDesc) | **ACME `StaticObjectManager.cs`** — includes CreaturePalette, GfxObjRemapping, HiddenParts | ACViewer `Render/` namespace | ACME has the complete creature appearance pipeline in one file. |
--- a/docs/architecture/acdream-architecture.md
+++ b/docs/architecture/acdream-architecture.md
@ -17,10 +17,15 @@ A modern C# .NET 10 Asheron's Call client that:

 **The code is modern. The behavior is retail.**

-Every AC-specific algorithm is ported faithfully from the decompiled retail
-client (docs/research/decompiled/, 688K lines). The code AROUND those
-algorithms is modern C# with clean architecture. The plugin API exposes
-game state through well-defined interfaces that the retail client never had.
+Every AC-specific algorithm is ported faithfully from the **named retail
+decomp** at `docs/research/named-retail/` — Sept 2013 EoR build PDB
+(18,366 named functions, 5,371 named struct types) + Binary Ninja
+pseudo-C with 99.6% function-name recovery + verbatim retail header
+struct definitions. The older Ghidra `FUN_xxx` chunks at
+`docs/research/decompiled/` (688K lines) remain a fallback for the
+obfuscated/packed minority. The code AROUND those algorithms is modern
+C# with clean architecture. The plugin API exposes game state through
+well-defined interfaces that the retail client never had.

 ---

--- a/docs/plans/2026-04-11-roadmap.md
+++ b/docs/plans/2026-04-11-roadmap.md
@ -223,6 +223,41 @@ Not detailed here; each gets its own brainstorm when it becomes relevant.

 ---

+### Phase R — Retail research infrastructure
+
+**Goal:** sustainable, scalable access to retail-client decompilation —
+named symbols, struct layouts, wire schemas — so every future port is
+a 5-second grep instead of 30-minute archaeology.
+
+**Sub-pieces:**
+
+- **✓ SHIPPED — R.1 — Named-retail corpus committed.** Shipped 2026-04-25
+  (commit `a9a01d8`). `docs/research/named-retail/{acclient_2013_pseudo_c.txt,
+  acclient.h, acclient.c}` + `docs/research/data/spells.csv` + vendored
+  `references/actestclient/`. 1.4 M lines of named pseudo-C (99.6% function
+  naming) + 70K lines of verbatim retail headers + 3,956 spells with `Family`
+  for buff stacking + machine-readable wire schema in `messages.xml`.
+- **✓ SHIPPED — R.2 — pdb-extract tool + JSON sidecars.** Shipped 2026-04-25
+  (commit `69d884a`). `tools/pdb-extract/pdb_extract.py` reads
+  `refs/acclient.pdb` (Sept 2013 EoR build) and emits `symbols.json`
+  (18,366 named functions) + `types.json` (5,371 named struct types) to
+  `docs/research/named-retail/`. Pure Python, no deps, runs in <1 s.
+- **✓ SHIPPED — R.3 — actestclient vendored.** Shipped 2026-04-25 alongside
+  R.1. `references/actestclient/` (covered by `references/` gitignore)
+  includes the canonical `messages.xml` AC-protocol wire schema.
+
+**Acceptance:** Step 0 of `CLAUDE.md`'s development workflow points at
+`named-retail/` first; subsequent issue closures (#6 / #7 / #9 / #11)
+all consume this foundation.
+
+**Effects on other phases:** Issue closures unblocked by Phase R land
+under their existing letter phases (#6 enchantment buffs → Phase D /
+F.5 maintenance; #7 PlayerDescription trailer → Phase H.1 / F.2
+maintenance). The PDB symbols + headers also accelerate any future
+port in any phase — no separate listing here.
+
+---
+
 ## Cross-cutting work tracked in parallel

 - **Test coverage.** Each phase lands with unit + integration tests in `tests/`. Current count: 98 Core + 96 Core.Net = 194. Keep the ratio as new phases land.
--- a/memory/project_collision_port.md
+++ b/memory/project_collision_port.md
@ -62,10 +62,19 @@ ground truth. ACE has the ENTIRE system already in C#:
 - `references/ACE/Source/ACE.Server/Physics/Collision/CollisionInfo.cs`
 - `references/ACE/Source/ACE.Server/Physics/Collision/ObjectInfo.cs`

-### Decompiled ground truth:
- `docs/research/decompiled/chunk_00530000.c` — BSP, Polygon, Sphere collision
- `docs/research/decompiled/chunk_00500000.c` — PhysicsObj, transition callers
- `docs/research/acclient_function_map.md` — mapped functions
+### Decompiled ground truth (named-retail is now primary, 2026-04-25):
+- **`docs/research/named-retail/acclient_2013_pseudo_c.txt`** — grep for
+  `BSPTree::`, `BSPNode::`, `BSPLeaf::`, `CPolygon::`, `CCylSphere::`,
+  `Transition::`, `CPhysicsObj::`, `SpherePath::` to find named bodies.
+- **`docs/research/named-retail/acclient.h`** — verbatim retail struct
+  layouts for the BSP / Sphere / Transition types.
+- **`docs/research/named-retail/symbols.json`** — name↔address lookup.
+- `docs/research/decompiled/chunk_00530000.c` — older Ghidra fallback for
+  BSP / Polygon / Sphere collision (FUN_xxx names).
+- `docs/research/decompiled/chunk_00500000.c` — older Ghidra fallback for
+  PhysicsObj / transition callers.
+- `docs/research/acclient_function_map.md` — hand-curated cross-port index
+  (ACE / ACME mappings + struct-offset notes).

 ### Pseudocode (already written):
 - `docs/research/transition_pseudocode.md` — full system documented
@ -90,10 +99,12 @@ ground truth. ACE has the ENTIRE system already in C#:

 For EVERY function:

-1. **DECOMPILE FIRST.** Find the matching function in the decompiled
-   client (`docs/research/decompiled/`). Use the function map at
-   `docs/research/acclient_function_map.md`. If not mapped, search
-   by characteristic constants or struct offsets.
+1. **GREP NAMED FIRST, then DECOMPILE FALLBACK.** Search the named
+   retail decomp first: `grep -n "ClassName::Method" docs/research/named-retail/acclient_2013_pseudo_c.txt`.
+   For struct layouts: `grep -n "^struct ClassName" docs/research/named-retail/acclient.h`.
+   Only if the named pseudo-C lacks a function (rare), fall back to the
+   older `docs/research/decompiled/` chunks via the function map at
+   `docs/research/acclient_function_map.md`.

 2. **CROSS-REFERENCE ACE.** Read ACE's C# port of the same function.
   ACE provides naming and structure. Note any differences.
--- a/memory/project_named_decompilation.md
+++ b/memory/project_named_decompilation.md
@ -0,0 +1,84 @@
+# Project: named-retail decompilation foundation
+
+**Agreed 2026-04-25.** Read once per session. Foundation commits:
+`a9a01d8` (corpus), `69d884a` (pdb-extract tool).
+
+## What changed
+
+The retail-client decompilation has gone from "FUN_xxx Ghidra chunks +
+71 hand-mapped functions" to **"18,366 named retail functions + 5,371
+named struct types + verbatim retail headers"**, all committed under
+`docs/research/named-retail/`. The `acclient.pdb` (Sept 2013 EoR build,
+MSVC 7.00) is the source of truth.
+
+## Files at `docs/research/named-retail/`
+
+| File | What | When to use |
+|---|---|---|
+| `acclient_2013_pseudo_c.txt` (62 MB / 1.44 M lines) | Binary Ninja pseudo-C with 99.6% naming. Address-prefixed lines. | **Primary symbol lookup.** Grep by `class::method`. |
+| `acclient.h` (1.7 MB / 70K lines) | IDA-decompiled retail headers. Every struct verbatim. | **Struct field names + offsets.** Grep by struct name. |
+| `acclient.c` (46 MB / 1.3 M lines) | IDA full-binary export. Mixed FUN_/named, but has named struct fields. | **Cross-reference when pseudo-C body is corrupt.** |
+| `symbols.json` (3 MB) | 18,366 entries: `address` + `name` (demangled `Class::Method`) + `mangled` (raw MSVC ABI). | **Programmatic name↔address lookup via jq.** |
+| `types.json` (506 KB) | 5,371 named class/struct records with size + kind. | **Programmatic type-layout queries.** |
+
+## Workflow change — STEP 0 GREP NAMED FIRST
+
+`CLAUDE.md` workflow now starts with **Step 0** before the older
+DECOMPILE step:
+
+```bash
+# Find a function body
+grep -n "CEnchantmentRegistry::EnchantAttribute" \
+     docs/research/named-retail/acclient_2013_pseudo_c.txt
+
+# Find a struct
+grep -n "^struct.*CEnchantmentRegistry" \
+     docs/research/named-retail/acclient.h
+
+# Programmatic
+cat docs/research/named-retail/symbols.json | \
+    jq '.[] | select(.name == "CEnchantmentRegistry::EnchantAttribute")'
+```
+
+The older `docs/research/decompiled/` Ghidra chunks remain a fallback
+for the obfuscated/packed minority that pseudo-C lacks. ~5 second
+grep replaces ~30 minute archaeology for typical lookups.
+
+## Hard rules
+
+1. **Grep `named-retail/` first.** Always. The "Do not guess" rule in
+   CLAUDE.md is upgraded: with the PDB available, guessing is no
+   longer recoverable error — it's negligence.
+2. **Match by name, not raw address.** The PDB build has a ~0xC00
+   byte delta vs the binary that produced our older chunks. Address
+   lookups in `symbols.json` are correct; the older
+   `acclient_function_map.md` had several mid-body addresses (issue
+   #9 sweep corrected them).
+3. **Regenerate JSON sidecars when needed:**
+   ```powershell
+   py tools\pdb-extract\pdb_extract.py refs\acclient.pdb
+   ```
+4. **`refs/` stays gitignored.** It's per-developer download cache;
+   the committed extracts at `docs/research/named-retail/` are what
+   subagents and post-compaction sessions inherit.
+
+## Counts (Sept 2013 EoR build)
+
+- 18,366 named public function symbols (`S_PUB32`, code flag set)
+- 5,371 unique named class/struct types (filtered for non-forward-declared)
+- 1,053 .obj compilation units recorded in DBI
+- Build root preserved in PDB: `d:\ac1_sep13\`
+
+## Why this matters
+
+Closes / unblocks issues #6 (enchantment buffs in vital max — uses
+`CEnchantmentRegistry::EnchantAttribute` at `0x594570`), #7
+(PlayerDescription trailer — uses `CPlayerSystem::Handle_PlayerDescription`
+at `0x5636A0`), plus reduces the cost of every future port from "30
+min archaeology" to "5 sec grep".
+
+## Links
+
+- Plan: `C:/Users/erikn/.claude/plans/ticklish-conjuring-cake.md` (8-phase foundation)
+- Tool: `tools/pdb-extract/` (pure Python, no deps)
+- Reference: `docs/research/named-retail/README.md`
--- a/memory/project_retail_research_index.md
+++ b/memory/project_retail_research_index.md
@ -8,6 +8,13 @@ all Opus-4.7 high-effort).
 subsystem. Open the slice for that subsystem BEFORE writing the first
 line — every slice has pseudocode + C# port sketch + citations.

+> **2026-04-25 update — named-retail foundation.** First stop for any
+> AC-specific question is now `docs/research/named-retail/` (PDB-named
+> Sept 2013 EoR decomp; 18,366 named functions, 5,371 named struct
+> types, 1.4 M lines of pseudo-C). The slices below remain useful for
+> their pseudocode + C# port sketches and the deeper research notes
+> they synthesize. See `memory/project_named_decompilation.md`.
+
 ---

 ## UI layer (`docs/research/retail-ui/`) — 6 slices, ~30,000 words