fix(spells): load complete retail DAT catalog
Replace the incomplete 3,956-row production CSV with one immutable projection of all 6,266 records in portal.dat. Preserve retail formula targeting, shared Spellbook/MagicRuntime metadata ownership, and fail startup when the required SpellTable is absent. Co-authored-by: OpenAI Codex <codex@openai.com>
This commit is contained in:
Erik
parent
09612f9981
commit
0eab7497c1
19 changed files with 633 additions and 114 deletions
91
docs/research/2026-07-15-retail-spell-catalog-pseudocode.md
Normal file
91
docs/research/2026-07-15-retail-spell-catalog-pseudocode.md
Normal file
|
|
@ -0,0 +1,91 @@
|
|||
# Retail spell catalog projection
|
||||
|
||||
## Scope and oracles
|
||||
|
||||
The client behavior oracle remains the September 2013 named retail binary:
|
||||
|
||||
- `CSpellTable::UnPack @ 0x00597550`
|
||||
- `CSpellTable::InqSpellBase @ 0x005670B0`
|
||||
- `CSpellBase::UnPack @ 0x00597290`
|
||||
- `CSpellBase::InqSpellFormula @ 0x00596EB0`
|
||||
- `CSpellBase::InqTargetType @ 0x00597230`
|
||||
- `CSpellBase::IsUntargeted @ 0x005974A0`
|
||||
- `CSpellBase::InqSpellLevelByRoughHeuristic @ 0x00597260`
|
||||
- `SpellFormula::GetTargetingType @ 0x005BC910`
|
||||
- `SpellComponentTable::GetTargetTypeFromComponentID @ 0x005BBF50`
|
||||
|
||||
The installed end-of-retail `client_portal.dat` is the content oracle. Its
|
||||
SpellTable DID `0x0E00000E` contains 6,266 spell records. The historical
|
||||
`docs/research/data/spells.csv` contains only 3,956 records and omits 2,310
|
||||
spell IDs which the server can legitimately teach a character. It is retained
|
||||
only as a research fixture and is no longer copied into production output.
|
||||
|
||||
ACE's `SpellBase` and `SpellComponentsTable.GetSpellWords` ports were used as
|
||||
independent schema/decryption and spell-word cross-checks (ACE commit
|
||||
`650c5b75ae909957feaf58db320e46be16502653`). `DatCollection` remains the sole
|
||||
DAT reader in acdream.
|
||||
|
||||
## Catalog projection
|
||||
|
||||
```text
|
||||
LoadMagicCatalog(dats):
|
||||
componentTable = dats.Get(0x0E00000F)
|
||||
spellTable = dats.Get(0x0E00000E) or fail startup
|
||||
|
||||
for each (spellId, CSpellBase) in spellTable, preserving every record:
|
||||
formula = the decrypted component list, at most eight SCIDs
|
||||
formulaTarget = GetTargetingType(formula)
|
||||
|
||||
metadata[spellId] =
|
||||
name, description, school, icon, category, flags
|
||||
base mana/range, power, economy and component loss
|
||||
formula version and decrypted components
|
||||
meta-spell type and duration/degrade/portal fields
|
||||
caster/target/fizzle effects and recovery fields
|
||||
display order, non-component target type and mana modifier
|
||||
spell words derived from Herb + Powder + lower-cased Potion
|
||||
spellbook level from the rough power-component heuristic
|
||||
untargeted = (formulaTarget == 0)
|
||||
|
||||
return one immutable SpellTable shared by Spellbook and MagicRuntime
|
||||
```
|
||||
|
||||
Learned spell IDs remain server-authoritative. `PlayerDescription` and live
|
||||
spell events populate `Spellbook`; the local DAT table supplies presentation
|
||||
and formula metadata for those IDs. Installing metadata refreshes any learned
|
||||
or enchantment state that arrived before Silk's `OnLoad`, and catalog
|
||||
replacement is rejected after the one startup install.
|
||||
|
||||
## Formula target type versus server target field
|
||||
|
||||
These are deliberately separate concepts:
|
||||
|
||||
- Retail client target compatibility calls `CSpellBase::InqTargetType`, which
|
||||
decrypts the formula and calls `SpellFormula::GetTargetingType`.
|
||||
- `CSpellBase::_non_component_target_type` is also preserved from DAT. ACE uses
|
||||
it for authoritative server-side target validation and redirect behavior.
|
||||
|
||||
Static disassembly of the matching retail executable confirms the native
|
||||
vtable offset hidden by the pseudo-C. `GetTargetingType` starts with component
|
||||
slot 4, scans slots 5 through 7 until the formula ends, and maps the final
|
||||
populated component. It does not search for the first component with a known
|
||||
target mapping.
|
||||
|
||||
```text
|
||||
GetTargetingType(components[8]):
|
||||
if component slot 4 is absent: return 0
|
||||
last = 4
|
||||
while last < 7 and components[last + 1] != 0:
|
||||
last++
|
||||
return GetTargetTypeFromComponentID(components[last])
|
||||
|
||||
GetTargetTypeFromComponentID(component):
|
||||
0x31..0x38, 0x3C..0x3E, 0xBE => 0x00000010
|
||||
0x39 => 0x00088B8F
|
||||
0x3B => 0x10010000
|
||||
otherwise => 0
|
||||
```
|
||||
|
||||
The old CSV's exported target mask disagrees with this retail formula result
|
||||
for 378 of its 3,956 shared spell IDs. Production therefore never overlays CSV
|
||||
target values onto the DAT catalog.
|
||||
|
|
@ -1,29 +1,29 @@
|
|||
# Game-data tables
|
||||
|
||||
Reference data tables loaded by acdream at runtime or used as research
|
||||
input. Tracked in git so subagents + post-compaction sessions inherit
|
||||
them without round-tripping through `refs/`.
|
||||
Reference data tables used as research input and focused test fixtures.
|
||||
Tracked in git so later sessions inherit them without round-tripping through
|
||||
external repositories. Production spell metadata comes directly from the
|
||||
installed retail DATs through `DatCollection`.
|
||||
|
||||
## Contents
|
||||
|
||||
| File | Loader | Notes |
|
||||
|------|--------|-------|
|
||||
| `spells.csv` | `src/AcDream.Core/Spells/SpellTable.cs` (load via `SpellTable.LoadFromCsv` at `GameWindow.OnLoad`). Copied to `bin/<config>/net10.0/data/spells.csv` via `<None Update>` in `AcDream.App.csproj`. | 3,956 retail spells × 35 columns. Key fields: `Spell ID`, `Name`, `School`, `Family` (buff stacking bucket — issue #6 needs this), `IconId [Hex]`, `Mana`, `Duration`, `IsDebuff`, `IsFellowship`, `Description`. |
|
||||
| `spells.csv` | Legacy `SpellTable.LoadFromCsv` test/tool input only; not copied to production output. | Historical 3,956-row server-side export. The installed end-of-retail SpellTable contains 6,266 records, so this file must never be treated as a complete client catalog. |
|
||||
|
||||
## Format note — `spells.csv`
|
||||
|
||||
RFC 4180 with header row. The `Description` column is quoted with embedded
|
||||
commas; everything else is plain. See `SpellTable` parser for the canonical
|
||||
column ordering.
|
||||
commas; everything else is plain. See `SpellTable` for the legacy parser.
|
||||
|
||||
## Provenance
|
||||
|
||||
`spells.csv` came from a server-side database export of the retail
|
||||
spell table (likely a community-shared dump from circa 2017-2019 with
|
||||
~99% field coverage). Spell IDs match the wire opcodes used by ACE.
|
||||
The `Family` column is the AC retail spell-family enum used for buff
|
||||
stacking.
|
||||
`spells.csv` came from a server-side database export of the retail spell table,
|
||||
likely a community-shared dump from circa 2017–2019. Spell IDs match the wire
|
||||
values used by ACE, but the file omits 2,310 records present in the installed
|
||||
end-of-retail DAT and some exported target masks differ from the retail
|
||||
client's formula-derived result.
|
||||
|
||||
If the file ever needs updating from a fresher source, replace
|
||||
`spells.csv` in this directory; the `<None Update PreserveNewest>`
|
||||
copy will sync it to bin output on next build.
|
||||
Do not update production behavior by replacing this file. The canonical
|
||||
runtime projection and its retail references are documented in
|
||||
`docs/research/2026-07-15-retail-spell-catalog-pseudocode.md`.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue