acdream/docs/research/2026-07-15-retail-spell-catalog-pseudocode.md
Erik 0eab7497c1 fix(spells): load complete retail DAT catalog
Replace the incomplete 3,956-row production CSV with one immutable projection of all 6,266 records in portal.dat. Preserve retail formula targeting, shared Spellbook/MagicRuntime metadata ownership, and fail startup when the required SpellTable is absent.

Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-15 21:17:13 +02:00

91 lines
3.8 KiB
Markdown

# Retail spell catalog projection
## Scope and oracles
The client behavior oracle remains the September 2013 named retail binary:
- `CSpellTable::UnPack @ 0x00597550`
- `CSpellTable::InqSpellBase @ 0x005670B0`
- `CSpellBase::UnPack @ 0x00597290`
- `CSpellBase::InqSpellFormula @ 0x00596EB0`
- `CSpellBase::InqTargetType @ 0x00597230`
- `CSpellBase::IsUntargeted @ 0x005974A0`
- `CSpellBase::InqSpellLevelByRoughHeuristic @ 0x00597260`
- `SpellFormula::GetTargetingType @ 0x005BC910`
- `SpellComponentTable::GetTargetTypeFromComponentID @ 0x005BBF50`
The installed end-of-retail `client_portal.dat` is the content oracle. Its
SpellTable DID `0x0E00000E` contains 6,266 spell records. The historical
`docs/research/data/spells.csv` contains only 3,956 records and omits 2,310
spell IDs which the server can legitimately teach a character. It is retained
only as a research fixture and is no longer copied into production output.
ACE's `SpellBase` and `SpellComponentsTable.GetSpellWords` ports were used as
independent schema/decryption and spell-word cross-checks (ACE commit
`650c5b75ae909957feaf58db320e46be16502653`). `DatCollection` remains the sole
DAT reader in acdream.
## Catalog projection
```text
LoadMagicCatalog(dats):
componentTable = dats.Get(0x0E00000F)
spellTable = dats.Get(0x0E00000E) or fail startup
for each (spellId, CSpellBase) in spellTable, preserving every record:
formula = the decrypted component list, at most eight SCIDs
formulaTarget = GetTargetingType(formula)
metadata[spellId] =
name, description, school, icon, category, flags
base mana/range, power, economy and component loss
formula version and decrypted components
meta-spell type and duration/degrade/portal fields
caster/target/fizzle effects and recovery fields
display order, non-component target type and mana modifier
spell words derived from Herb + Powder + lower-cased Potion
spellbook level from the rough power-component heuristic
untargeted = (formulaTarget == 0)
return one immutable SpellTable shared by Spellbook and MagicRuntime
```
Learned spell IDs remain server-authoritative. `PlayerDescription` and live
spell events populate `Spellbook`; the local DAT table supplies presentation
and formula metadata for those IDs. Installing metadata refreshes any learned
or enchantment state that arrived before Silk's `OnLoad`, and catalog
replacement is rejected after the one startup install.
## Formula target type versus server target field
These are deliberately separate concepts:
- Retail client target compatibility calls `CSpellBase::InqTargetType`, which
decrypts the formula and calls `SpellFormula::GetTargetingType`.
- `CSpellBase::_non_component_target_type` is also preserved from DAT. ACE uses
it for authoritative server-side target validation and redirect behavior.
Static disassembly of the matching retail executable confirms the native
vtable offset hidden by the pseudo-C. `GetTargetingType` starts with component
slot 4, scans slots 5 through 7 until the formula ends, and maps the final
populated component. It does not search for the first component with a known
target mapping.
```text
GetTargetingType(components[8]):
if component slot 4 is absent: return 0
last = 4
while last < 7 and components[last + 1] != 0:
last++
return GetTargetTypeFromComponentID(components[last])
GetTargetTypeFromComponentID(component):
0x31..0x38, 0x3C..0x3E, 0xBE => 0x00000010
0x39 => 0x00088B8F
0x3B => 0x10010000
otherwise => 0
```
The old CSV's exported target mask disagrees with this retail formula result
for 378 of its 3,956 shared spell IDs. Production therefore never overlays CSV
target values onto the DAT catalog.