acdream/docs/research/2026-07-15-retail-spell-catalog-pseudocode.md
Erik 0eab7497c1 fix(spells): load complete retail DAT catalog
Replace the incomplete 3,956-row production CSV with one immutable projection of all 6,266 records in portal.dat. Preserve retail formula targeting, shared Spellbook/MagicRuntime metadata ownership, and fail startup when the required SpellTable is absent.

Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-15 21:17:13 +02:00

3.8 KiB

Retail spell catalog projection

Scope and oracles

The client behavior oracle remains the September 2013 named retail binary:

  • CSpellTable::UnPack @ 0x00597550
  • CSpellTable::InqSpellBase @ 0x005670B0
  • CSpellBase::UnPack @ 0x00597290
  • CSpellBase::InqSpellFormula @ 0x00596EB0
  • CSpellBase::InqTargetType @ 0x00597230
  • CSpellBase::IsUntargeted @ 0x005974A0
  • CSpellBase::InqSpellLevelByRoughHeuristic @ 0x00597260
  • SpellFormula::GetTargetingType @ 0x005BC910
  • SpellComponentTable::GetTargetTypeFromComponentID @ 0x005BBF50

The installed end-of-retail client_portal.dat is the content oracle. Its SpellTable DID 0x0E00000E contains 6,266 spell records. The historical docs/research/data/spells.csv contains only 3,956 records and omits 2,310 spell IDs which the server can legitimately teach a character. It is retained only as a research fixture and is no longer copied into production output.

ACE's SpellBase and SpellComponentsTable.GetSpellWords ports were used as independent schema/decryption and spell-word cross-checks (ACE commit 650c5b75ae909957feaf58db320e46be16502653). DatCollection remains the sole DAT reader in acdream.

Catalog projection

LoadMagicCatalog(dats):
    componentTable = dats.Get(0x0E00000F)
    spellTable = dats.Get(0x0E00000E) or fail startup

    for each (spellId, CSpellBase) in spellTable, preserving every record:
        formula = the decrypted component list, at most eight SCIDs
        formulaTarget = GetTargetingType(formula)

        metadata[spellId] =
            name, description, school, icon, category, flags
            base mana/range, power, economy and component loss
            formula version and decrypted components
            meta-spell type and duration/degrade/portal fields
            caster/target/fizzle effects and recovery fields
            display order, non-component target type and mana modifier
            spell words derived from Herb + Powder + lower-cased Potion
            spellbook level from the rough power-component heuristic
            untargeted = (formulaTarget == 0)

    return one immutable SpellTable shared by Spellbook and MagicRuntime

Learned spell IDs remain server-authoritative. PlayerDescription and live spell events populate Spellbook; the local DAT table supplies presentation and formula metadata for those IDs. Installing metadata refreshes any learned or enchantment state that arrived before Silk's OnLoad, and catalog replacement is rejected after the one startup install.

Formula target type versus server target field

These are deliberately separate concepts:

  • Retail client target compatibility calls CSpellBase::InqTargetType, which decrypts the formula and calls SpellFormula::GetTargetingType.
  • CSpellBase::_non_component_target_type is also preserved from DAT. ACE uses it for authoritative server-side target validation and redirect behavior.

Static disassembly of the matching retail executable confirms the native vtable offset hidden by the pseudo-C. GetTargetingType starts with component slot 4, scans slots 5 through 7 until the formula ends, and maps the final populated component. It does not search for the first component with a known target mapping.

GetTargetingType(components[8]):
    if component slot 4 is absent: return 0
    last = 4
    while last < 7 and components[last + 1] != 0:
        last++
    return GetTargetTypeFromComponentID(components[last])

GetTargetTypeFromComponentID(component):
    0x31..0x38, 0x3C..0x3E, 0xBE => 0x00000010
    0x39                              => 0x00088B8F
    0x3B                              => 0x10010000
    otherwise                         => 0

The old CSV's exported target mask disagrees with this retail formula result for 378 of its 3,956 shared spell IDs. Production therefore never overlays CSV target values onto the DAT catalog.