# DatReaderWriter (DRW) — developer reference

Status as of **2026-04-26**. Source for "what's safe to call, what's
broken, how to upgrade." Update this doc when DRW version bumps or when
we discover a new safety/coverage gap.

## 1. Current version

- **Package:** `Chorizite.DatReaderWriter`
- **Version:** `2.1.7` (NuGet)
- Pinned in `src/AcDream.Core/AcDream.Core.csproj` and
  `src/AcDream.Cli/AcDream.Cli.csproj`. App + Core both transitively
  use that one version.
- Some `tools/*` (WeatherSetupProbe, WeatherEnumerator, SkyObjectInspect,
  dump-keymap, PesChainAudit) `ProjectReference` the vendored repo
  directly instead of NuGet — see §7.

## 2. Source repo

- **Origin:** https://github.com/Chorizite/DatReaderWriter
- **Vendored at:** `references/DatReaderWriter/` (HEAD `63e84b6 "Bump
  version"` — no git tag matching the NuGet version; the repo doesn't
  tag releases, ship version is `GitVersion.yml`-driven).
- **License:** MIT, © 2024 ACClientLib (`references/DatReaderWriter/LICENSE.txt`).
- **Targets:** `net8.0`, `netstandard2.0`, `net48`. We consume from
  `net10.0` projects; the `net8.0` TFM resolves cleanly via netstandard
  forward-compat.
- **Project layout:** `DatReaderWriter/` (library), `…SourceGenerator/`
  (Roslyn generator emits packing code), `…Tests/`, `…Benchmarks/`.

## 3. Types we use

Top-level entry points:

- `DatReaderWriter.DatCollection` — the four-database container
  (`Portal`, `Cell`, `HighRes`, `Local`).
- `DatReaderWriter.Options.DatCollectionOptions` (caching mode, paths).
- `DatCollection.Get<T>(id)` / `Portal.Get<T>(id)` — generic typed read.
- `DatCollection.GetAllIdsOfType<T>()` — used by CLI enumerators.

`DatReaderWriter.DBObjs.*` consumed in `src/`:

- `LandBlock`, `LandBlockInfo`, `EnvCell`, `Region`
- `GfxObj`, `Setup`, `Surface`, `SurfaceTexture` (via SurfaceDecoder)
- `Animation`, `MotionTable`
- `ParticleEmitter`, `PhysicsScript`
- `Environment` (note: collides with `System.Environment`, must be FQN
  in `AcDream.Cli/Program.cs`)
- `Wave` / sound DBObjs (via `DatSoundCache`, `SoundCookbook`)

`DatReaderWriter.Types.*` consumed:

- `AnimationFrame`, `MotionData`
- `Polygon`, `VertexArray`, `Sphere` (BSPQuery, collision)
- `SoundEntry`, `AnimationHook`, `CreateParticleHook`
- `BSPNode` / `BSPTree` / `PhysicsBSPNode` / `CellBSPNode` /
  `DrawingBSPNode` (BSP traversal in `BSPQuery`, `PhysicsDataCache`)

`DatReaderWriter.Enums.*` consumed:

- `MotionCommand` (aliased `DRWMotionCommand` in MotionCommandResolver)
- `Sound` (aliased `DRWSound`)
- `StipplingType`, plus various surface/material flags

## 4. Known broken / missing types

- **`RenderMaterial`** — not implemented. Per upstream README "Known
  Issues." We do not consume it.
- **`LayoutDesc`** — README marks it "supported but structure will need
  cleanup." Treat fields as unstable across versions. We do not depend
  on it directly.
- **`ParticleEmitterInfo`** — DRW exposes the **`ParticleEmitter`**
  DBObj only. There is no separate `ParticleEmitterInfo` type. Our
  `Vfx/VfxModel.cs` comment that mentions "ParticleEmitterInfo dat" is
  conceptual (matches the retail header struct name); the actual lookup
  in `EmitterDescLoader.cs` is `dats.Portal.Get<ParticleEmitter>(id)`
  with a `using DatParticleEmitter = DatReaderWriter.DBObjs.ParticleEmitter;`
  alias because we have our own `ParticleEmitter` runtime class in
  `AcDream.Core.Vfx`.
- **Anything not in `DBObjs/`** — only 5 DBObj files exist
  (`ActionMap`, `DBProperties`, `Iteration`, `LandBlock`,
  `MasterProperty`) at the top level of that folder; the rest (GfxObj,
  Setup, etc.) live elsewhere in the source tree. If a type doesn't
  resolve, search the DRW source — don't assume it's missing.

## 5. Unsafe patterns

**`DatCollection` is NOT thread-safe.** `DatBinReader` holds a buffer
position field per database; concurrent `Get<T>` calls from two threads
corrupt each other and produce silently half-populated payloads
(half-empty `LandBlock.Height[]` in our case). See
`memory/feedback_phase_a1_hotfix_saga.md` rule #2 — Phase A.1 burned
~3 hotfixes mis-diagnosing this as something else.

**Rules:**

1. All `DatCollection.Get<T>()` calls run on the render thread.
2. `LandblockStreamer` runs synchronously (see header comment, lines
   17–32 of `src/AcDream.App/Streaming/LandblockStreamer.cs`). The
   worker-thread design was reverted; the channel-based outbox is
   retained so the moment a thread-safe wrapper exists we can swap
   loaders back to async without touching call sites.
3. Don't claim "DRW is thread-safe per its docs." There are no such
   docs. The README says nothing about concurrency.
4. The async surface area (`*Async` methods) is for I/O, not
   concurrent reads — calling them from two threads is still unsafe.

**Caching:** `DatCollectionOptions` caching mode is `OnDemand`. Don't
flip to `None` without re-benchmarking — terrain mesh build hits the
same `LandBlock` repeatedly.

## 6. How to upgrade

1. Bump `Version="2.1.7"` in `src/AcDream.Core/AcDream.Core.csproj` and
   `src/AcDream.Cli/AcDream.Cli.csproj` (and any `tools/*` using the
   NuGet package, e.g. `TextureDump`).
2. Pull `references/DatReaderWriter/` to the matching commit so the
   vendored copy stays in sync (used by tools that `ProjectReference`
   it directly + as a search target for "what's in DRW today").
3. `dotnet restore && dotnet build` from repo root.
4. Run the full test suite (`dotnet test`). Watch for compile breaks
   on type renames (DRW has historically renamed `DBObj` types between
   minor versions).
5. Smoke-test live: launch against the local ACE server and walk
   Holtburg → Foundry. Watch for half-loaded landblocks (missing
   heights / null surfaces) — that's the canonical "thread-safety
   regression slipped in" signature.

**Risks per area:**

- **DBObj field renames** — DRW source-generates packing; field renames
  break every consumer. Grep before bumping.
- **Enum value reordering** — `MotionCommand`, `Sound`,
  `StipplingType` are enum-by-name in our code, so adds are safe but
  removals will break.
- **BSP type changes** — `BSPNode` family is consumed in `BSPQuery.cs`
  (collision). Any signature change there is a high-risk diff that
  needs the conformance sweep re-run.

## 7. NuGet vs vendored

- **Use NuGet** (`Chorizite.DatReaderWriter` package) for everything
  shipped in `src/` and any `tools/` that doesn't need DRW internals.
  This is the default. Currently `AcDream.Core`, `AcDream.Cli`, and
  `tools/TextureDump`.
- **Vendor (`ProjectReference` to `references/DatReaderWriter/…`)** when:
  - The tool needs a DRW type or method that isn't `public` in the
    NuGet build (some helpers are `internal`).
  - You're prototyping a DRW patch you intend to upstream — edit in
    place, build, validate, then PR upstream.
  - Diagnostic / probe tools (`WeatherSetupProbe`, `WeatherEnumerator`,
    `SkyObjectInspect`, `dump-keymap`, `PesChainAudit`) where iteration
    speed against DRW source matters more than ship discipline.
- **Never mix** within one assembly. A project either NuGet-references
  DRW or `ProjectReference`s the vendored copy. Both produces type-
  identity errors at the boundary.
- **Production code stays on NuGet.** When a vendored tool proves a
  DRW change is needed, upstream it, wait for a release, and bump §6.