Commit graph

2 commits

Author SHA1 Message Date
Erik
859cf5ec02 refactor(pipeline): MP1b review - unify DatCollectionAdapter + TOC/log test gaps
Adversarially-verified review findings 7 and 8:

(7) The DatCollection->IDatReaderWriter adapter existed as THREE
near-identical copies (App-internal original, Bake's copy, Content.
Tests' copy) — a structure where adapter drift is exactly what the
live-vs-pak equivalence suite cannot detect (both sides would only
drift together if they shared one implementation). Now ONE public
AcDream.Content.DatCollectionAdapter next to IDatReaderWriter (GL-free
home established in MP1a), carrying App's FULL behavior including the
[dat-miss] TryGet tripwire log (which now also covers the bake tool
and the equivalence suite) and the caching/locking. All three copies
deleted; WbMeshAdapter (App), BakeRunner (Bake), and
PakEquivalenceTests (Content.Tests) resolve the shared class.
Iteration properties return the REAL dat iterations — the App copy's
hardcoded 0 was a stub nothing read; the unification intentionally
keeps truth (noted in the doc comment). Verified post-move: no
Silk.NET anywhere in Content / Bake / Content.Tests / Bake.Tests
resolved dependency graphs.

(8) Two test gaps closed in PakRoundTripTests: (a) direct on-disk TOC
sortedness — blobs added in DESCENDING key order, then the raw file
bytes parsed (not through the reader) and every TOC entry asserted
strictly ascending; (b) corrupt-blob logging — five repeated reads
through both public paths (TryReadObjectMeshData + ContainsKey) with
stderr captured, asserting exactly ONE [pak-corrupt] line for the
victim key.

Full suite: 4120 tests, 0 failures (Content.Tests 56, Bake.Tests 1,
plus the pre-existing 4 skips).
2026-07-05 22:18:30 +02:00
Erik
86e0dc4655 fix(pipeline): MP1b review - bake CLI determinism + bounded memory
Adversarially-verified review findings 1 and 10:

(1) The bake pipeline no longer accumulates every decoded ObjectMeshData
in one ConcurrentBag before writing (multi-GB OOM risk on the full
bake), and no longer writes blobs in thread-completion order (which
violated the plan's "bakes must be byte-reproducible run-to-run").
New shape in BakeRunner: build the FULL id list, sort by PakKey, chunk
into 512-id batches; Parallel.ForEach WITHIN each batch; sort each
batch's results by key and AddBlob sequentially; release the batch.
Batches are contiguous key ranges, so the blob region lands in global
key order regardless of thread scheduling, and peak memory is one
batch's output. Side-staged particle-preload meshes drain per batch
into a key-deduped map (first instance wins — per-id extraction output
is deterministic, so instance choice cannot affect bytes) and are
written after all batches, sorted by key, skipping keys already
written.

Program.cs is now a thin arg-parsing shell over the public BakeRunner
so the new dat-gated byte-reproducibility test can drive the REAL
pipeline: tests/AcDream.Bake.Tests (new project, rule 6; registered in
slnx; no Silk.NET in its resolved dependency graph — verified) bakes
the same 9-id mixed fixture twice with DIFFERENT thread counts (8 vs
3 — thread scheduling was the nondeterminism source) and asserts the
two pak files are byte-identical. Ran for real against the dats on
this machine: green.

(10) The isSetup argument for EnvCell extraction now matches at both
call sites (BakeRunner and PakEquivalenceTests both pass false) and is
documented at each: the runtime's own request sites
(WbMeshAdapter.IncrementRefCount / EnsureLoaded) pass isSetup: false
for every MeshRef id including cell-geometry ids. The parameter is
currently dead in MeshExtractor.PrepareMeshData (dispatch is on the
resolved dat type), but two disagreeing call sites were a latent trap.

Header FormatVersion is no longer set by the bake (PakWriter stamps it
per review finding 2, previous commit).
2026-07-05 22:14:31 +02:00