Commit graph

2 commits

Author SHA1 Message Date
Erik
86e0dc4655 fix(pipeline): MP1b review - bake CLI determinism + bounded memory
Adversarially-verified review findings 1 and 10:

(1) The bake pipeline no longer accumulates every decoded ObjectMeshData
in one ConcurrentBag before writing (multi-GB OOM risk on the full
bake), and no longer writes blobs in thread-completion order (which
violated the plan's "bakes must be byte-reproducible run-to-run").
New shape in BakeRunner: build the FULL id list, sort by PakKey, chunk
into 512-id batches; Parallel.ForEach WITHIN each batch; sort each
batch's results by key and AddBlob sequentially; release the batch.
Batches are contiguous key ranges, so the blob region lands in global
key order regardless of thread scheduling, and peak memory is one
batch's output. Side-staged particle-preload meshes drain per batch
into a key-deduped map (first instance wins — per-id extraction output
is deterministic, so instance choice cannot affect bytes) and are
written after all batches, sorted by key, skipping keys already
written.

Program.cs is now a thin arg-parsing shell over the public BakeRunner
so the new dat-gated byte-reproducibility test can drive the REAL
pipeline: tests/AcDream.Bake.Tests (new project, rule 6; registered in
slnx; no Silk.NET in its resolved dependency graph — verified) bakes
the same 9-id mixed fixture twice with DIFFERENT thread counts (8 vs
3 — thread scheduling was the nondeterminism source) and asserts the
two pak files are byte-identical. Ran for real against the dats on
this machine: green.

(10) The isSetup argument for EnvCell extraction now matches at both
call sites (BakeRunner and PakEquivalenceTests both pass false) and is
documented at each: the runtime's own request sites
(WbMeshAdapter.IncrementRefCount / EnsureLoaded) pass isSetup: false
for every MeshRef id including cell-geometry ids. The parameter is
currently dead in MeshExtractor.PrepareMeshData (dispatch is on the
resolved dat type), but two disagreeing call sites were a latent trap.

Header FormatVersion is no longer set by the bake (PakWriter stamps it
per review finding 2, previous commit).
2026-07-05 22:14:31 +02:00
Erik
55f16a0205 test(pipeline): MP1b - live-vs-pak equivalence suite (dat-gated)
PakEquivalenceTests runs MeshExtractor LIVE and bakes the SAME fixture
ids to a temp pak, reads it back via PakReader, and deep-compares every
field via the Task 3 ObjectMeshDataEquality comparator. Since the bake
tool and the live client drive the identical MeshExtractor code (MP1a),
this proves the pak ROUND-TRIP preserves what extraction actually
produces on real content — the serializer's job, not a re-verification
of the extraction algorithm itself (the existing Conformance suite owns
that).

Fixture set (>= the plan's minimums): 10 GfxObjs (3 known-tricky ids
reused from Issue119UpNullGfxObjDumpTests - 0x010002B4, 0x010008A8,
0x010014C3 - plus 7 more from dat order), 3 Setups reused from door/
tower conformance fixtures (0x020019FF, 0x020005D8, 0x020003F2), 5
EnvCells walked from the Holtburg landblock 0xA9B40000's
LandBlockInfo.NumCells range (same idiom as
StipplingSurfaceEquivalenceTests).

Skips cleanly when dats are absent via ContentConformanceDats.
ResolveDatDir(), a deliberate small duplicate of
ConformanceDats.ResolveDatDir()'s exact pattern (env var then
Documents/Asheron's Call fallback) since Content.Tests cannot reference
the AcDream.Core.Tests project. ContentTestDatCollectionAdapter is a
third small copy of the IDatReaderWriter dat-access glue (alongside
AcDream.App's internal original and AcDream.Bake's copy) for the same
layering reason (structure rule 6: tests live in the project matching
the layer under test).

Ran for real against the dats on this machine (not just CI-skip path):
1 test, all fixture ids extracted live with zero failures, baked,
read back, and field-for-field identical. Full solution dotnet test:
4106 tests total (43 Content.Tests + 385 Core.Net.Tests + 425
UI.Abstractions.Tests + 722 App.Tests incl. 2 pre-existing skips + 2531
Core.Tests incl. 2 pre-existing skips) — 0 failures.
2026-07-05 21:33:32 +02:00