Adversarially-verified review findings 1 and 10:
(1) The bake pipeline no longer accumulates every decoded ObjectMeshData
in one ConcurrentBag before writing (multi-GB OOM risk on the full
bake), and no longer writes blobs in thread-completion order (which
violated the plan's "bakes must be byte-reproducible run-to-run").
New shape in BakeRunner: build the FULL id list, sort by PakKey, chunk
into 512-id batches; Parallel.ForEach WITHIN each batch; sort each
batch's results by key and AddBlob sequentially; release the batch.
Batches are contiguous key ranges, so the blob region lands in global
key order regardless of thread scheduling, and peak memory is one
batch's output. Side-staged particle-preload meshes drain per batch
into a key-deduped map (first instance wins — per-id extraction output
is deterministic, so instance choice cannot affect bytes) and are
written after all batches, sorted by key, skipping keys already
written.
Program.cs is now a thin arg-parsing shell over the public BakeRunner
so the new dat-gated byte-reproducibility test can drive the REAL
pipeline: tests/AcDream.Bake.Tests (new project, rule 6; registered in
slnx; no Silk.NET in its resolved dependency graph — verified) bakes
the same 9-id mixed fixture twice with DIFFERENT thread counts (8 vs
3 — thread scheduling was the nondeterminism source) and asserts the
two pak files are byte-identical. Ran for real against the dats on
this machine: green.
(10) The isSetup argument for EnvCell extraction now matches at both
call sites (BakeRunner and PakEquivalenceTests both pass false) and is
documented at each: the runtime's own request sites
(WbMeshAdapter.IncrementRefCount / EnsureLoaded) pass isSetup: false
for every MeshRef id including cell-geometry ids. The parameter is
currently dead in MeshExtractor.PrepareMeshData (dispatch is on the
resolved dat type), but two disagreeing call sites were a latent trap.
Header FormatVersion is no longer set by the bake (PakWriter stamps it
per review finding 2, previous commit).
New offline console tool driving AcDream.Content.MeshExtractor (the SAME
extraction code the live client runs, per MP1a) to produce a versioned
pak file. Arguments: --dat-dir (required), --out (default acdream.pak
next to the dats), --ids/--landblocks (dev filters), --threads (default
Environment.ProcessorCount).
Enumeration mirrors existing idioms rather than inventing new ones:
GfxObj/Setup ids via DatCollection.GetAllIdsOfType<T>() (src/AcDream.Cli/
Program.cs); EnvCell ids by walking dats.Cell.Tree for LandBlockInfo
entries (low 16 bits == 0xFFFE) and then the firstCellId+NumCells range
per landblock (the same idiom as GameWindow.BuildPhysicsDatBundle /
BuildInteriorEntitiesForStreaming). GetAllIdsOfType<T>() does not cover
cell-dat range-based types, hence the manual walk.
BakeDatCollectionAdapter is a from-scratch copy of AcDream.App.Rendering.
Wb.DatCollectionAdapter (which is `internal` to AcDream.App and, more
importantly, referencing AcDream.App would drag Silk.NET/GL into the
bake tool — a hard violation of "no Silk.NET anywhere"). It's plain dat-
access glue, not an AC-specific algorithm, so a small duplicate is the
right call over inventing a shared-but-App-rooted package.
Particle-preload GfxObjs MeshExtractor side-stages mid-extraction
(sideStagedSink, thread-safe ConcurrentQueue per MP1a's documented
contract) are deduped against the primary GfxObj set and against each
other before being written as their own GfxObjMesh entries.
ConsoleErrorLogger is a minimal hand-rolled ILogger (stderr, Warning+)
rather than NullLogger — MeshExtractor's LogError/LogWarning calls on
malformed dat entries must stay visible per the project's "logger
injection for silent catches" lesson.
Progress line every 5s (baked/total, failures, elapsed, ETA); a
malformed dat entry is caught per-id and counted as a failure, never
fatal — matches the runtime's own per-id try/catch behavior. Failures
report is printed at the end, capped at 200 lines.
Smoke-tested against the real dats on this machine: --ids
0x01000001,0x02000001 baked 1 GfxObj (40 vertices) + 1 Setup (34 parts)
in 1.4s with 0 failures; a follow-up run adding EnvCell 0xA9B40100
(Holtburg's first interior cell) baked all three asset types
successfully. PakReader opened both scratch paks and read back
deep-correct ObjectMeshData; scratch files deleted after verification.
*.pak added to .gitignore in this commit per the plan.
