PAK v2 resource campaign

Status: ACTIVE (2026-08-27)

Objective and release gates

Ship one crash-safe prepared-asset format migration that:

reduces the complete installed package from 29,908,271,024 bytes to at most 5 GiB;
preserves decoded geometry, material metadata, texture bytes, deterministic baking, corruption isolation, and random-access loading;
does not regress cold or warm world-reveal latency or frame-time percentiles;
reduces bake CPU/memory and client retained texture memory where the data permits it, without changing the rendered result;
gives launcher users a clear one-time update message and progress, while retaining the last verified package until the replacement is validated;
passes two different worker-count bakes with identical SHA-256, the complete installed-DAT bake, content equivalence, performance, solution, Windows CI, and release gates.

Measured format-1 baseline

The installed package was parsed from its actual TOC, not estimated:

Partition	Physical blobs	Physical bytes
GfxObj render meshes	15,318	9,306,115,868
Setup render meshes	5,935	4,078,139
EnvCell render meshes	17,117	20,232,745,510
All collision payloads	12,938	30,392,894
EnvCell topology	729,888	255,512,622
TOC	2,232,170 rows	53,572,080

Total: 29,908,271,024 bytes. Render payloads account for approximately 29.54 GB and 99% of physical payload bytes. The collision and index data are not the size problem. Format 1 already aliases duplicate complete EnvCell blobs, but each remaining mesh embeds another copy of every decoded RGBA texture it uses.

Historical complete-bake baseline: 80.5-107.5 seconds, 4.43-4.89 GB peak working set, and 3.81-4.21 GB peak private bytes.

Format 2 contract

The 64-byte header and 24-byte sorted TOC row remain fixed. Format version is 2 and bake recipe is 6.

A new TexturePayload key partition (type 8) owns globally shared texture byte arrays. Mesh payloads store the texture payload key while retaining their own exact dimensions, format, upload metadata, surface identity, translucency, culling, and index data.
Texture payload keys are the first 56 bits of SHA-256 under the type-8 namespace. The writer retains and compares the full bytes for the complete bake, making even a truncated-hash collision a loud bake failure rather than silent substitution.
Every physical blob is independently encoded. The high bit of the TOC length marks compression; the low 31 bits are the stored length. A compressed blob contains a four-byte decoded-length prefix followed by Brotli. Small or insufficiently compressible blobs remain exactly raw. CRC-32 covers stored bytes, then decompression is independently validated.
Random access remains one binary search plus one mmap copy for raw blobs. Compressed blobs add decompression only when the writer proved a material size win. Texture payloads use a bounded, thread-safe 256 MiB / 2,048-entry LRU; concurrently decoded meshes converge on one shared array instance.
Whole-file compression is forbidden. It would destroy random access and make a small world reveal depend on unrelated content.

Determinism and publication

Asset traversal and mesh serialization remain sorted. A texture is emitted at its first deterministic encounter, so its physical order is independent of worker completion order. Aliases preserve the source row's exact offset, encoded length/flags, and CRC.

Recipe 5 to 6 is a mandatory full rebuild. The launcher builds acdream.pak.candidate beside the active package, validates format, recipe, DAT iterations, TOC counts, size, completion protocol, and SHA-256, then uses the existing atomic promotion/backup transaction. Cancellation or failure keeps the verified format-1 package. No overlay may cross this format change.

Checkpoint evidence

The first installed-DAT mixed sample (four GfxObj, three Setup, three EnvCell, all corresponding collision/topology payloads) produced 58 keys, 29 globally deduplicated texture payloads, and 57 physical blobs. Decoded payload was 3.5 MiB and stored payload 1.0 MiB (3.62x); output was 1.0 MiB. Eight-worker and three-worker bakes had the identical SHA-256 78886DFA28A3EDF9368A1E25C9B02A3B69ADC5DFCC01358D64A073B549B5B532.

This sample is a checkpoint only. The size and performance release gates are decided by the complete installed-DAT package and connected world-loading measurements.

Work ledger

Measure the format-1 package by TOC partition.
Implement and unit-test format-2 external texture references, adaptive independent compression, corruption handling, bounded sharing, and byte determinism.
Integrate format-2 accounting and strict validation into the bake.
Publish the recipe-6 mandatory full-rebuild launcher migration.
Add launcher disk-space preflight and explicit long-work detail.
Complete installed-DAT equivalence and dual-worker full bakes.
Measure/tune package size, bake CPU/memory, read CPU/allocations, cold and warm reveal latency, and frame-time percentiles.
Evaluate source-native BC texture retention only if it remains visually exact and does not shift mip-generation work into the reveal frame.
Pass complete tests, Windows CI, merge, push, and release gates.

5.3 KiB Raw Blame History