109 lines
5.3 KiB
Markdown
109 lines
5.3 KiB
Markdown
# PAK v2 resource campaign
|
|
|
|
Status: ACTIVE (2026-08-27)
|
|
|
|
## Objective and release gates
|
|
|
|
Ship one crash-safe prepared-asset format migration that:
|
|
|
|
- reduces the complete installed package from 29,908,271,024 bytes to at
|
|
most 5 GiB;
|
|
- preserves decoded geometry, material metadata, texture bytes, deterministic
|
|
baking, corruption isolation, and random-access loading;
|
|
- does not regress cold or warm world-reveal latency or frame-time percentiles;
|
|
- reduces bake CPU/memory and client retained texture memory where the data
|
|
permits it, without changing the rendered result;
|
|
- gives launcher users a clear one-time update message and progress, while
|
|
retaining the last verified package until the replacement is validated;
|
|
- passes two different worker-count bakes with identical SHA-256, the complete
|
|
installed-DAT bake, content equivalence, performance, solution, Windows CI,
|
|
and release gates.
|
|
|
|
## Measured format-1 baseline
|
|
|
|
The installed package was parsed from its actual TOC, not estimated:
|
|
|
|
| Partition | Physical blobs | Physical bytes |
|
|
|---|---:|---:|
|
|
| GfxObj render meshes | 15,318 | 9,306,115,868 |
|
|
| Setup render meshes | 5,935 | 4,078,139 |
|
|
| EnvCell render meshes | 17,117 | 20,232,745,510 |
|
|
| All collision payloads | 12,938 | 30,392,894 |
|
|
| EnvCell topology | 729,888 | 255,512,622 |
|
|
| TOC | 2,232,170 rows | 53,572,080 |
|
|
|
|
Total: 29,908,271,024 bytes. Render payloads account for approximately
|
|
29.54 GB and 99% of physical payload bytes. The collision and index data are
|
|
not the size problem. Format 1 already aliases duplicate complete EnvCell
|
|
blobs, but each remaining mesh embeds another copy of every decoded RGBA
|
|
texture it uses.
|
|
|
|
Historical complete-bake baseline: 80.5-107.5 seconds, 4.43-4.89 GB peak
|
|
working set, and 3.81-4.21 GB peak private bytes.
|
|
|
|
## Format 2 contract
|
|
|
|
The 64-byte header and 24-byte sorted TOC row remain fixed. Format version is
|
|
2 and bake recipe is 6.
|
|
|
|
1. A new `TexturePayload` key partition (type 8) owns globally shared texture
|
|
byte arrays. Mesh payloads store the texture payload key while retaining
|
|
their own exact dimensions, format, upload metadata, surface identity,
|
|
translucency, culling, and index data.
|
|
2. Texture payload keys are the first 56 bits of SHA-256 under the type-8
|
|
namespace. The writer retains and compares the full bytes for the complete
|
|
bake, making even a truncated-hash collision a loud bake failure rather
|
|
than silent substitution.
|
|
3. Every physical blob is independently encoded. The high bit of the TOC
|
|
length marks compression; the low 31 bits are the stored length. A
|
|
compressed blob contains a four-byte decoded-length prefix followed by
|
|
Brotli. Small or insufficiently compressible blobs remain exactly raw.
|
|
CRC-32 covers stored bytes, then decompression is independently validated.
|
|
4. Random access remains one binary search plus one mmap copy for raw blobs.
|
|
Compressed blobs add decompression only when the writer proved a material
|
|
size win. Texture payloads use a bounded, thread-safe 256 MiB / 2,048-entry
|
|
LRU; concurrently decoded meshes converge on one shared array instance.
|
|
5. Whole-file compression is forbidden. It would destroy random access and
|
|
make a small world reveal depend on unrelated content.
|
|
|
|
## Determinism and publication
|
|
|
|
Asset traversal and mesh serialization remain sorted. A texture is emitted at
|
|
its first deterministic encounter, so its physical order is independent of
|
|
worker completion order. Aliases preserve the source row's exact offset,
|
|
encoded length/flags, and CRC.
|
|
|
|
Recipe 5 to 6 is a mandatory full rebuild. The launcher builds
|
|
`acdream.pak.candidate` beside the active package, validates format, recipe,
|
|
DAT iterations, TOC counts, size, completion protocol, and SHA-256, then uses
|
|
the existing atomic promotion/backup transaction. Cancellation or failure
|
|
keeps the verified format-1 package. No overlay may cross this format change.
|
|
|
|
## Checkpoint evidence
|
|
|
|
The first installed-DAT mixed sample (four GfxObj, three Setup, three EnvCell,
|
|
all corresponding collision/topology payloads) produced 58 keys, 29 globally
|
|
deduplicated texture payloads, and 57 physical blobs. Decoded payload was
|
|
3.5 MiB and stored payload 1.0 MiB (3.62x); output was 1.0 MiB. Eight-worker
|
|
and three-worker bakes had the identical SHA-256
|
|
`78886DFA28A3EDF9368A1E25C9B02A3B69ADC5DFCC01358D64A073B549B5B532`.
|
|
|
|
This sample is a checkpoint only. The size and performance release gates are
|
|
decided by the complete installed-DAT package and connected world-loading
|
|
measurements.
|
|
|
|
## Work ledger
|
|
|
|
- [x] Measure the format-1 package by TOC partition.
|
|
- [x] Implement and unit-test format-2 external texture references, adaptive
|
|
independent compression, corruption handling, bounded sharing, and byte
|
|
determinism.
|
|
- [x] Integrate format-2 accounting and strict validation into the bake.
|
|
- [x] Publish the recipe-6 mandatory full-rebuild launcher migration.
|
|
- [ ] Add launcher disk-space preflight and explicit long-work detail.
|
|
- [ ] Complete installed-DAT equivalence and dual-worker full bakes.
|
|
- [ ] Measure/tune package size, bake CPU/memory, read CPU/allocations, cold
|
|
and warm reveal latency, and frame-time percentiles.
|
|
- [ ] Evaluate source-native BC texture retention only if it remains visually
|
|
exact and does not shift mip-generation work into the reveal frame.
|
|
- [ ] Pass complete tests, Windows CI, merge, push, and release gates.
|