Commit graph

16 commits

Author SHA1 Message Date
Erik
15ed57a1e7 fix(rendering): carry retail SetSurface state to detail draws
Resolve exact SetSurface blend, alpha-test, and fog state once during extraction and preserve it through recipe-10 prepared payloads, Wb/EnvCell command data, Vulkan pipelines, push constants, and both ordinary/atmospheric one-pass shaders. Preserve AP-240 Wb pure-Clip immediate opaque/A2C while EnvCell uses retail premultiplied Clip; detail-off routing remains unchanged. Correct AP-232 and register the remaining detail-off state divergence.

Mutation first failures (all restored):
- raw Add->SRCALPHA/ONE: WalkStaticStreamPopulatorTests.ImmediateBuildingDetail_UsesExactResolvedSetSurfaceState line 1278, expected wb-mesh-raw-additive-1x.
- inverse-add->alpha-add: same test line 1278, expected wb-mesh-inverse-additive-1x.
- remove Env inverse: EnvCellAlphaDrawSourceTests.DetailOn_EveryEnvCellFamilyDrawsOnceInPlaceWithAuthoredOpacity line 132, expected envcell-inverse.
- raw IsAdditive precedence: same test line 132, Translucent|Clip|Additive expected envcell-alpha.
- Wb paletted ParamB=0: OrderPreservingSubmitterTests line 333, expected 0.392156869.
- Wb DDS ParamB=0.05: same test line 333, expected 0.784313738.
- disable Alpha+Clip test: WalkStaticStreamPopulatorTests line 1286, expected 0.784313738.
- always fog raw Add: same test line 1287, expected no-fog true.
- disable fog non-Add: same test line 1287, expected no-fog false.
- X=a*qA: RetailDetailTextureContractTests line 261, shared squared-alpha substring absent.
- X includes base alpha: same test line 262, forbidden baseTexel.a present.
- CLIP uses 0.05: EnvCellAlphaDrawSourceTests.ClipShaders_UseGreaterEqualForThePerRangeReference line 367.
- second detail draw: EnvCell detail-on line 131, collection contained 2 draws.
- straight-alpha substitute: Wb immediate line 1278, expected wb-mesh-additive-1x.
- omit ordered detail arm: OrderPreservingSubmitterTests line 318, expected (77,3.5), got (0,0).
- omit atmospheric combine: RetailDetailTextureContractTests line 260, shared include absent.
- drop serialized opacity: ObjectMeshDataSerializerTests line 292, expected opacity bits, got 1.0.
- stale detail arm: atmospheric adjacency line 402, expected slot 0, got 77.
- per-frame surface map: EnvCell warmed allocation line 285, expected 0 B, got 204800 B.
2026-09-05 02:03:13 +02:00
Erik
75664805f8 feat(rendering): port retail one-pass detail material
Replace the building and EnvCell detail replay with retail's exact single-pass stage result, including authored surface opacity, squared detail alpha, final-alpha clipping, and the original subset pipeline/order. Arm the ordered walk command in place to close #471, delete the replay pipelines/shaders, and advance prepared content to recipe 9.

Mutation witnesses (each restored before commit):
- X=a*qA: RetailDetailTextureContractTests.BothShaderFamiliesUseTheSharedOnePassSourceAndDebugPrecedesDetailSample line 174, missing materialAlpha * detail.a * detail.a.
- X*=base alpha: same test line 175, forbidden baseTexel.a found.
- CLIP against base alpha: EnvCellAlphaDrawSourceTests.ClipShaders_UseGreaterEqualForThePerRangeReference line 260, final-X conditional missing.
- second detail draw: EnvCellAlphaDrawSourceTests.DetailOn_EveryEnvCellFamilyDrawsOnceInPlaceWithAuthoredOpacity line 105, Assert.Single saw 2 MDI calls.
- straight-alpha substitution: WalkStaticStreamPopulatorTests.ImmediateBuildingDetail_RetainsOriginalFramebufferFamily line 1244, Additive first failed (only wb-mesh-alpha-1x recorded; InvAlpha also failed).
- omit ordered arm: OrderPreservingSubmitterTests.PrepareThenDraw_OrdinaryBuildingClipBuildingOrdinary_ArmsOnePassInPlace line 305, expected (77,3.5), got (0,0).
- omit atmospheric combine: RetailDetailTextureContractTests.BothShaderFamiliesUseTheSharedOnePassSourceAndDebugPrecedesDetailSample line 173, atmospheric shared include missing.
- drop serialized opacity: ObjectMeshDataSerializerTests.SurfaceOpacity_RoundTripsBitExactlyAndDeterministically line 288, first reported 0.5 bits 1056964608 vs 1065353216.
- stale detail arm: ordered adjacency test line 307, expected following ordinary (0,0), got (77,3.5).
- per-frame surface map: EnvCellAlphaDrawSourceTests.ProductionWholeLeaf_WarmedScanSubmitRhiAndFilteredReplayDoNotAllocate line 178, expected 0 B, got 147456 B.

Verification before commit: shader compiler 23/23; focused App 213/213; Content 75/75; Core Wb 10/10; launcher migration 6/6; Release solution build 0 warnings / 0 errors; git diff --check clean.
2026-09-05 02:03:13 +02:00
Erik
acf172469e feat(content): S1 exact CellStruct surface-index construction, recipe 8
Campaign OVERHAUL S1 chunk A. Retail's D3DPolyRender::ConstructMesh
@0x0059DFA0 is ported as one pure Core descriptor plus the Content
extraction that consumes it:

- side candidates come only from sides_type (0/1/2); NoPos/NoNeg mean
  UV-array absence only and never suppress a side (CPolygon::UnPack
  @0x00538650);
- ST_DOUBLE's second copy is reversed with a negative normal; ST_BOTH's
  negative side has a negative normal and forward fan order (reverse is on
  the copy ordinal, not the side ordinal);
- an absent UV-index array is UV index 0 (ConstructMesh @0x0059E691
  xor ebx,ebx, arbitrated on the PDB-paired binary); copyVert @0x0059C080
  zeroes coordinates only for a negative or out-of-range index or a vertex
  without UVs, never by clamping to slot 0;
- the subset owner is the source surface-array index, emitted in ascending
  slot order with retail's per-slot mask (2 > 8 > 4 precedence, positive
  surface OR on signed stippling > 0);
- built-EnvCell admission is (Surface.Type & (BASE1_IMAGE|BASE1_CLIPMAP))
  != 0 after surface resolution (DrawEnvCell @0x0059F170 -> DrawMesh
  @0x0059D4A0 arg4=1); untextured slots are constructed but not emitted;
- cell batches carry SourceSurfaceIndex, RetailSurfaceMask, RawSurfaceType,
  IsCellShell, and fixed clockwise raster cull (RenderMeshSubset
  @0x0059CA10); authored sides_type is no longer stored as GPU cull.

Prepared-mesh serializer gains the four fields; bake recipe 7 -> 8 with a
FullRebuild migration; pak format stays 2 (pinned). Ordinary GfxObj
extraction is unchanged. AP-234's register row and CellMesh unification
land in chunk B.

Core: 32 descriptor tests. Content: 170/170. Bake: 18/18. Launcher.Core:
365/365 (Lane!=Linux). Solution Release build 0 warnings / 0 errors.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 18:40:33 +02:00
Erik
b3b7d922f1 checkpoint(render): preserve pre-overhaul investigation state 2026-09-01 18:04:24 +02:00
Erik
45ba42a3a7 fix(ci): make launcher space guidance culture invariant
All checks were successful
CI / linux-portable (push) Successful in 3m23s
CI / windows-gate (push) Successful in 5m33s
CI / release (push) Successful in 2m8s
2026-08-27 20:40:50 +02:00
Erik
0dd966f3a0 Optimize prepared asset package v2 2026-08-27 20:09:09 +02:00
Erik
0c699240e0 fix(ci): make release gates portable and deterministic
All checks were successful
CI / linux-portable (push) Successful in 3m46s
CI / windows-gate (push) Successful in 6m21s
CI / release (push) Successful in 2m15s
2026-08-25 19:38:10 +02:00
Erik
af9327a17b feat(launcher): stabilize prepared content updates
Some checks failed
CI / linux-portable (push) Failing after 3m12s
CI / windows-gate (push) Failing after 6m35s
CI / release (push) Has been skipped
2026-08-25 19:17:13 +02:00
Erik
517d17b4b3 fix #426: extract solid-colour (NO_POS_UVS) faces; skip untextured subsets only on building shells and cells like retail
The Holtburg windmill axle (GfxObj 0x010010CE, 8 polygons, all
Stippling.NoPos + SurfaceType.Base1Solid) extracted to a 0-vertex mesh.
NoPos ("NO_POS_UVS", acclient.h:7380-7388) means "this side has no
texture coordinates" — true of every solid-colour polygon, since
nothing samples them — not "there is no positive face". Extraction read
it as the latter and dropped the polygon entirely, client-wide, for
every untextured polygon on every object.

Retail's D3DPolyRender::DrawMesh (@0x0059d4a0, named-retail decomp
~line 426048) draws an untextured subset on an ordinary object exactly
like a textured one; the only retail cases that skip an untextured
subset are a building shell (RenderDeviceD3D::DrawBuilding @0x0059f2a0
sets ObjBuildingOrBuildingPart=1) or an EnvCell interior
(RenderDeviceD3D::DrawEnvCell @0x0059f170, arg4=1). The #119
investigation's "retail's skipNoTexture never draws them either"
conclusion was itself wrong as a general rule.

- MeshExtractor.PrepareGfxObjMeshData / GfxObjMesh.Build: emit the
  positive side whenever PosSurface is a valid index, regardless of
  NoPos; the existing UV-index-0 fallback already produces zero
  texcoords for a NoPos polygon with no UVs on the wire.
- RetailUntexturedSurfacePolicy.IsUntextured(SurfaceType): the one
  place that answers "is this surface textured"
  ((type & (Base1Image|Base1ClipMap)) == 0), replacing the old
  `isSolid = NoPos || Base1Solid` (which also mis-classified a NEG-side
  batch by the POS-side's NoPos flag).
- RetailUntexturedSubsetPolicy.Draws(isBuildingShell, isUntextured):
  the shared draw-time gate wired into WbDrawDispatcher.ClassifyBatches,
  .PackedOracle.ClassifyPackedBatches, and
  .DirectionalShadows.AddDirectionalShadowBatches — one predicate so the
  three walks cannot drift (Campaign VM VM6 lesson).
- CellMesh.cs / MeshExtractor.PrepareCellStructMeshData deliberately
  KEEP their NoPos-gated skip for cell-wall geometry — retail's
  DrawEnvCell really does skip untextured subsets there; register row
  AP-234 documents the NoPos-vs-Surface.Type approximation.
- PakFormat.CurrentBakeToolVersion 4->5 (LauncherInstallRecordStore in
  lockstep): a pak baked by an older tool is missing every untextured
  face. No bake was run as part of this commit.

Also fixed: WorldBuilder's own upstream ObjectMeshManager.cs has the
identical NoPos bug (ObjectMeshManager.cs:959,984) — our port had
faithfully carried it over, and our own conformance test
(Build_NoPosFlag_OnlyEmitsNegSide) asserted the bug as correct WB
conformance. Renamed/reworded to Build_NoPosFlag_EmitsBothPosAndNegSide
with a citation for why retail decomp overrides WB here.

Issue119UpNullGfxObjDumpTests re-run against the installed DAT:
#119's own two objects (0x010002B4 9/9 polys, 0x010008A8 1/1 poly) now
gate DRAWS on every polygon instead of extracting to nothing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 11:20:24 +02:00
Erik
7037681a1f fix: state the verification cache's real limit instead of a claim that is false on ZFS
All checks were successful
CI / linux-portable (push) Successful in 2m56s
CI / windows-gate (push) Successful in 4m53s
CI / release (push) Successful in 2m5s
Run 174's Linux job failed on
ASilentlyCorruptedPackageOfTheSameSizeStillFailsAndDropsTheCache. It passed in
isolation on that same machine, and passed under full-suite load there too, so
it looked like a flake. It is not.

Measured on the runner:

    same-mtime collisions: 141 / 200
    fs type: zfs

Its /tmp is ZFS, whose timestamp granularity is coarse enough that a same-size
rewrite usually lands on the SAME last-write time. So the startup fast path —
size plus write time — cannot see that modification, and the test was right to
fail. LU1's commit message claimed "truncating or touching the package still
blocks launch"; on a coarse-timestamp filesystem the second half of that is
false. NTFS's 100 ns resolution is why it never showed on Windows.

Rather than relax the test until it passes, the contract is now stated as two
facts that are true everywhere instead of one that is not:

- A same-size corruption whose write time moves is caught at startup. The test
  moves the timestamp explicitly instead of trusting the clock, so it asserts
  the mechanism rather than the filesystem's resolution.
- A corruption preserving BOTH size and write time is NOT caught at startup and
  IS caught by a forced full verification — which is exactly what the
  launcher's Verify files button runs. New test, so the escape hatch is
  covered rather than merely mentioned.

PreparedAssetVerificationCache now documents the limit with the measurement, so
the next reader does not have to rediscover it from a red pipeline.

Verified on Windows (11 passed) and five consecutive runs on the ZFS runner
itself (11 passed each). Full solution 14,375 passed, 0 failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 22:03:52 +02:00
Erik
00d1278228 feat(launcher): LU1 — stop hashing 28 GB before the launcher window appears
Measured on the user's machine: %LOCALAPPDATA%\acdream\pak\acdream.pak is
29,908,271,024 bytes and SHA-256 over it takes 24.1 s at 1.16 GB/s. App
.OnFrameworkInitializationCompleted ran exactly that hash synchronously,
before constructing the window, and the digest came back identical to the one
install.json already recorded. So the launcher took roughly half a minute to
appear in order to re-confirm a fact that had not changed. A friend does not
see it only because they have no package installed yet — verification
short-circuits at "nothing installed" — so it would hit them the moment
first-run setup finished.

Startup now checks the cheap facts (size, last-write time) and skips only the
hash, and only when a previous FULL hash of that same file agreed with the
install record. Everything that should hash still does: install, update, the
crash-recovery backup path, and a new explicit "Verify files" button.

The remembered fact lives in a SIDECAR (install.verification.json), not as a
new field on the install record: LauncherInstallRecordStore reads install.json
with JsonUnmappedMemberHandling.Disallow, so a new property there would make an
older launcher build reject the record outright and demand a fresh ~28 GB bake
after a rollback. An unknown sidecar is simply ignored by builds that predate
it. The cache type never throws — it sits in front of a guarantee, so every
failure mode (missing, corrupt, unknown schema, unwritable) degrades to
"hash it again" rather than to a failed launch.

Two subtleties worth keeping:
- The write time is re-read after the hash and the entry is only written when
  it is unchanged. A writer racing a multi-second hash would otherwise be
  remembered under the OLD timestamp, and the next startup would trust a
  digest that never covered those bytes.
- A hash that disagrees with the record invalidates the entry, so a stale
  "verified" fact cannot outlive the evidence that produced it.

Tests: PreparedAssetVerificationCacheTests (10) counts hash invocations through
the store's injectable hasher and covers second-startup skip, forced full
verification, touched package, same-size silent corruption, resize, a cache
digest that disagrees with the record, three unreadable-cache shapes, and
backup recovery still hashing. Plus two LauncherWindowViewModel tests for the
Verify files command. Launcher.Core 335 passed, Launcher 57 passed.

Note for the first run after this ships: the very first startup still pays one
full hash to learn the digest for the installed file, and every startup after
that is instant.

Campaign LU slice LU1. Plan: docs/plans/2026-08-19-launcher-usability-campaign.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 18:40:34 +02:00
Erik
c155db74d1 test: introduce Lane=Timing for load-sensitive tests, and stop chasing them individually
All checks were successful
CI / linux-portable (push) Successful in 3m27s
CI / windows-gate (push) Successful in 5m34s
CI / release (push) Successful in 1m54s
Four separate fixes each surfaced a different member of the same family, and
one of them (serializing Core.Net.Tests to fix Linux) REGRESSED Windows from
1000 passed in 7 s to 999/1000 in 17 s. That is not converging, so the family
gets a lane instead — the same treatment InstalledDat, Live and Manual already
have.

Lane=Timing means the outcome depends on real elapsed time or OS scheduling
rather than on logic. Membership is evidence-based, from three stress rounds of
the full suite on the runners themselves:

  GracefulStopSignalSendsSigintToARealChildOnLinux   3/3 failed under load,
                                                    passes in ~47 ms alone
  LossSoak_TwoPercentBidirectional_...              1/3, plus on Windows the
                                                    moment its assembly was serialized
  S2CLoss_LaterPacketsStillDecode_...               1/3
  PausedSelector_SeededDroppedServerReady_...       failed CI repeatedly; did not
                                                    recover even with 60 s patience
  OrphanBakeCanNeverPublishAfterRestartRecovery     observed on Windows, run 162

Nothing is weakened or deleted: 997 Core.Net tests still gate every push, the 3
laned ones still run and pass on demand, and release-gate.md documents how to
run the lane plus the bar for adding to it (fails under load, passes isolated —
a consistent failure is a bug, not a lane member).

Also removes the ad-hoc Core.Net parallelism special-case from the Linux job,
which this supersedes.
2026-08-19 15:29:10 +02:00
Erik
79a4489e03 test: replace final fixed-delay oracles 2026-08-18 13:12:26 +02:00
Erik
208a70ac83 fix(launcher): guard orphan bake publication 2026-08-14 21:02:27 +02:00
Erik
3f68895120 fix(launcher): harden installer transactions 2026-08-14 20:36:11 +02:00
Erik
ff6ebb6a6a feat(launcher): add verified first-run installer 2026-08-14 20:06:37 +02:00