feat(core): S1 one CellStruct interpretation, installed-DAT scan, retire AP-234

Campaign OVERHAUL S1 chunk B.

- CellMesh.Build's render role is deleted; CellMesh.HasDrawableGeometry
  is the exact predicate (any sides_type candidate whose resolved surface
  passes the built-EnvCell (Surface.Type & 6) != 0 test), and the
  streaming build job consumes it. Core and Content no longer carry two
  CellStruct interpretations.
- Installed-DAT scan over the complete Environment corpus (5,346
  landblocks, 729,888 cells, 8,601,560 polygons, 8,608,746 candidates):
  3,197 old-rule admissions were untextured surfaces the NoPos proxy let
  through; zero new-only; zero unexplained; zero unknown sides values;
  counts pinned as goldens against the contract's DAT hashes.
- Canonical pins: 0xF4180104 has eight ST_DOUBLE clip-map polygons and 44
  drawable side calls; the canonical NoPos surfaces are type 0x11 and are
  constructed but skipped; two independent extractions of all ten
  canonical cells hash identically.
- AP-234 retired from the divergence register (161 active rows) with
  ConstructMesh 0x0059DFA0 / DrawEnvCell 0x0059F170 / DrawMesh 0x0059D4A0.
- StipplingSurfaceEquivalenceTests no longer claims a build-time NoPos
  skip; the cell half is report-only, the building half stays pinned.
- Inventory and policy remarks updated; plan ledger records chunk A and
  the capture kit.

Core Meshing+Conformance 95/95, Content 208/208 (Lane!=Timing,
Lane!=PreparedPackage), App EnvCell/LandblockBuild/Streaming 432/432,
Release solution build 0 warnings / 0 errors.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Erik 2026-09-02 19:11:17 +02:00
parent acf172469e
commit b681717c30
9 changed files with 631 additions and 154 deletions

View file

@ -0,0 +1,420 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Security.Cryptography;
using AcDream.Core.Meshing;
using DatReaderWriter;
using DatReaderWriter.DBObjs;
using DatReaderWriter.Enums;
using DatReaderWriter.Options;
using Xunit.Abstractions;
using RetailCullMode = DatReaderWriter.Enums.CullMode;
namespace AcDream.Content.Tests;
/// <summary>
/// OH2/S1 chunk B (docs/research/2026-09-01-overhaul/oh2-cellstruct-surface-contract.md
/// §10.3/§10.4): the installed-DAT proof that retail's exact CellStruct
/// surface/subset construction (<see cref="CellStructSideCandidates"/>,
/// <c>MeshExtractor.PrepareCellStructMeshData</c>,
/// <see cref="RetailUntexturedSurfacePolicy"/>) replaces the retired
/// build-time <c>Stippling.NoPos</c> approximation (AP-234) without
/// changing what the COMPLETE installed corpus admits.
///
/// DAT identity this evidence was captured against (contract §1):
/// client_portal.dat DC6E500BA22E6B186DB7171E3F3345238B6444C85D798ADC85E550973B8D12E4
/// client_cell_1.dat 6DB0ABF00FBCEED62C3F1EE842EE7C1F423D732BED77A5B7C102EE89A52AB99E
/// client_highres.dat 503E0828D14F2F9CCBC31431E1055AC188464BF4B499DE37F4C3D5B2D9F3E727
/// client_local_English.dat E85C820280C88FAC7DF6C8043F5E24596E9C8774193AF4123D756546F78FB2BB
/// A hash change on any of these DATs is expected to move the pinned counts
/// below; that is the point of pinning them (a silent regression in the
/// extraction algorithm itself would move them too, on the SAME dat set).
/// </summary>
[Trait("Lane", "InstalledDat")]
public sealed class CellStructSurfaceConstructionInstalledDatTests {
private readonly ITestOutputHelper _out;
public CellStructSurfaceConstructionInstalledDatTests(ITestOutputHelper output) => _out = output;
// Contract §10.3's ten canonical evidence cells: eight cathedral-area
// cells sharing three Environments, plus the Facility Hub's two-cell
// vertical stair pair.
private static readonly uint[] CanonicalCellIds = {
0xF4180100u, 0xF4180101u, 0xF4180104u, 0xF4180106u, 0xF4180107u,
0xF4180112u, 0xF4180113u, 0xF4180114u,
0x8A02015Eu, 0x8A02015Fu,
};
private static MeshExtractor NewExtractor(DatCollection dats) {
var reader = new DatCollectionAdapter(dats);
return new MeshExtractor(reader, new TestConsoleLogger(), sideStagedSink: null);
}
// ------------------------------------------------------------------
// §10.4 — complete installed-DAT admission-delta scan.
// ------------------------------------------------------------------
/// <summary>
/// Walks every EnvCell actually present in client_cell_1.dat (not the
/// contract's exploratory 89th-cell stride sample) and, for every
/// polygon construction candidate
/// (<see cref="CellStructSideCandidates.GetCandidates"/>), classifies
/// admission under the OLD rule (<c>!NoPos</c> on the positive side,
/// <c>!NoNeg</c> on the ST_BOTH negative side — MeshExtractor's
/// pre-OH2 <c>hasPos</c>/<c>hasNeg</c> gate, contract §5.1) against the
/// exact NEW rule (resolved <c>Surface.Type &amp; (BASE1_IMAGE|BASE1_CLIPMAP)
/// ) != 0</c>). Every delta must be explained by the surface-type fact
/// that produced it: this is the complete-corpus successor to the
/// contract §6 exploratory stride sample (which found 55 concrete
/// "not-NoPos + untextured" false admissions under the OLD rule).
/// </summary>
[Fact]
public void InstalledDatScan_EveryOldNewAdmissionDeltaIsExplainedBySurfaceType() {
var datDir = ContentConformanceDats.ResolveDatDir();
if (datDir is null) { Assert.Fail("Lane=InstalledDat requires an installed retail DAT directory; see docs/release-gate.md."); return; }
var stopwatch = Stopwatch.StartNew();
using var dats = new DatCollection(datDir, DatAccessType.Read);
long landblocksVisited = 0;
long landblocksWithCells = 0;
long cellsVisited = 0;
long cellsWithGeometry = 0;
long missingEnvironmentOrStruct = 0;
long polygons = 0;
long candidates = 0;
long unknownSidesType = 0;
long unresolvedSurfaceSlots = 0;
long oldOnly = 0; // old=emit, new=skip
long newOnly = 0; // old=skip, new=emit
var unexplained = new List<string>();
// Per-EnvCell surface cache: one wall/floor texture is referenced by
// many polygons within a single cell; re-resolving the same Surface
// dat record per polygon would be wasted I/O across ~10^5 EnvCells.
var slotCache = new Dictionary<int, Surface?>();
// Neither EnvCell nor LandBlockInfo carries a usable
// Chorizite.DatReaderWriter DBObjType id-range/mask (both are
// landblock-RELATIVE types — every landblock reuses the same low
// 16-bit id space 0x0100..0xFFFD for its own cells — so
// DatCollection.GetAllIdsOfType<EnvCell>()/<LandBlockInfo>() return
// EMPTY: verified empirically 2026-09-02, then confirmed by
// decompiling DatReaderWriter.Lib.DBObjAttributeCache via ilspycmd
// (EnvCell's own [DBObjType] attribute carries FirstId=LastId=
// MaskId=0; DatDatabase.GetAllIdsOfType only consults
// Tree.GetFilesInRange(FirstId,LastId), never the per-landblock
// catch-all TypeFromId path DBObjTypeFromId falls back to for the
// Cell dat). The complete corpus therefore requires the same
// landblock-prefix walk LandblockBuildFactory/StipplingSurface
// EquivalenceTests already use per-fixture, run over the WHOLE
// 0x0000-0xFFFF landblock-prefix space instead of a hardcoded
// fixture list — exactly the shape of the contract §6 exploratory
// sample's own "734,976 candidate EnvCell ids", but complete
// (every candidate, not every 89th).
for (uint landblockPrefix = 0; landblockPrefix <= 0xFFFFu; landblockPrefix++) {
uint lbId = landblockPrefix << 16;
var lbInfo = dats.Get<LandBlockInfo>(lbId | 0xFFFEu);
if (lbInfo is null) continue;
landblocksVisited++;
if (lbInfo.NumCells == 0) continue;
landblocksWithCells++;
for (uint low = 0x0100u; low < 0x0100u + lbInfo.NumCells; low++) {
uint envCellId = lbId | low;
cellsVisited++;
var envCell = dats.Get<EnvCell>(envCellId);
if (envCell is null || envCell.EnvironmentId == 0) continue;
uint envId = 0x0D000000u | envCell.EnvironmentId;
var environment = dats.Get<DatReaderWriter.DBObjs.Environment>(envId);
if (environment is null || !environment.Cells.TryGetValue(envCell.CellStructure, out var cellStruct)) {
missingEnvironmentOrStruct++;
continue;
}
cellsWithGeometry++;
slotCache.Clear();
Surface? ResolveSlot(int slot) {
if (slotCache.TryGetValue(slot, out var cached)) return cached;
Surface? resolved = null;
if (slot >= 0 && slot < envCell.Surfaces.Count) {
uint surfaceId = 0x08000000u | envCell.Surfaces[slot];
resolved = dats.Get<Surface>(surfaceId);
}
slotCache[slot] = resolved;
return resolved;
}
foreach (var (polyId, poly) in cellStruct.Polygons) {
if (poly.VertexIds.Count < 3) continue;
polygons++;
ReadOnlySpan<CellStructSideCandidate> polyCandidates =
CellStructSideCandidates.GetCandidates((int)poly.SidesType);
if (polyCandidates.Length == 0) {
unknownSidesType++;
continue;
}
foreach (var candidate in polyCandidates) {
candidates++;
bool isPositive = candidate.SurfaceSlot == CellStructPolygonSurfaceSide.Positive;
short slotRaw = isPositive ? poly.PosSurface : poly.NegSurface;
bool oldAdmitted = isPositive
? !poly.Stippling.HasFlag(StipplingType.NoPos)
: !poly.Stippling.HasFlag(StipplingType.NoNeg);
Surface? surface = ResolveSlot(slotRaw);
if (surface is null) {
unresolvedSurfaceSlots++;
continue;
}
bool newAdmitted = (surface.Type & (SurfaceType.Base1Image | SurfaceType.Base1ClipMap)) != 0;
if (oldAdmitted && !newAdmitted) {
oldOnly++;
bool untextured = (surface.Type & (SurfaceType.Base1Image | SurfaceType.Base1ClipMap)) == 0;
if (!untextured)
unexplained.Add($"OLD-ONLY unexplained: envCell=0x{envCellId:X8} env=0x{envId:X8} struct={envCell.CellStructure} poly={polyId} side={candidate.SurfaceSlot} slot={slotRaw} type=0x{(uint)surface.Type:X8}");
}
else if (!oldAdmitted && newAdmitted) {
newOnly++;
bool uvAbsenceBitSet = isPositive
? poly.Stippling.HasFlag(StipplingType.NoPos)
: poly.Stippling.HasFlag(StipplingType.NoNeg);
if (!uvAbsenceBitSet)
unexplained.Add($"NEW-ONLY unexplained: envCell=0x{envCellId:X8} env=0x{envId:X8} struct={envCell.CellStructure} poly={polyId} side={candidate.SurfaceSlot} slot={slotRaw} type=0x{(uint)surface.Type:X8} stippling={poly.Stippling}");
}
}
}
}
}
stopwatch.Stop();
_out.WriteLine($"duration: {stopwatch.Elapsed}");
_out.WriteLine($"landblocksVisited={landblocksVisited} landblocksWithCells={landblocksWithCells}");
_out.WriteLine($"cellsVisited={cellsVisited} cellsWithGeometry={cellsWithGeometry} missingEnvironmentOrStruct={missingEnvironmentOrStruct}");
_out.WriteLine($"polygons={polygons} candidates={candidates} unknownSidesType={unknownSidesType} unresolvedSurfaceSlots={unresolvedSurfaceSlots}");
_out.WriteLine($"oldOnly={oldOnly} newOnly={newOnly} unexplained={unexplained.Count}");
foreach (var line in unexplained.Take(50)) _out.WriteLine(" " + line);
Assert.Empty(unexplained);
// Golden counts pinned against the DAT identity in the class
// docstring (contract §1). A DAT change or an extraction-algorithm
// regression moves these; that visibility is the point of pinning
// them (contract §10.4's "run the complete installed Environment/
// EnvCell corpus... every old/new admission delta must have the
// exact Surface.Type & 6 explanation").
Assert.Equal(CellStructInstalledDatGolden.CellsVisited, cellsVisited);
Assert.Equal(CellStructInstalledDatGolden.CellsWithGeometry, cellsWithGeometry);
Assert.Equal(CellStructInstalledDatGolden.MissingEnvironmentOrStruct, missingEnvironmentOrStruct);
Assert.Equal(CellStructInstalledDatGolden.Polygons, polygons);
Assert.Equal(CellStructInstalledDatGolden.Candidates, candidates);
Assert.Equal(CellStructInstalledDatGolden.UnknownSidesType, unknownSidesType);
Assert.Equal(CellStructInstalledDatGolden.UnresolvedSurfaceSlots, unresolvedSurfaceSlots);
Assert.Equal(CellStructInstalledDatGolden.OldOnly, oldOnly);
Assert.Equal(CellStructInstalledDatGolden.NewOnly, newOnly);
}
// ------------------------------------------------------------------
// §10.3 — canonical evidence pins.
// ------------------------------------------------------------------
/// <summary>
/// Pin 1: 0xF4180104's eight ST_DOUBLE clip-map polygons and 44 total
/// drawable side calls across the WHOLE cell — retail's 36 ordinary
/// (forward, positive-normal) drawable candidates plus the 8 additional
/// reversed negative-normal copies the ST_DOUBLE polygons contribute
/// (contract §6): the old NoPos-proxy audit's "36-positive-only"
/// headline never counted the second copy at all.
/// </summary>
[Fact]
public void Cell0xF4180104_HasEightStDoubleClipMapPolygonsAndFortyFourDrawableSideCalls() {
var datDir = ContentConformanceDats.ResolveDatDir();
if (datDir is null) { Assert.Fail("Lane=InstalledDat requires an installed retail DAT directory; see docs/release-gate.md."); return; }
using var dats = new DatCollection(datDir, DatAccessType.Read);
const uint envCellId = 0xF4180104u;
var (envCell, _, cellStruct) = ResolveCanonicalCell(dats, envCellId);
int stDoubleCount = 0;
int drawableSideCalls = 0;
var slotCache = new Dictionary<int, Surface?>();
Surface? ResolveSlot(int slot) {
if (slotCache.TryGetValue(slot, out var cached)) return cached;
Surface? resolved = null;
if (slot >= 0 && slot < envCell.Surfaces.Count) {
uint surfaceId = 0x08000000u | envCell.Surfaces[slot];
resolved = dats.Get<Surface>(surfaceId);
}
slotCache[slot] = resolved;
return resolved;
}
foreach (var poly in cellStruct.Polygons.Values) {
if (poly.VertexIds.Count < 3) continue;
if (poly.SidesType == RetailCullMode.None) stDoubleCount++; // raw sides_type 1 = ST_DOUBLE
foreach (var candidate in CellStructSideCandidates.GetCandidates((int)poly.SidesType)) {
bool isPositive = candidate.SurfaceSlot == CellStructPolygonSurfaceSide.Positive;
short slotRaw = isPositive ? poly.PosSurface : poly.NegSurface;
var surface = ResolveSlot(slotRaw);
if (surface is null) continue;
if (!RetailUntexturedSurfacePolicy.IsUntextured(surface.Type))
drawableSideCalls++;
}
}
_out.WriteLine($"0xF4180104: stDoubleCount={stDoubleCount} drawableSideCalls={drawableSideCalls}");
Assert.Equal(8, stDoubleCount);
Assert.Equal(44, drawableSideCalls);
}
/// <summary>
/// Pin 2: across all ten canonical cells, every polygon whose positive
/// side is NoPos-stippled resolves to a Surface whose raw Type is
/// exactly 0x11 (BASE1_SOLID | TRANSLUCENT — untextured), AND that
/// surface slot is absent from the emitted prepared mesh (constructed —
/// the slot has at least one admission candidate by
/// <see cref="CellStructSideCandidates.GetCandidates"/> construction —
/// but skipped at built-EnvCell admission, contract §4/§6).
/// </summary>
[Fact]
public void CanonicalCells_NoPosSurfacesAreType0x11AndConstructedButSkipped() {
var datDir = ContentConformanceDats.ResolveDatDir();
if (datDir is null) { Assert.Fail("Lane=InstalledDat requires an installed retail DAT directory; see docs/release-gate.md."); return; }
using var dats = new DatCollection(datDir, DatAccessType.Read);
var extractor = NewExtractor(dats);
int noPosSurfacesChecked = 0;
foreach (uint envCellId in CanonicalCellIds) {
var (envCell, _, cellStruct) = ResolveCanonicalCell(dats, envCellId);
var noPosSlots = new HashSet<int>();
foreach (var poly in cellStruct.Polygons.Values) {
if (poly.VertexIds.Count < 3) continue;
if (!poly.Stippling.HasFlag(StipplingType.NoPos)) continue;
if (poly.PosSurface < 0 || poly.PosSurface >= envCell.Surfaces.Count) continue;
uint surfaceId = 0x08000000u | envCell.Surfaces[poly.PosSurface];
var surface = dats.Get<Surface>(surfaceId);
Assert.NotNull(surface);
Assert.Equal(0x11u, (uint)surface!.Type);
noPosSlots.Add(poly.PosSurface);
noPosSurfacesChecked++;
}
if (noPosSlots.Count == 0) continue;
ObjectMeshData? mesh = extractor.PrepareMeshData(envCellId | 0x1_0000_0000UL, isSetup: false);
Assert.NotNull(mesh);
var emittedSlots = CellSurfaceSubsets.InAscendingSurfaceOrder(mesh!)
.Select(b => b.SourceSurfaceIndex)
.ToHashSet();
foreach (int slot in noPosSlots)
Assert.DoesNotContain(slot, emittedSlots);
}
_out.WriteLine($"noPosSurfacesChecked={noPosSurfacesChecked} across {CanonicalCellIds.Length} canonical cells");
Assert.True(noPosSurfacesChecked > 0, "expected at least one NoPos polygon across the canonical cells (contract §6)");
}
/// <summary>
/// Pin 3: two independent extraction runs of every canonical cell
/// produce a bit-identical SHA-256 over the emitted subset records
/// (SourceSurfaceIndex, RawSurfaceType, RetailSurfaceMask, index count,
/// total mesh vertex count), walked in
/// <see cref="CellSurfaceSubsets.InAscendingSurfaceOrder"/> — proving
/// the ascending-source-surface-index order is independent of
/// dictionary/hash iteration order (contract §3.6, §10.3).
/// </summary>
[Fact]
public void CanonicalCells_EmittedSubsetHash_IsDeterministicAcrossTwoRuns() {
var datDir = ContentConformanceDats.ResolveDatDir();
if (datDir is null) { Assert.Fail("Lane=InstalledDat requires an installed retail DAT directory; see docs/release-gate.md."); return; }
using var dats = new DatCollection(datDir, DatAccessType.Read);
foreach (uint envCellId in CanonicalCellIds) {
// Fresh MeshExtractor per run: proves determinism is a property
// of the extraction algorithm itself, not of any state an
// extractor instance happens to retain (decoded-texture cache,
// solid-color cache) across calls.
var extractor1 = NewExtractor(dats);
var extractor2 = NewExtractor(dats);
ObjectMeshData? mesh1 = extractor1.PrepareMeshData(envCellId | 0x1_0000_0000UL, isSetup: false);
ObjectMeshData? mesh2 = extractor2.PrepareMeshData(envCellId | 0x1_0000_0000UL, isSetup: false);
Assert.NotNull(mesh1);
Assert.NotNull(mesh2);
string hash1 = HashCellSubsets(mesh1!);
string hash2 = HashCellSubsets(mesh2!);
_out.WriteLine($"0x{envCellId:X8} hash={hash1}");
Assert.Equal(hash1, hash2);
}
}
private static string HashCellSubsets(ObjectMeshData mesh) {
using var stream = new System.IO.MemoryStream();
using (var writer = new System.IO.BinaryWriter(stream, System.Text.Encoding.UTF8, leaveOpen: true)) {
writer.Write(mesh.Vertices.Length);
foreach (var batch in CellSurfaceSubsets.InAscendingSurfaceOrder(mesh)) {
writer.Write(batch.SourceSurfaceIndex);
writer.Write(batch.RawSurfaceType);
writer.Write(batch.RetailSurfaceMask);
writer.Write(batch.Indices.Count);
}
}
return Convert.ToHexString(SHA256.HashData(stream.ToArray()));
}
private static (EnvCell envCell, uint envId, DatReaderWriter.Types.CellStruct cellStruct) ResolveCanonicalCell(
DatCollection dats, uint envCellId) {
var envCell = dats.Get<EnvCell>(envCellId);
Assert.NotNull(envCell);
uint envId = 0x0D000000u | envCell!.EnvironmentId;
var environment = dats.Get<DatReaderWriter.DBObjs.Environment>(envId);
Assert.NotNull(environment);
Assert.True(environment!.Cells.TryGetValue(envCell.CellStructure, out var cellStruct),
$"EnvCell 0x{envCellId:X8}: Environment 0x{envId:X8} has no CellStruct {envCell.CellStructure}");
return (envCell, envId, cellStruct!);
}
}
/// <summary>
/// Golden counts for
/// <see cref="CellStructSurfaceConstructionInstalledDatTests.InstalledDatScan_EveryOldNewAdmissionDeltaIsExplainedBySurfaceType"/>,
/// pinned against the DAT identity in that class's docstring. Regenerate by
/// reading the test's own diagnostic output after a deliberate DAT/
/// extraction-algorithm change and updating every field together — a
/// partial update would silently weaken the pin.
/// </summary>
internal static class CellStructInstalledDatGolden {
// Captured 2026-09-02 against the DAT identity in the class docstring
// (contract §1); landblocksVisited=5346, landblocksWithCells=3405 are
// printed diagnostics only (not pinned — they describe the world's
// overall building/dungeon density, orthogonal to this contract).
public const long CellsVisited = 729888;
public const long CellsWithGeometry = 729888;
public const long MissingEnvironmentOrStruct = 0;
public const long Polygons = 8601560;
public const long Candidates = 8608746;
public const long UnknownSidesType = 0;
public const long UnresolvedSurfaceSlots = 0;
// oldOnly (old=emit/new=skip, i.e. "not-NoPos + untextured" — the exact
// false-admission class the contract's own exploratory 1-in-89 stride
// sample estimated at 55 hits in 97,020 polygons): the complete corpus
// finds 3,197 in 8,601,560 polygons, same order of magnitude and same
// direction. newOnly (old=skip/new=emit, i.e. "NoPos + textured") is
// exactly 0 across the WHOLE installed corpus — the complete-corpus
// confirmation of the stride sample's own "NoPos + textured surface = 0"
// finding (contract §6).
public const long OldOnly = 3197;
public const long NewOnly = 0;
}