acdream/tests/AcDream.Content.Tests/InstalledSetupBspPrimitiveDispatchTests.cs
Erik b52967def3 fix(physics): AP-156 — flood the BSP sphere where the geometry is, not at the part origin
The AP-152 retail review (docs/research/2026-08-06-ap152-review-retail.md)
FAILED `4abd1b5e` and is right. `ShadowObjectRegistry.BuildFloodSpheres` took
each physics-BSP part's ROOT BOUNDING SPHERE RADIUS
(FlatCollisionAssetBuilder.cs:393 -> LiveEntityCollisionBuilder.cs:137) and
centred it on the PART ORIGIN (ShadowShapeBuilder.cs:194), discarding the root
sphere's own Origin.

Re-measured independently against the installed client_portal.dat, reproducing
the reviewer's numbers exactly: 376 of 973 physics-BSP parts have
|origin| > radius/2, worst 20.762 m on a 27.708 m sphere (gfx 0x010036DD,
Setup 0x0200129A). Over the 172 Setups AP-152 moved onto that path the emitted
flood FAILED TO CONTAIN the object's own BSP sphere for 170 of them (73
CylSphere-bearing, 97 Sphere-bearing), worst shortfall 9.911 m on Setup
0x02000255 — whose one part's sphere sits 9.911 m above the part origin — and
for 43 the post-AP-152 flood was strictly SMALLER than the pre-AP-152 one.
Indoor flooding is 3-D (CellTransit.cs:601 routes every id & 0xFFFF >= 0x0100
candidate through FindTransitCellsSphere), so a tall prop or door slab was
absent from EnvCells it physically occupies and therefore never a broadphase
candidate there (TransitionTypes.cs:3763 iterates only entries already in the
cell). That is the #98 / #168 class AP-152 exists to remove.

Retail, re-disassembled from the PDB-paired binary (check_exe_pdb.py MATCH,
CodeView GUID 9e847e2f-777c-4bd9-886c-22256bb87f32), every address resolved
back through named-retail/symbols.json:

  CGfxObj::physics_sphere is [gfxobj+0x74] (physics_bsp is [+0x78], as
  CPartArray::CacheHasPhysicsBSP @0x00518110 reads at 0x00518127), and
  acclient pseudo-C 0x00534b5b assigns it BSPTREE::GetSphere(physics_bsp).

  BSPTREE::GetSphere @0x005397e0
    8b01        mov eax,[ecx]     ; BSPTREE::root_node
    83c004      add eax,4         ; past BSPNODE::vfptr -> CSphere sphere
  So retail's per-part flood sphere IS the BSP root bounding sphere,
  ORIGIN INCLUDED (acclient.h: BSPNODE { vfptr; CSphere sphere; ... },
  CSphere { Vector3 center; float radius; } -> radius at +0xc).

  CPhysicsObj::find_bbox_cell_list @0x00510fc0 adds the object's own cell and
  then walks the PART ARRAY: 0x00511012 call 0x518160
  (CPartArray::calc_cross_cells_static), which dispatches [edx+0x7c] with
  (num_parts, parts, cellarray). Its EnvCell body,
  CEnvCell::find_transit_cells @0x0052cae0:
    0x0052cb31  mov edx,[eax+0x20]   ; CPhysicsPart::gfxobj (CGfxObj**)
    0x0052cb36  mov esi,[ecx+0x74]   ; physics_sphere (else +0x90 drawing)
    0x0052cb4c  add eax,0x30         ; CPhysicsPart::pos
    0x0052cb5a  call Position::localtolocal   ; transform the sphere CENTRE
    0x0052cb65  fadd [esi+0xc]       ; only NOW the radius
  Retail transforms the centre through the part's own Position before it ever
  touches the radius. Carrying the radius alone is not an approximation of
  that; it is a different sphere.

Changes:

* `ShadowShape` gains `BoundsCenter` — the bounding sphere's centre in the
  shape's own local frame, scaled like LocalPosition and Radius. Zero for
  Cylinder/Sphere shapes, whose LocalPosition already IS their centre.

* `ShadowShapeBuilder.FromSetup` gains a `physicsBspBounds` resolver that
  supplies radius AND centre from ONE call, replacing the placeholder radius
  plus a downstream substitution. `LiveEntityCollisionBuilder` now holds a
  single `Func<uint, FlatCollisionSphere?>` and derives its dispatch predicate
  from it, so the gate and the geometry cannot disagree and the radius cannot
  be taken while the origin is dropped. That split is what produced this bug;
  it no longer exists.

* `FromLandblockBspParts` carries the centre too. A landblock-baked part array
  is the same CPartArray walk, so stair runs, fences and rock clusters had the
  identical defect. Both storage forms (flat BSP and the graph fallback) are
  covered.

* `BuildFloodSpheres` places each sphere at
  partWorldPos + rotate(BoundsCenter, partWorldRot), composed exactly as the
  ShadowEntry rows are.

* The 10-sphere clamp now applies to the CYLSPHERE branch only. Retail's clamp
  is inside CObjCell::find_cell_list @0x0052b9f0
  (0x0052ba21 cmp eax,0xa / 0x0052ba28 mov ebp,0xa); the BSP walk has none and
  the sorting-sphere overload @0x0052b990 takes one sphere. 7 installed Setups
  carry more than 10 physics-BSP parts (max 49, Setup 0x02001A91) and their
  tail parts were dropped from the flood entirely. Without this the new
  containment assertion would have covered shapes production never floods
  from.

Register. AP-155 was two divergences with different code paths, populations
and gates under one id; it is NARROWED to its static-publication half and its
flood half is split out as AP-156 WITH ITS DIRECTION CORRECTED. AP-155(b)
recorded the approximation as over-inclusive — "floods MORE cells rather than
fewer, the safe direction for membership" — and that false direction was the
stated reason the residual was safe to defer. It was under-inclusive for 170
of 172. AP-156 records the correction, this fix, and the one genuine residual:
acdream's sphere-vs-portal traversal where retail walks each part's sphere
against the cell's own portal planes. AP-155(b)'s "acdream approximates
retail's bounding BOX" was wrong too — find_bbox_cell_list forms no box.
AP-157 filed for the review's F4: retail's third branch floods from ONE
CPartArray::GetSortingSphere @0x00518b00 ([partArray+0x54]+0x70 =
CSetup::sorting_sphere; 4,154 of 5,935 installed Setups carry a non-zero one)
where acdream floods from every Sphere shape, and acdream's cylinder flood
ignores CylHeight. Deliberately NOT bundled here: different branch, disjoint
population, different live gate. Active AP rows 107 -> 109, literal count.

Tests. Both flood tests the review named substituted a CONCENTRIC Radius = 14f
at LocalPosition = Zero — the one configuration in which the defect cannot
appear. Every fixture is now off-centre by default, and
`FromSetup_CylSphereAndBspSetup_FloodsTheBspFootprint` drives the production
`physicsBspBounds` seam instead of hand-substituting. Five new facts: the
flood centres on BoundsCenter not the part origin; it rotates BoundsCenter by
the part rotation; it caps cylspheres at ten but never the BSP parts; the
landblock path carries the scaled centre in both storage forms; and an
installed-DAT containment sweep asserting every emitted BSP flood sphere
contains that part's real bounding sphere at entity scale 1.75, behind four
external controls — 973 parts, 376 off-centre, 172 affected, and 170
would-fail-if-the-origin-were-discarded, the last of which fails if the
population ever stops exercising the field.

Nine sabotages, each reverted and re-verified:
  A drop BoundsCenter from the flood       -> 3 Core
  B rotate by entity rot, not part rot     -> 1 Core (the rotation fact only)
  C FromSetup discards the origin          -> 1 Core + 2 App + 1 Content
     (the shipped defect, now caught in three projects)
  D drop entScale on BoundsCenter          -> 2 App + 1 Content
  E landblock flat branch drops the centre -> 1 Core
  F landblock graph branch drops it        -> 1 Core
  G drop partScale on the landblock centre -> 1 Core
  H re-apply the 10-cap to every branch    -> 1 Core
  I remove the cylsphere cap               -> 1 Core
AP-152's own two sabotages re-run against this tree: the step-0 gate disabled
still reddens exactly its five facts with Headless 89/89 green, and
cylinder-first flooding still reddens exactly one.

Clean Release build after deleting all 44 bin/obj: 0 errors, 21 pre-existing
warnings. Complete suite 11,208 passed / 4 skipped / 0 failed, +5 on the
11,203 baseline at 4abd1b5e — Core 4264 -> 4268, Content 126 -> 127, App
unchanged (one rename, not an addition). No new skips.

NOT yet gated live. This moves shadow-cell membership for real objects, in
both directions, and the connected session must look for both: props and doors
that START blocking from a neighbouring cell (the 73 CylSphere+BSP Setups),
AND ones that STOP blocking (the 99 Sphere+BSP Setups can shrink; 43 shrink
below their pre-4abd1b5e size, which is the regression this fixes). Tall
indoor props and door slabs — the ones whose sphere sits metres above the part
origin — are where the change is largest.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-06 15:52:04 +02:00

375 lines
16 KiB
C#

using System.Globalization;
using System.Numerics;
using AcDream.Core.Physics;
using DatReaderWriter;
using DatReaderWriter.DBObjs;
using DatReaderWriter.Enums;
using DatReaderWriter.Options;
using DatReaderWriter.Types;
namespace AcDream.Content.Tests;
/// <summary>
/// AP-152 population + behaviour proof over the installed client_portal.dat.
///
/// <para>
/// Retail dispatches a Setup's collision geometry EXCLUSIVELY, BSP first, at
/// both consumers: <c>CPhysicsObj::FindObjCollisions</c> @0x0050f050 tests
/// <c>HAS_PHYSICS_BSP_PS</c> at <c>0x0050f165</c> and leaves the BSP branch
/// through the unconditional <c>0x0050f19d jmp 0x50f2b0</c>, past both the
/// CylSphere loop (0x50f1a2) and the Sphere loop (0x50f21d); and
/// <c>CPhysicsObj::calc_cross_cells</c> @0x00515230 tests the same flag at
/// <c>0x00515285</c> and routes to <c>CPhysicsObj::find_bbox_cell_list</c>
/// @0x00510fc0 at <c>0x0051528f jne 0x515305</c>, never reaching its
/// cylsphere (0x005152d1) or sorting-sphere (0x005152fb) branches.
/// </para>
///
/// <para>
/// This sweep pins the affected population and asserts that
/// <see cref="ShadowShapeBuilder.FromSetup"/> emits NO primitive for any of
/// it. Retail derives the dispatch flag from the parts themselves
/// (<c>CPartArray::CacheHasPhysicsBSP</c> @0x00518110 ORs 0x10000 on the first
/// part whose <c>gfxobj-&gt;physics_bsp</c> is non-null), which is exactly the
/// predicate used here.
/// </para>
/// </summary>
public sealed class InstalledSetupBspPrimitiveDispatchTests
{
// EXTERNAL constants. The four bucket controls are the ones already
// committed by the AP-22 reachability sweep (measured by an independent
// raw client_portal.dat B-tree parse that validated itself by byte
// accounting); the affected counts were measured on 2026-08-06 by a
// separate DatReaderWriter sweep that reproduced FromSetup's steps rather
// than calling it.
//
// They are deliberately NOT derived from the predicates below. A broken
// enumeration, a wrong dat path, or a silently-empty decode all satisfy
// the affected-count claim vacuously and are caught only by the controls.
private const int ExpectedSetups = 5935;
private const int ExpectedWithCylinder = 678;
private const int ExpectedSphereOnlyNoCylinder = 3605;
private const int ExpectedWithoutAnyPrimitive = 1652;
private const int ExpectedAffected = 172;
private const int ExpectedAffectedCylinderBearing = 73;
private const int ExpectedAffectedSphereBearing = 99;
private const int ExpectedWithPhysicsBspPart = 530;
[Fact]
public void InstalledSetups_WithBothAPrimitiveAndAPhysicsBspPart_EmitOnlyBspShapes()
{
string? datDir = ContentConformanceDats.ResolveDatDir();
if (datDir is null)
return;
using var dats = new DatCollection(datDir, DatAccessType.Read);
// Production physics-BSP predicate, FlatCollisionAssetBuilder.cs:377-380.
var physicsBspCache = new Dictionary<uint, bool>();
bool HasPhysicsBsp(uint gfxObjId)
{
if (physicsBspCache.TryGetValue(gfxObjId, out bool cached))
return cached;
bool result =
dats.Portal.TryGet<GfxObj>(gfxObjId, out GfxObj? gfx)
&& gfx is not null
&& gfx.Flags.HasFlag(GfxObjFlags.HasPhysics)
&& gfx.PhysicsBSP?.Root is not null
&& gfx.VertexArray is not null;
physicsBspCache[gfxObjId] = result;
return result;
}
int total = 0;
int withCylinder = 0;
int sphereOnly = 0;
int withoutPrimitive = 0;
int withPhysicsBspPart = 0;
int affected = 0;
int affectedCylinderBearing = 0;
int affectedSphereBearing = 0;
var affectedThatStillEmitAPrimitive = new List<uint>();
foreach (uint id in dats.GetAllIdsOfType<Setup>())
{
if (!dats.Portal.TryGet<Setup>(id, out Setup? setup) || setup is null)
continue;
total++;
bool hasCylinder = false;
foreach (var cyl in setup.CylSpheres)
{
if (cyl.Radius > 0f) { hasCylinder = true; break; }
}
bool hasSphere = false;
foreach (var sph in setup.Spheres)
{
if (sph.Radius > 0f) { hasSphere = true; break; }
}
// FromSetup step 2 is gated on CylSpheres.Count == 0, so a Setup
// with both only ever emitted Cylinders.
bool emitsSphere = setup.CylSpheres.Count == 0 && hasSphere;
if (hasCylinder) withCylinder++;
else if (emitsSphere) sphereOnly++;
else withoutPrimitive++;
bool hasBspPart = false;
foreach (uint partId in setup.Parts)
{
if (HasPhysicsBsp(partId)) { hasBspPart = true; break; }
}
if (hasBspPart) withPhysicsBspPart++;
if (!hasBspPart || !(hasCylinder || emitsSphere))
continue;
affected++;
if (hasCylinder) affectedCylinderBearing++;
else affectedSphereBearing++;
// The behaviour: for every affected Setup the production builder
// must emit BSP shapes only.
IReadOnlyList<ShadowShape> shapes =
ShadowShapeBuilder.FromSetup(setup, 1f, HasPhysicsBsp);
bool clean = shapes.Count > 0;
foreach (ShadowShape shape in shapes)
{
if (shape.CollisionType != ShadowCollisionType.BSP)
{
clean = false;
break;
}
}
if (!clean)
affectedThatStillEmitAPrimitive.Add(id);
}
// Positive controls first — without these the claim below is
// satisfiable by an empty enumeration.
Assert.Equal(ExpectedSetups, total);
Assert.Equal(ExpectedWithCylinder, withCylinder);
Assert.Equal(ExpectedSphereOnlyNoCylinder, sphereOnly);
Assert.Equal(ExpectedWithoutAnyPrimitive, withoutPrimitive);
Assert.Equal(ExpectedWithPhysicsBspPart, withPhysicsBspPart);
Assert.Equal(ExpectedAffected, affected);
Assert.Equal(ExpectedAffectedCylinderBearing, affectedCylinderBearing);
Assert.Equal(ExpectedAffectedSphereBearing, affectedSphereBearing);
Assert.Empty(affectedThatStillEmitAPrimitive);
}
// EXTERNAL constants for the containment sweep, measured 2026-08-06 by a
// scratch DatReaderWriter sweep that reproduced the geometry by hand
// rather than calling ShadowShapeBuilder, and independently reproduced by
// the AP-152 retail reviewer's own sweep. NOT derived from the code under
// test.
//
// OffCentreParts is the population control: without it, a build in which
// every BSP root sphere happened to sit at its part origin would satisfy
// the containment claim vacuously.
// WouldFailIfOriginDiscarded is the DEFECT control: it re-runs the
// pre-fix composition (radius carried, root-sphere origin dropped) and
// pins how many of the affected Setups it breaks. If that number ever
// goes to zero the fixture population has stopped exercising the field
// and the containment assertion below has stopped meaning anything.
private const int ExpectedPhysicsBspParts = 973;
private const int ExpectedOffCentreParts = 376; // |origin| > radius/2
private const int ExpectedWouldFailIfOriginDiscarded = 170; // of 172
private const int ExpectedDeepestBspPartArray = 49; // Setup 0x02001A91
/// <summary>
/// AP-156. Every flood sphere acdream emits for a physics-BSP part must
/// CONTAIN that part's real collision geometry.
///
/// <para>
/// A GfxObj's physics BSP is authored in the GfxObj's own coordinates and
/// its root bounding sphere is usually not centred on that origin — 376
/// of the 973 installed physics-BSP parts sit further from it than half
/// their own radius, worst 20.762 m on a 27.708 m sphere (gfx 0x010036DD,
/// Setup 0x0200129A). acdream used to take the sphere's radius and drop
/// its origin, flooding from the part origin instead: over the 172
/// AP-152 Setups that failed to contain the object's own BSP sphere for
/// 170 of them, worst shortfall 9.911 m (Setup 0x02000255, whose single
/// part's root sphere sits 9.911 m above the part origin). Indoor floods
/// are 3-D (<c>CellTransit.BuildShadowCellSet</c> routes every candidate
/// with <c>id &amp; 0xFFFF &gt;= 0x0100</c> through
/// <c>FindTransitCellsSphere</c>), so a tall prop or door slab simply was
/// not registered in the EnvCells it occupies — never a broadphase
/// candidate there, the #98 / #168 class.
/// </para>
///
/// <para>
/// Retail: <c>CGfxObj::physics_sphere</c> (<c>[gfxobj+0x74]</c>) is
/// assigned <c>BSPTREE::GetSphere(physics_bsp)</c> @0x005397e0 — the root
/// <c>BSPNODE</c>'s <c>CSphere</c>, past its 4-byte vftable — and
/// <c>CEnvCell::find_transit_cells</c> @0x0052cae0, the part-array
/// overload reached from <c>CPhysicsObj::find_bbox_cell_list</c>
/// @0x00510fc0 via <c>CPartArray::calc_cross_cells_static</c> @0x00518160,
/// transforms that sphere's CENTRE through the part's own Position at
/// <c>[part+0x30]</c> before reading its radius at <c>[esi+0xc]</c>.
/// </para>
/// </summary>
[Fact]
public void InstalledSetups_BspFloodSpheres_ContainTheirOwnBoundingSpheres()
{
string? datDir = ContentConformanceDats.ResolveDatDir();
if (datDir is null)
return;
using var dats = new DatCollection(datDir, DatAccessType.Read);
var boundsCache = new Dictionary<uint, FlatCollisionSphere?>();
FlatCollisionSphere? Bounds(uint gfxObjId)
{
if (boundsCache.TryGetValue(gfxObjId, out FlatCollisionSphere? cached))
return cached;
FlatCollisionSphere? result = null;
if (dats.Portal.TryGet<GfxObj>(gfxObjId, out GfxObj? gfx)
&& gfx is not null
&& gfx.Flags.HasFlag(GfxObjFlags.HasPhysics)
&& gfx.PhysicsBSP?.Root is not null
&& gfx.VertexArray is not null
&& gfx.PhysicsBSP.Root.BoundingSphere is { } bs)
{
result = new FlatCollisionSphere(bs.Origin, bs.Radius);
}
boundsCache[gfxObjId] = result;
return result;
}
const float EntScale = 1.75f; // not 1: a dropped scale must show up
int bspParts = 0;
int offCentreParts = 0;
int affected = 0;
int wouldFailIfOriginDiscarded = 0;
float worstShortfall = 0f;
uint worstShortfallSetup = 0u;
int mostBspShapesOnOneSetup = 0;
var uncontained = new List<uint>();
foreach (uint id in dats.GetAllIdsOfType<Setup>())
{
if (!dats.Portal.TryGet<Setup>(id, out Setup? setup) || setup is null)
continue;
// Independent oracle: resolve the placement frame from the raw
// Setup and place each part's TRUE root sphere by hand.
AnimationFrame? placement = null;
if (setup.PlacementFrames.TryGetValue(Placement.Resting, out var resting))
placement = resting;
else if (setup.PlacementFrames.TryGetValue(Placement.Default, out var def))
placement = def;
else foreach (var kvp in setup.PlacementFrames) { placement = kvp.Value; break; }
var truth = new List<(Vector3 Centre, float Radius)>();
for (int i = 0; i < setup.Parts.Count; i++)
{
FlatCollisionSphere? b = Bounds((uint)setup.Parts[i]);
if (b is null) continue;
bspParts++;
if (b.Value.Origin.Length() > b.Value.Radius / 2f)
offCentreParts++;
Vector3 partOrigin = Vector3.Zero;
Quaternion partRot = Quaternion.Identity;
if (placement is not null && i < placement.Frames.Count)
{
partOrigin = placement.Frames[i].Origin;
partRot = placement.Frames[i].Orientation;
}
truth.Add((
(partOrigin + Vector3.Transform(b.Value.Origin, partRot)) * EntScale,
b.Value.Radius * EntScale));
}
if (truth.Count == 0) continue;
bool hasCylinder = false;
foreach (var cyl in setup.CylSpheres) if (cyl.Radius > 0f) { hasCylinder = true; break; }
bool hasSphere = false;
foreach (var sph in setup.Spheres) if (sph.Radius > 0f) { hasSphere = true; break; }
if (hasCylinder || (setup.CylSpheres.Count == 0 && hasSphere))
affected++;
// Production emission, through the production bounds seam.
IReadOnlyList<ShadowShape> shapes = ShadowShapeBuilder.FromSetup(
setup,
EntScale,
id => Bounds(id) is not null,
physicsBspBounds: Bounds);
// ShadowObjectRegistry.BuildFloodSpheres' composition, at an
// entity placed at the world origin with identity rotation.
// EVERY BSP shape contributes: retail's BSP branch has no
// sphere cap (the 10-clamp at 0x0052ba21 is inside the cylsphere
// overload only), and 7 installed Setups carry more than 10
// physics-BSP parts — capping here would silently exclude their
// tail from the containment claim below.
var flood = new List<(Vector3 Centre, float Radius)>();
var floodIfOriginDiscarded = new List<(Vector3 Centre, float Radius)>();
foreach (ShadowShape shape in shapes)
{
if (shape.CollisionType != ShadowCollisionType.BSP) continue;
flood.Add((
shape.LocalPosition
+ Vector3.Transform(shape.BoundsCenter, shape.LocalRotation),
shape.Radius));
floodIfOriginDiscarded.Add((shape.LocalPosition, shape.Radius));
}
if (flood.Count > mostBspShapesOnOneSetup)
mostBspShapesOnOneSetup = flood.Count;
float Shortfall(List<(Vector3 Centre, float Radius)> spheres)
{
float worst = 0f;
foreach ((Vector3 tc, float tr) in truth)
{
float best = float.MaxValue;
foreach ((Vector3 fc, float fr) in spheres)
{
float need = (tc - fc).Length() + tr - fr;
if (need < best) best = need;
}
if (best > worst) worst = best;
}
return worst;
}
const float Tolerance = 1e-3f;
float shortfall = Shortfall(flood);
if (shortfall > Tolerance)
{
uncontained.Add(id);
if (shortfall > worstShortfall)
{
worstShortfall = shortfall;
worstShortfallSetup = id;
}
}
if ((hasCylinder || (setup.CylSpheres.Count == 0 && hasSphere))
&& Shortfall(floodIfOriginDiscarded) > Tolerance)
{
wouldFailIfOriginDiscarded++;
}
}
// Population + defect controls first.
Assert.Equal(ExpectedPhysicsBspParts, bspParts);
Assert.Equal(ExpectedOffCentreParts, offCentreParts);
Assert.Equal(ExpectedAffected, affected);
Assert.Equal(ExpectedWouldFailIfOriginDiscarded, wouldFailIfOriginDiscarded);
// Cap control: the deepest installed part array is 49 BSP shapes, so
// the containment claim genuinely covers Setups past the retired
// 10-sphere clamp rather than stopping short of them.
Assert.Equal(ExpectedDeepestBspPartArray, mostBspShapesOnOneSetup);
// The fact.
Assert.True(
uncontained.Count == 0,
$"{uncontained.Count} Setups flood from a sphere that does not contain "
+ $"their own physics-BSP bounding sphere; worst shortfall "
+ $"{worstShortfall.ToString("F3", CultureInfo.InvariantCulture)} m on "
+ $"Setup 0x{worstShortfallSetup:X8}.");
}
}