diff --git a/docs/research/2026-06-24-collision-inclusion-audit.md b/docs/research/2026-06-24-collision-inclusion-audit.md new file mode 100644 index 00000000..6130ce15 --- /dev/null +++ b/docs/research/2026-06-24-collision-inclusion-audit.md @@ -0,0 +1,189 @@ +# Collision-inclusion audit — retail vs acdream ("1 fix for all collision") — 2026-06-24 + +**Mode:** report-first / brainstorm-gated (DO-NOT-RETRY collision area). No code written. +**Method:** 10 parallel named-retail readers (5 retail-model + 5 acdream-channel) → synthesis → **adversarial verification of every claimed deviation against the actual cited code** (the [[feedback_verify_subagent_claims_against_source]] discipline). Plus orchestrator first-hand reads of `CollisionExemption.cs`, `EntityCollisionFlags.cs`, `PhysicsDataCache.cs`, `TransitionTypes.cs`, `GameWindow.cs:7388-7565`, and the divergence register. +**Branch:** `claude/thirsty-goldberg-51bb9b`. Named-retail + `references/{ACE,ACViewer,holtburger}` + `symbols.json` live in the MAIN repo. + +--- + +## TL;DR — the unifying insight + +Retail's collision model is **two clean layers**: + +- **Layer B (registration / broad phase)** — every object with a *DAT physics shape* is written **unconditionally** into the per-cell `shadow_object_list` of every cell its sphere overlaps. **No attribute is tested at registration** (not STATIC, not ETHEREAL, not HIDDEN — only `PARTICLE_EMITTER_PS 0x1000` diverts to a particle list). The only thing that decides "is this registerable" is **does it have a shape** (`physics_bsp` from DAT bit-0, or a `CSetup` cylsphere/sphere). Source: `CObjCell::find_cell_list@0x0052b4e0`, `CPhysicsObj::add_shadows_to_cells@0x00514ae0`. +- **Layer A (inclusion predicate / narrow phase)** — `CPhysicsObj::FindObjCollisions@0x0050f050` decides per-query whether the registered object actually blocks, via a flag predicate (ETHEREAL+IGNORE, viewer-vs-creature, IGNORE_CREATURES, PvP pool), then dispatches **one** shape branch (BSP-only iff `HAS_PHYSICS_BSP_PS 0x10000`, else CylSphere-then-Sphere). + +**acdream already has this two-layer shape** (per-cell `ShadowObjectRegistry` + query-time `CollisionExemption`). The user's intuition is correct, and the divergences are now **narrow and enumerable** — they are NOT a scattered mess of per-channel filters anymore (the #147 portal-count skip was the last of that family and is already deleted in `9743537`). What remains: + +1. **The shape-source rule is impure** — acdream fabricates collision from the **render-mesh AABB** for scenery + outdoor meshes (D1, the prime deviation; retail *never* consults render bounds). This is the one place acdream **over-includes** vs retail. +2. **The query predicate is incomplete** — ETHEREAL-alone short-circuit (D2), no true sphere primitive (D3), PvP/missile terms hardcoded false (watch-item W1). +3. **One cached collision transform can go stale** — EnvCell cell-struct cache lacks the per-apply rebase that buildings got in #146 (D8). Plus two indoor-channel omissions that are latent today (D4 entry-restrictions, D5 obstruction_ethereal). + +So **"1 fix for all collision" = make the shape-source rule pure (DAT-only) + complete the query predicate + give every cached transform the same per-apply rebase.** It is a *unification of three things*, not one line — but it collapses to **one `ObjectCollides(...)` predicate + one registration rule + one rebase rule** that every channel funnels through. + +--- + +## Verdict table (6 confirmed deviations, 2 refuted false-positives) + +| ID | Sev | Layer | Title | Verdict | Register | +|----|-----|-------|-------|---------|----------| +| **D1** | **HIGH** | shape-source | Render-mesh-AABB → synthetic collision cylinder for scenery (#101 residual) | **CONFIRMED** (high) | extend **AP-2** (currently cites only the Setup case at `PhysicsDataCache.cs:96`; the scenery `0x8…` site at `GameWindow.cs:7408` is uncovered) | +| **D2** | MED | predicate | ETHEREAL-alone instant-skip; retail needs ETHEREAL **and** IGNORE_COLLISIONS | **CONFIRMED** (high) | **AD-7** exists | +| **D3** | LOW | shape-source | Setup `Sphere`s coerced to short cylinders (no true sphere primitive) | **CONFIRMED** (high) | **NEW ROW** needed | +| **D4** | LOW | predicate | EnvCell path omits `check_entry_restrictions` (access-locked cells) | **CONFIRMED** (high) | **NEW ROW** (no effect in dev content) | +| **D5** | MED | predicate/state | EnvCell never clears `obstruction_ethereal` — **field doesn't even exist** in acdream `SpherePath` | **CONFIRMED** (high) | **NEW ROW** (couples with D2) | +| **D8** | MED | stale-frame | EnvCell cell-struct `WorldTransform` never evicted/rebased (asymmetric with buildings #146) | **CONFIRMED** (med) | **NEW ROW** (latent; masked by single-block dungeon collapse) | +| ~~D6~~ | — | dispatch | "EnvCell never calls `placement_insert`" | **REFUTED** | acdream *does* branch — inside `BSPQuery.FindCollisions` (`BSPQuery.cs:1717`), not at the call site. Functionally identical to retail. **No fix.** | +| ~~D7~~ | — | terrain | "Terrain silently passes through when no landblock registered" | **REFUTED** | **Retail does the same** — `LScape::get_landcell@0x00505f10` returns null for an unloaded block, `transitional_insert@0x0050b6f0` returns OK_TS on null cell. This is the **#135/#138 streaming-gap free-fall**, a streaming-foundation issue, NOT collision-inclusion. | + +--- + +## The retail model (the oracle table) + +### Layer A — the inclusion predicate (`CPhysicsObj::FindObjCollisions@0x0050f050`) + +PhysicsState flags and whether each gates collision (verified against `acclient.h` + the decomp body, lines 276776-276996, cross-checked vs ACE `PhysicsObj.cs:381-454`): + +| Flag | Value | Gates collision? | Mechanism | +|---|---|---|---| +| `ETHEREAL_PS` | 0x4 | **Only with 0x10** | Instant-skip (return OK_TS, no shape test) requires **both** 0x4 AND 0x10 (`276782`). 0x4 alone → set `sphere_path.obstruction_ethereal=1` and **still run shape test** (`276795-276806`); downstream step-down passes through but contact is recorded. | +| `IGNORE_COLLISIONS_PS` | 0x10 | with 0x4 | (above) | +| `HAS_PHYSICS_BSP_PS` | 0x10000 | **dispatch** | Set (and not missile-ignored, not PvP-exempt `ebp_1==0`) → **BSP-only** via `CPartArray::FindObjCollisions` (`276858-276961`). Clear → CylSphere loop then Sphere branch. **Mutually exclusive** — never both. | +| `STATIC_PS` | 0x1 | no (response only) | Decides hit is reported as `collided_with_environment` vs object collision (`276969-276981`); does not skip. | +| `REPORT_COLLISIONS_PS` | 0x8 | no | Gates `DoCollisionEnd/Begin` *callback delivery* only (`278614`), not geometry. | +| `NODRAW_PS` | 0x20 | **no** | Not read anywhere in the collision path. Render-only. (Confirms handoff hypothesis.) | +| `MISSILE_PS` | 0x40 | no | Marks target `isCreature=true` for sphere dispatch; `missile_ignore` (`274390`) makes a missile mover skip missile targets. | +| `HIDDEN_PS` | 0x4000 | indirect | `set_hidden` *also* sets 0x10 + clears 0x8 as a side-effect (`282992`,`283029`); but HIDDEN alone doesn't set ETHEREAL, so Gate-1 still needs 0x4. | +| `SCRIPTED_COLLISION_PS` | 0x8000 | **no** | Never read in the physics collision path. | +| `FROZEN_PS` | 0x1000000 | no | Gates `update_object` (sim skip) only (`283955`). | + +Plus the non-flag gates: mover≠self, not-creature-when-viewer/IGNORE_CREATURES mover, PvP pool (both-players → skip unless target Impenetrable OR both PK OR both PKLite). + +### Layer B — registration / broad phase + +`shadow_object_list` membership predicate (`CObjCell::find_cell_list@0x0052b4e0`): +- **(1)** object is NOT a pure particle emitter (`PARTICLE_EMITTER_PS 0x1000`), **(2)** at least one collision sphere overlaps the cell volume (indoor: `sphere_intersects_cell != OUTSIDE`; outdoor: sphere center maps to the cell grid square + boundary neighbours). **No other attribute tested at registration.** A single object lands in **multiple** cells' lists (the doorway-door-in-both-cells answer that the A6.P4/BR-7 flood already ports). + +### The four collision channels (per-cell dispatch order: env → building → objects) + +| Channel | Function | Inclusion rule | +|---|---|---| +| **Terrain** | `CLandCell::find_env_collisions@0x00532f20` | `check_entry_restrictions` → `find_terrain_poly` (always 2 triangles) → water-skip clause → `validate_walkable`. Always solid where the landblock is loaded. | +| **EnvCell (indoor)** | `CEnvCell::find_env_collisions@0x0052c130` | `check_entry_restrictions` → **zero `obstruction_ethereal`** → if `structure->physics_bsp != null`: `find_collisions` (or `placement_insert` for INITIAL_PLACEMENT). **No building leg.** | +| **Building shell** | `CBuildingObj::find_building_collisions@0x006b5300` via `CSortCell::find_collisions@0x005340a0` | In `CLandBlockInfo.buildings` for an outdoor block AND `makeBuilding` succeeded AND origin landcell non-null. **Portal count / portal list / model id NOT examined.** BSP on `part_array->parts[0]`. (This is the #147 truth.) | +| **Object shadows** | `CObjCell::find_obj_collisions@0x0052b750` → Layer A | Per-cell shadow list iteration. | + +### Shape assignment (the rule D1 violates) + +A `CPhysicsObj` gets a collision shape from **DAT only**: a part's `CGfxObj.physics_bsp` (deserialized by `CGfxObj::Serialize@0x00534970` *only* when serialized-flags bit-0 / `num_physics_polygons>0`), or `CSetup` cylsphere/sphere arrays. `CPartArray::InitParts@0x00517F40` builds parts from the Setup list only. When none exist, `CPhysicsPart::find_obj_collisions@0x0050D8D0` returns OK (passable) — **no synthesis from drawing geometry.** The render-mesh `gfx_bound_box` is used for cell-registration + frustum culling, **never** collision. + +--- + +## The acdream model (per-channel de-facto predicate) + +| Channel | acdream predicate (de-facto) | Code | Verdict vs retail | +|---|---|---|---| +| **Terrain** | foot-XY inside a registered landblock's `[0,192)²` AND outdoor cell. null landblock → pass through. | `PhysicsEngine.SampleTerrainWalkable` `:261`; `TransitionTypes.cs:2229-2247` | **Faithful** (D7 refuted — retail passes through unloaded blocks too). | +| **Building** | outdoor cell AND `GetBuilding != null` AND `ModelId != 0` AND `GetGfxObj(ModelId).BSP.Root != null` | `TransitionTypes.cs:2805-2819` | **Faithful** post-#146/#147. `ModelId==0 → inert` ≈ retail "no parts[0]". Per-apply rebase via `RemoveBuildingsForLandblock` (#146). | +| **Shadow objects** | source hi-byte 0x01/0x02 AND not building shell AND not phantom Setup/GfxObj AND ≥1 non-zero-radius shape **OR scenery mesh-AABB fallback** | `GameWindow.cs:7185-7565`; `ShadowObjectRegistry.cs`; `ShadowShapeBuilder.cs` | **D1 (mesh-AABB), D3 (sphere-as-cyl)** + watch-items. | +| **EnvCell** | cell in BFS shadow-set AND `CellPhysics.BSP.Root != null` | `TransitionTypes.cs:2062-2207`; `CellTransit.cs` | **D4, D5, D8.** | +| **Query exemption** | `CollisionExemption.ShouldSkip` — ETHEREAL(0x4) alone, viewer/creature, IGNORE_CREATURES, PvP pool | `CollisionExemption.cs:59` | **Faithful except D2** (ETHEREAL-alone). PvP block is a clean port (cross-checked vs ACE). | + +--- + +## Confirmed deviations — detail + +### D1 — Render-mesh-AABB synthetic collision cylinder (HIGH, the prime deviation) +- **Retail:** shape from DAT `physics_bsp`/`CSetup` only; no shape ⇒ passable (`CPhysicsPart::find_obj_collisions@0x0050D8D0` returns OK when `physics_bsp==null`; `CGfxObj::Serialize@0x00534970` gates `physics_bsp` on flags bit-0; `CPartArray::InitParts@0x00517F40`). +- **acdream:** for scenery (`0x80000000` IDs) with no CylSphere/BSP, synthesizes a clamped `[0.30,1.50]m` cylinder from the **world-space render-mesh AABB** and registers it. `GameWindow.cs:7421-7425` gate: `!isPhantomSetup && !isPhantomGfxObj && !_isLandblockStab && (_isOutdoorMesh || (entityBsp==0 && entityCyl==0))`. For scenery `_isOutdoorMesh` is always true (`:7151`); the #101 phantom gates only cover `0x01…` GfxObjs (`IsPhantomGfxObjSource`, `PhysicsDataCache.cs:399-403`) and zero-radius Setups — a Setup-sourced scenery with `Radius>0.0001` and no `physics_bsp` still gets the synthesized cylinder. +- **Why it exists:** the comment admits a *gameplay* motive — "catches trees whose BSP is only on the canopy (player walks under)". This is exactly the attribute/shape category-error the user wants gone: acdream invents collision retail doesn't have. +- **Register:** **AP-2** must be widened to cover `GameWindow.cs:7408` (it currently cites only `PhysicsDataCache.cs:96`). + +### D2 — ETHEREAL-alone instant-skip (MED) +- **Retail:** Gate-1 needs **both** 0x4 AND 0x10 (`276782`); 0x4 alone → `obstruction_ethereal=1` + still run shape test (`276795-276806`), consumed by BSP solid-containment gates (`321692`,`323742`,`324573`). +- **acdream:** `CollisionExemption.cs:78` skips on 0x4 alone. Documented shim (**AD-7**) because ACE `Door.Open()` broadcasts `0x0001000C` (ETHEREAL only). The `obstruction_ethereal` set-and-continue path is unported. + +### D3 — No true sphere primitive (LOW) +- **Retail:** `HAS_PHYSICS_BSP_PS` clear → CylSphere then **Sphere via `CSphere::intersects_sphere`** (`276917`) — a real sphere test. +- **acdream:** `ShadowShapeBuilder.cs:68-81` coerces `Setup.Spheres` → short cylinders (`height = radius*2`); `ShadowCollisionType` enum (`ShadowObjectRegistry.cs:537`) has only `{BSP, Cylinder}` — no Sphere. **New register row.** + +### D4 — EnvCell omits `check_entry_restrictions` (LOW, no current effect) +- **Retail:** `CEnvCell::find_env_collisions@0x0052c130` calls `check_entry_restrictions@0x0052b6d0` first; returns COLLIDED_TS when an access-locked cell's `restriction_obj` rejects the mover. +- **acdream:** `TransitionTypes.cs:2062-2207` has no such gate. No access-locked cells in dev content ⇒ no observable effect, but structurally absent. **New register row.** + +### D5 — `obstruction_ethereal` never cleared — field absent entirely (MED) +- **Retail:** zeros `sphere_path.obstruction_ethereal` on every `find_env_collisions` call (`0x0052c144`) before BSP dispatch. +- **acdream:** the field **does not exist** in `SpherePath` (`TransitionTypes.cs:315-435`) — so the retail BSP solid-weakening gate keyed on it is simply not modelled. Couples with D2 (port the set/clear/consume sites together). **New register row.** + +### D8 — EnvCell cell-struct transform never evicted/rebased (MED, latent) +- **Retail:** no persistent client-side streaming-relative transform cache; cell geometry rebuilt from live cell pointers (eviction analog: `CEnvCell::release`). +- **acdream:** `PhysicsDataCache._cellStruct` (`:179`) is first-wins `ContainsKey` with **no eviction** — contrast `RemoveBuildingsForLandblock` (`:472`, the #146 fix). `PhysicsEngine.RemoveLandblock` (`:121-141`) clears `_landblocks`/`ShadowObjects`/`CellGraph` but never cells. A dungeon streamed out→in after a `_liveCenter` recenter would hold a stale `WorldTransform` (off by `(oldCenter−newCenter)·192m`). Masked today by the single-block collapse+recenter dungeon pattern. **New register row.** This is the #146 stale-frame class the handoff predicted "almost certainly more of these" — and there is exactly one more. + +--- + +## Refuted (do NOT chase — anti-rabbit-hole) + +- **D6 (placement_insert):** acdream **does** route INITIAL_PLACEMENT to `PlacementInsert` — the branch is inside `BSPQuery.FindCollisions` (`BSPQuery.cs:1717`, `if (path.InsertType == Placement || obj.Ethereal)`), not at the `FindEnvCollisions` call site. Identical functional result. +- **D7 (terrain pass-through):** retail `LScape::get_landcell@0x00505f10` returns null for an unloaded block and `transitional_insert@0x0050b6f0` returns OK_TS on a null cell — **retail passes through unloaded terrain too.** The free-fall is the **streaming-gap** (#135/#138), owned by the cell-relative-frame work, not collision-inclusion. +- **Building `ModelId==0` filter** (`TransitionTypes.cs:2813`): **checked, faithful** — equals retail's "no `parts[0]` ⇒ no shell BSP". +- **#147 portal-count skip:** already deleted (`9743537`). The last scattered attribute-as-filter is gone. +- **Registration-time attribute filters:** acdream's registration is already overlap-based with filtering deferred to query time (`CollisionExemption`) — **architecturally already retail-shaped.** The remaining divergences are shape-source + predicate-detail + one stale cache, not a registration-filter mess. + +--- + +## Watch-items (sub-deviations not promoted — note, don't necessarily fix now) + +- **W1 — PvP/missile dispatch terms hardcoded false** (`TransitionTypes.cs:2629`): the cyl-vs-BSP dispatch assumes `pvpExempt=false && missile_ignore=false`. Harmless in M1.5 (no PK/missiles), but a missile/PK mover would wrongly take BSP-only against all BSP targets. Belongs in the unified predicate when PK/missiles ship. +- **W2 — `BldPortalInfo.ExactMatch`** decoded but unconsumed (`BuildingPhysics.cs:74`, `CellTransit.CheckBuildingTransit`): a *transit-feature*, not collision-inclusion — explicitly out of scope for this phase. +- **W3 — `ShadowShapeBuilder` default radius** `2f` (`:97`) vs `GameWindow` stab fallback `1f` (`:7215`) when `BoundingSphere` is null — minor inconsistency. +- **W4 — `CLandCell` ENTIRELY_WATER skip clause** (state 0x04 / 0x40) may be unimplemented in the terrain path — verify before deep-water content. +- **W5 — static-prune `VisibleCellIds` vs retail `do_not_load` stab_list** (`CellTransit.cs:549-562`): approximation; verify against the CEnvCell dat stab_list field. + +--- + +## The unified target — "1 fix for all collision" (PROPOSAL — brainstorm-gate before coding) + +One **`ObjectCollides(targetState, moverState, targetFlags, shape)`** predicate + one **registration rule** + one **rebase rule**, that every channel funnels through: + +**(a) Pure shape resolution (one helper, DAT-only).** Shape comes ONLY from `physics_bsp` (DAT bit-0) or `CSetup` cylsphere/sphere; an object with no DAT shape registers **no shape** and is passable. *Delete the mesh-AABB synthesis (D1).* + +**(b) The query predicate matching `FindObjCollisions@0x0050f050`:** instant-skip iff (0x4 AND 0x10); 0x4-alone → set `ObstructionEthereal` + continue (D2/D5); viewer/creature; IGNORE_CREATURES/creature; PvP pool; then dispatch BSP-only iff `HAS_PHYSICS_BSP_PS && !pvpExempt && !missileIgnore` else CylSphere→**Sphere** (D3). Wire the PvP/missile terms (W1). + +**(c) One overlap-based per-cell registration** (already mostly present via `ShadowObjectRegistry` + the A6.P4/BR-7 flood) — **no attribute filter at registration**; everything funnels through it (statics, scenery, weenies, building shell, cell BSP). + +**(d) One per-apply rebase** — every cached collision transform (cells, buildings, shadow positions) rebased to the current streaming center on each per-landblock apply via a symmetric `Remove*ForLandblock` pre-clear. *Add `RemoveCellsForLandblock` (D8).* (Or fold into the #145 cell-relative-frame port so collision frames can never go stale — the handoff's preferred long-term home.) + +### Deletions +- `GameWindow.cs:7408-7565` mesh-AABB synthesized-cylinder fallback (D1). +- `CollisionExemption.cs:78` ETHEREAL-alone instant-skip → replace with both-bits Gate-1 + `ObstructionEthereal`-set-and-continue (D2); move ACE door-open compat to the **wire/adaptation layer**, not the collision predicate. +- `ShadowShapeBuilder.cs:68-81` sphere→cylinder coercion → true Sphere shape + primitive (D3). +- `TransitionTypes.cs:2629` hardcoded `pvpExempt/missileIgnore=false` → wire real terms (W1). + +### Risks (each needs a user visual gate) +- **D1 removal makes shape-less scenery walk-through** — retail-faithful, but a *visible behavior change* (trees you currently bump may become passable; trees with real canopy-only BSP regain retail behavior). **Highest-risk item; gate carefully.** +- **D2 re-solidifies ACE-opened doors** unless the door-open wire path sends `ETHEREAL|IGNORE_COLLISIONS` in the **same** change — must land together or doors become impassable. +- **D2/D5 obstruction_ethereal** touches BSP solid-containment gates — port set+clear+consume **together**; a partial port hardens/weakens walls unpredictably. +- **W1 PvP/missile wiring** has no live M1.5 test path — risk of regressing the A6.P7 BSP-only door dispatch; gate + conformance-test the door case. +- **D3 sphere primitive** is new physics code — conformance-test vs `CSphere::intersects_sphere` golden values, not eyeballed. +- **D8 eviction** interacts with #135/#138 dungeon collapse/recenter — adding eviction could surface a re-stream ordering bug currently masked by "never evict"; verify against teleport-OUT. + +### Suggested slicing (for the brainstorm) +1. **Shape-source purity (D1)** — delete mesh-AABB synthesis; the highest-value, most-isolated, retail-clarifying change. Visual gate: scenery + Holtburg. +2. **Sphere primitive (D3)** — unblocks faithful shape dispatch; conformance-tested. +3. **ETHEREAL/obstruction_ethereal (D2+D5)** — predicate + state field + door wire compat, landed together. +4. **Cell-transform rebase (D8)** — or fold into the #145 cell-relative-frame phase. +5. **Predicate wiring + entry-restrictions (W1+D4)** — lowest urgency (no M1.5 effect); wire as conservative no-ops now, activate when PK/missiles/locked-cells ship. + +--- + +## Divergence-register bookkeeping required +- **Widen AP-2** to cite `GameWindow.cs:7408` (scenery mesh-AABB), not only the Setup case (D1). +- **AD-7** already covers D2 — keep; retire it when the obstruction_ethereal port lands. +- **AP-6** already covers the analytic-cylinder-vs-retail-CylSphere dispatch — relevant to D3/W1. +- **New rows:** D3 (sphere-as-cylinder), D4 (entry-restrictions omitted), D5 (obstruction_ethereal absent from SpherePath), D8 (cell-struct transform never evicted). Optional: a terrain-pass-through note for the D7 streaming-gap (clarify it's #135/#138, not a collision bug) so nobody re-files it. + +## Apparatus for the implementation phase (reuse, don't rebuild) +`ACDREAM_PROBE_BUILDING` (`[entity-source]` BSP-vs-Cylinder classification per static — directly shows which objects got the D1 synthetic cylinder), `ACDREAM_CAPTURE_RESOLVE`, `Issue147ArwicBuildingsDumpTests` (dat-dump pattern), the `CellarUp*`/`HouseExitWalk*`/`CornerFlood*` replay harnesses, and a new `CSphere::intersects_sphere` conformance fixture for D3. + +--- +*Generated by a 19-agent verified workflow (`wf_b2f8da74-9fa`): 10 readers → synthesis → 8 adversarial verifiers. 6/8 candidate deviations survived adversarial verification; 2 were refuted (D6, D7) with the retail evidence corrected.* diff --git a/docs/superpowers/specs/2026-06-24-unified-collision-inclusion-design.md b/docs/superpowers/specs/2026-06-24-unified-collision-inclusion-design.md new file mode 100644 index 00000000..71cdce51 --- /dev/null +++ b/docs/superpowers/specs/2026-06-24-unified-collision-inclusion-design.md @@ -0,0 +1,223 @@ +# Unified verbatim-retail collision-inclusion — design spec — 2026-06-24 + +**Phase goal:** make acdream's "which objects collide, and how the engine decides that" +**verbatim with retail**, replacing the last non-faithful per-channel artifacts. World feel — +what you bump into vs. walk through — should match the retail client object-for-object. + +**Source of truth for the divergences:** [`docs/research/2026-06-24-collision-inclusion-audit.md`](../../research/2026-06-24-collision-inclusion-audit.md) +(19-agent verified audit: 6 confirmed deviations D1/D2/D3/D4/D5/D8, 2 refuted D6/D7). +**Read first:** `claude-memory/project_physics_collision_digest.md` (DO-NOT-RETRY table). + +**Branch:** `claude/thirsty-goldberg-51bb9b`. Named-retail + `references/` live in the MAIN repo. + +--- + +## 1. The retail model we are matching (two layers, four channels) + +Retail decides collision in two layers (verified addresses in the audit): + +- **Layer B — registration (broad phase).** Every object that has a *DAT physics shape* + registers a shadow into every cell its sphere overlaps (`CObjCell::find_cell_list@0x0052b4e0`, + `add_shadows_to_cells@0x00514ae0`). **No attribute is tested at registration** (only + `PARTICLE_EMITTER_PS 0x1000` diverts). "Registerable" ⇔ "has a DAT shape". +- **Layer A — inclusion predicate (narrow phase).** `CPhysicsObj::FindObjCollisions@0x0050f050` + decides per-query skip/dispatch via flag logic, then dispatches **one** shape branch + (BSP-only iff `HAS_PHYSICS_BSP_PS 0x10000`, else CylSphere-then-Sphere). + +The object's **shape comes from DAT only** — a part's `physics_bsp` (`CGfxObj::Serialize@0x00534970`, +gated on serialized-flags bit-0) or `CSetup` cylsphere/sphere (`CPartArray::InitParts@0x00517F40`). +No shape ⇒ `CPhysicsPart::find_obj_collisions@0x0050D8D0` returns OK (passable). **Render-mesh bounds +never enter collision.** + +The four channels stay **distinct** (retail keeps them separate; we do too): +terrain (`CLandCell::find_env_collisions`), EnvCell (`CEnvCell::find_env_collisions@0x0052c130`, +**no building leg**), building shell (`CBuildingObj::find_building_collisions@0x006b5300`, +portal-independent), object shadows (`CObjCell::find_obj_collisions@0x0052b750` → Layer A). + +acdream **already has this two-layer shape** (`ShadowObjectRegistry` + `CollisionExemption` + +`ShadowShapeBuilder` + the A6.P4/BR-7 flood). Verified faithful: the DAT loader reads +`gfxObj.PhysicsBSP` (gated on `HasPhysics`) + `setup.CylSpheres`/`setup.Spheres` +([`PhysicsDataCache.cs:61-68,157-158`](../../../src/AcDream.Core/Physics/PhysicsDataCache.cs)). +This spec closes the remaining gaps. + +--- + +## 2. Target architecture (the "1 fix" mapped onto retail's four pieces) + +1. **One DAT-only collision-shape authority.** `ShadowShapeBuilder` becomes the **single** place + an object's collision shape is derived — from `PhysicsBSP` / `CylSpheres` / `Spheres` only. + Every object-registration site (statics, scenery, weenies, building shells) funnels through it. + The inline mesh-AABB synthesis is **deleted** from `GameWindow.cs`. Because there is exactly one + shape site, the DAT-only rule cannot be violated per-site, and the W3 radius inconsistency + (`2f` vs `1f` fallback) dies. *(Satisfies CLAUDE.md Structure Rule #1 — removes a ~150-line + feature body from the 10k-line `GameWindow.cs` god-object.)* + +2. **One query-time object predicate.** `CollisionExemption` (the skip gate) + the cyl/sphere/BSP + dispatch form the object channel's Layer A, completed with the `ObstructionEthereal` state + (D2/D5) and the PvP/missile dispatch terms wired (W1). + +3. **Env / building / terrain stay distinct channels** (retail-faithful), each with its specific + deviation fixed in place. + +4. **One per-apply rebase invariant:** *every cached collision transform rebases to the current + streaming center on each landblock apply.* Buildings already comply (`RemoveBuildingsForLandblock`, + #146); cells get the symmetric `RemoveCellsForLandblock` (D8); shadow positions are already absolute. + +**What we do NOT do:** no `ObjectCollides` facade migrating all four channels onto one predicate +(retail keeps `find_env`/`find_building`/`find_obj` separate — collapsing them is *less* faithful). +No new abstraction layer; we complete the units that already implement retail's model. + +### Component boundaries + +| Unit | Responsibility | Depends on | Layer | +|---|---|---|---| +| `ShadowShapeBuilder` (extended) | **Sole** DAT-only collision-shape authority: object → `{BSP \| CylSphere[] \| Sphere[] \| None}`. No render bounds. | `PhysicsDataCache` (DAT shapes) | Core | +| `ShadowObjectRegistry` (unchanged) | Per-cell shadow lists; overlap-based flood registration (BR-7) | shape rows from builder | Core | +| `CollisionExemption` (extended) | Query-time skip gate + ObstructionEthereal decision | `PhysicsState`, `EntityCollisionFlags`, `ObjectInfoState` | Core | +| `SpherePath` (extended) | carries `ObstructionEthereal` state across the transition | — | Core | +| `BSPQuery` (extended) | sphere primitive; consume `ObstructionEthereal` in the solid-containment gate | shape rows | Core | +| `PhysicsDataCache` (extended) | `RemoveCellsForLandblock` symmetric with buildings | — | Core | +| `TransitionTypes` (edited) | EnvCell entry-restrictions + obstruction_ethereal clear; dispatch wiring | above | Core | +| `GameWindow` (reduced) | registration sites call the builder; **mesh-AABB block deleted** | builder | App | + +--- + +## 3. Slices (each: build+test green, conformance where applicable, its own visual gate) + +Collision is the DO-NOT-RETRY area — **grep named-retail → pseudocode → port → conformance-test** +for every slice; no guess-patches. Each slice is one-to-few commits and an independent visual gate. + +### Slice 1 — DAT-only shape authority + delete mesh-AABB (D1, D3-prep, W3) +- **Retail oracle:** `CPartArray::InitParts@0x00517F40`, `CGfxObj::Serialize@0x00534970` (physics_bsp + on flags bit-0), `CPhysicsPart::find_obj_collisions@0x0050D8D0` (null shape ⇒ OK/passable). +- **Change:** Make `ShadowShapeBuilder` the single object-shape entry point. **Delete** the mesh-AABB + synthesis block at [`GameWindow.cs:7408-7565`](../../../src/AcDream.App/Rendering/GameWindow.cs) + and the `IsPhantomGfxObjSource`/`isPhantomSetup` gates that only existed to fence it (the phantom + concept becomes the default: no DAT shape ⇒ no shape). Unify the static/live/scenery registration + sites to one builder call. Remove `ComputeVisualBounds` use *for collision* (visual bounds stay for culling). +- **Shape vs broad-phase sphere (do not conflate):** the *collision shape* is DAT-only and may be `None` + (object registers no shape, is passable — no synthetic radius). The *broad-phase registration sphere* + (which cells the flood touches) derives from the shape's own bounding sphere; a shapeless object does not + register at all. Kill the W3 fallback-radius (`2f` vs `1f`) by sourcing the registration radius from the + shape's bounding sphere, never a synthetic constant — and never let a render-mesh bound produce a + *collision* shape (a culling AABB is fine). +- **Acceptance:** every object's collision shape originates from `PhysicsBSP`/`CylSpheres`/`Spheres`; + an object with none registers no shape. `dotnet build`/`dotnet test` green. Existing replay + harnesses (`CellarUp*`, `HouseExitWalk*`, `CornerFlood*`) unchanged-green. +- **Register:** **delete** AP-2 (the divergence is gone). Note AP-6 still covers analytic cylinder math. +- **Visual gate:** walk Holtburg + open world; objects that become passable must be exactly those + retail walks through (DAT has no physics shape). The `ACDREAM_PROBE_BUILDING` `[entity-source]` + line (BSP-vs-Cylinder per static) is the lens. + +### Slice 2 — true Sphere collision primitive (D3) +- **Retail oracle:** `CSphere::intersects_sphere` (pc:276917), `CPartArray::GetSphere@0x00518070`. +- **Change:** add a `Sphere` variant to `ShadowCollisionType` (`ShadowObjectRegistry.cs:537`) and a + real sphere-vs-sweep primitive in `BSPQuery`; `ShadowShapeBuilder` emits `Sphere` (not coerced + cylinder) for `Setup.Spheres`. Dispatch order matches retail: `HAS_PHYSICS_BSP` clear ⇒ CylSphere + then Sphere. +- **Acceptance:** new conformance fixture vs `CSphere::intersects_sphere` golden values (port the + decomp's intersection math; do not eyeball). Build/test green. +- **Register:** D3 is a pre-existing *unregistered* gap being fixed; no row needed (fixed before it had one). +- **Visual gate:** an object whose Setup carries a Sphere (not CylSphere) collides correctly — pick a + known sphere-shape object from the Slice-1 inventory probe. + +### Slice 3 — ObstructionEthereal verbatim port (D2 + D5) — highest care +- **Retail oracle:** `FindObjCollisions@0x0050f050` Gate-1 (pc:276782, requires `0x4 AND 0x10`), + ETHEREAL-alone branch (pc:276795-276806, sets `obstruction_ethereal=1` and continues), the + per-call clear in `CEnvCell::find_env_collisions@0x0052c144` (pc:309580), and the consume sites + in the BSP solid-containment gate (pc:321692 / 323742 / 324573: `if (obstruction_ethereal || insert_type==PLACEMENT)`). +- **Change (port set + clear + consume *together*):** + 1. add `ObstructionEthereal` field to `SpherePath`; + 2. `CollisionExemption`: instant-skip only when `0x4 AND 0x10`; on `0x4`-alone set + `ObstructionEthereal=true` and **continue** to shape test (remove the AD-7 `0x4`-alone return); + 3. clear `ObstructionEthereal=0` at the front of the EnvCell **and** LandCell env dispatch + (this is D5 — the clear that currently doesn't exist because the field doesn't exist); + 4. consume it in `BSPQuery`'s placement/solid-containment gate alongside the existing + `InsertType.Placement` check. +- **The AD-7 shim is subsumed, not patched around.** ACE broadcasts ETHEREAL-only (`0x0001000C`) + for opened doors; the faithful obstruction_ethereal path makes that door passable the retail way, + so **no wire-layer compat is needed** — verify this is what happens, do not add a shim. +- **Acceptance:** build/test green; a door-collision replay (reuse `DoorCollisionApparatusTests` / + the flipped-door apparatus) shows ETHEREAL-alone → passable, ETHEREAL-absent → solid; wall + collision on non-ethereal cells **unchanged** (the consume site must not weaken normal walls). +- **Register:** **delete AD-7** (divergence retired). +- **Visual gate:** **the critical one** — open a door in-world (ACE), walk through it (passable), + close it, walk into it (solid); confirm no house wall went soft. Live: `notan`/`MittSnus81!`. + +### Slice 4 — RemoveCellsForLandblock (D8) — the anti-staleness invariant +- **Retail oracle:** the #146 pattern (`RemoveBuildingsForLandblock`, `PhysicsDataCache.cs:472`); + retail rebuilds cell transforms from live cell pointers (eviction analog `CEnvCell::release`). +- **Change:** add `PhysicsDataCache.RemoveCellsForLandblock(landblockId)` (drop `_cellStruct` entries + whose prefix matches), call it from `PhysicsEngine.RemoveLandblock` (`:121-141`) alongside the + existing `_landblocks`/`ShadowObjects`/`CellGraph` clears, so the next `CacheCellStruct` re-bases the + cell BSP `WorldTransform` against the current `_liveCenter`. Completes invariant #4 (all cached + collision transforms rebase per apply). +- **Acceptance:** build/test green; a teleport-OUT then re-stream of the same dungeon yields a fresh + cell transform (add a targeted test mirroring the building re-base test). Interacts with #135/#138 + — verify no re-stream ordering regression. +- **Register:** new row not needed (divergence eliminated, not introduced). +- **Visual gate:** teleport into a dungeon → out → back into the same dungeon; indoor walls collide + correctly each time (no stale-offset clip). + +### Slice 5 — entry-restrictions hook + predicate wiring (D4 + W1) — conservative +- **Retail oracle:** `CEnvCell::find_env_collisions@0x0052c130` (calls `check_entry_restrictions` + first, pc:309576), `CObjCell::check_entry_restrictions@0x0052b6d0`; dispatch terms at pc:276861. +- **Change:** add the `check_entry_restrictions` gate at the front of the EnvCell/LandCell env path + (returns OK when no `restriction_obj` — a no-op in current dev content but structurally present); + wire the real `pvpExempt`/`missile_ignore` terms into the cyl-vs-BSP dispatch + (`TransitionTypes.cs:2629`) instead of the hardcoded `false`. +- **Acceptance:** build/test green; no behavior change in dev content (no locked cells, no PK/missiles). + Conformance-test the dispatch against the door BSP-only case (must not regress A6.P7). +- **Register:** D4 gets a one-line "ported, inert until locked-cell content" note or no row if fully + faithful; W1 wiring removes the "hardcoded false" smell. +- **Visual gate:** none required (no live trigger in M1.5) — rides the suite + the Slice-3 door gate. + +--- + +## 4. Testing strategy + +- **Conformance tests** (Core): Slice 2's `CSphere::intersects_sphere` golden-value fixture is the + load-bearing one (new physics math). Slices 3/5 reuse the door apparatus. +- **Replay harnesses** (regression guard, every slice): `CellarUpTrajectoryReplayTests`, + `HouseExitWalk*`, `CornerFlood*`, `Issue147ArwicBuildingsDumpTests` must stay green — they pin the + shadow-list/building/cell machinery this phase touches. +- **Live capture** when a slice needs evidence: `ACDREAM_PROBE_BUILDING` (`[entity-source]` + BSP-vs-Cylinder per object — the Slice-1 lens), `ACDREAM_CAPTURE_RESOLVE` (per-resolve JSONL), + short targeted runs only. +- **Visual gates** are the acceptance test for feel — Slices 1, 3, 4 each require the user's eyes + (the only thing that genuinely requires stopping). Slices 2, 5 ride the suite. + +## 5. Risks (each gated) + +- **Slice 1 changes scenery feel** — objects with no DAT shape become passable. Retail-faithful *by + construction* (loader verified faithful, §1), but visible; the Holtburg/open-world gate confirms the + passable set matches retail. +- **Slice 3 is the riskiest** — `obstruction_ethereal` touches the BSP solid-containment gate. A + partial port (set-but-not-consumed, or cleared inconsistently) hardens or softens walls + unpredictably. Mitigation: port set+clear+consume in one slice; door gate + "walls unchanged" check. +- **Slice 4 interacts with dungeon collapse/recenter (#135/#138)** — eviction could surface a + re-stream ordering bug currently masked by "never evict." Mitigation: teleport-OUT gate. +- **Slice 5 wiring** has no live M1.5 trigger — risk is regressing the A6.P7 BSP-only door dispatch. + Mitigation: conformance-test the door case before/after. + +## 6. Divergence-register bookkeeping (per CLAUDE.md — same-commit rule) +- **Delete** AP-2 (Slice 1), AD-7 (Slice 3) — divergences retired, rows removed in the landing commit. +- **Keep** AP-6 (analytic cylinder math vs retail CylSphere — unchanged this phase). +- **Add** a clarifying note (optional) that terrain pass-through on an unstreamed block is the + #135/#138 streaming-gap (D7 refuted), not a collision divergence — so nobody re-files it. + +## 7. Out of scope (filed, not touched) +- **W2** — `BldPortalInfo.ExactMatch` decoded-but-unconsumed: a *transit-feature*, not collision-inclusion. +- **W4** — `CLandCell` ENTIRELY_WATER skip clause: revisit before deep-water content. +- **W5** — static-prune `VisibleCellIds` vs retail `do_not_load` stab_list: verify before it bites. +- **The full #145 cell-relative-frame port** — the eventual "collision frames can never go stale by + construction" home; Slice 4's `RemoveCellsForLandblock` is the interim faithful-enough invariant and + is a clean delete when #145 lands. +- **D6 / D7** — refuted by the audit's adversarial pass; not deviations. + +## 8. Definition of done +All five slices landed; AP-2 + AD-7 deleted from the register; conformance + replay suites green; +the three feel-gates (Slice 1 scenery, Slice 3 door, Slice 4 dungeon re-entry) passed by the user. +Collision-inclusion is verbatim with retail: an object collides iff retail would collide with it, +decided by one DAT-only shape authority + one query predicate + four faithful channels + one +anti-staleness invariant.