acdream/docs/research/2026-07-17-retail-world-selection-pseudocode.md
Erik 146a963aeb feat(selection): port retail polygon picking and vivid marker
Replace the projected Setup-sphere rectangle and independent physics-wall ray with retail's render-coupled picker: only visible server-object parts participate, each exact drawing sphere broad-phases the camera-eye ray, and first-in-DAT-order visual polygon hits globally outrank sphere fallbacks.

Replace the devtools-only procedural triangles with the retained gameplay VividTargetIndicator using retail client-enum surfaces 1..4, radar-blip colorization, Setup selection-sphere framing, and the exact eight-pixel viewport clamp.

Release build succeeds with zero warnings and all 5,886 tests pass with five intentional skips.

Co-authored-by: OpenAI Codex <codex@openai.com>
2026-07-17 21:32:51 +02:00

5.3 KiB

Retail world selection and vivid target indicator

Oracle: September 2013 EoR named retail client. Cross-checks: the extracted WorldBuilder mesh path (DAT geometry and current part transforms) and ACE object identity/qualities. Neither reference implementation supplies the client picker; the named retail functions below are authoritative.

Mouse selection

SmartBox::find_object @ 0x00451C60

remember the clicked viewport pixel
clear the previous per-frame selection accumulator
arm mouse selection for the next world draw

CPhysicsPart::Draw @ 0x0050D7A0, RenderDeviceD3D::DrawMesh @ 0x005A0860, and Render::GfxObjUnderSelectionRay @ 0x0054C740

for each physics part which reaches the normal visible-mesh draw path:
    reject parts which do not belong to a server object
    transform the one world selection ray into this part's local coordinates
    divide by the part's GfxObj scale (do not change the ray parameter units)

    sphere = gfxObj.drawing_sphere
    if sphere does not intersect the local ray:
        continue

    if a polygon has already won globally and sphereDistance is farther:
        continue

    if sphereDistance is nearer than the current sphere fallback:
        remember this object's id, part index, and sphereDistance

    for polygons in the GfxObj's stored flat DAT order:
        if polygon_hits_ray(polygon, localRay, out t):
            remember only this FIRST polygon hit for this part
            if t is nearer than the current global polygon winner:
                remember this object's id, part index, and t
            break

CPolygon::polygon_hits_ray @ 0x005395E0

if polygon.sides_type == ST_SINGLE (raw value 0)
and dot(ray.direction, polygon.plane.normal) > 0:
    miss                         // back face

denominator = dot(ray.direction, polygon.plane.normal)
if abs(denominator) < 0.0002:
    miss                         // parallel to the plane

t = plane/ray intersection time
if t < 0:
    miss                         // behind the viewer

point = ray.origin + t * ray.direction
normal = normalize(sum(cross(vertex[i] - vertex[0],
                             vertex[i+1] - vertex[0])))
for each ordered edge (previous -> current):
    inward = cross(normal, current - previous)
    if dot(point - previous, inward) < 0:
        miss
return hit

The broad-phase helper is retail CSphere::sphere_intersects_ray @ 0x005377A0: it rejects rays beginning in/on the sphere, requires squared ray direction length at least 0.0002, solves the quadratic in double precision, and chooses the near root when non-negative or the far root otherwise. The ray origin is Render::viewpoint (the camera eye), not the unprojected near-plane point; Render::pick_ray @ 0x0054B610 supplies only its direction.

Render::GetMouseSelectionObjectID @ 0x0054C950

if any polygon hit exists anywhere in the visible draw:
    return the closest polygon winner
if any drawing-sphere hit exists:
    return the closest sphere fallback
return no object

Important consequences:

  • The broad phase is each visible GfxObj part's drawing sphere, not a Setup selection sphere, physics collision shape, fixed creature hit box, or expanded screen rectangle.
  • A polygon hit on any visible part beats every sphere-only fallback.
  • Retail does not ray-test world-cell walls as a second, independent occluder. Occlusion is inherited from the same portal/viewcone draw traversal that decides which parts may call GfxObjUnderSelectionRay.
  • BSPTREE::GetSphere @ 0x005397E0 returns the drawing BSP root sphere exactly; the DAT source is GfxObj.DrawingBSP.Root.BoundingSphere.

Vivid target indicator

VividTargetIndicator::SetSelected @ 0x004F5CE0

if selected id is the player: clear it
if object is player-owned or IN_CONTAINER: clear it
publish the remaining id to SmartBox
if enabled and an id remains:
    color = gmRadarUI::GetBlipColor(selected id)
else:
    hide both indicator roots

VividTargetIndicator::Initialized @ 0x004F6C60

onscreenRoot = child 0x10000038
corners = children 0x10000039, 0x1000003A, 0x1000003B, 0x1000003C
offscreenRoot = child 0x10000045

The constructor resolves twelve source surfaces by calling DBObj::GetByEnum with RenderSurface DBO type 0x0C, client-enum category 0x10000009, and enum values 1..12. Values 1..4 are the four on-screen corners. CopyImage @ 0x004F5DD0 uses the active retail blit mode (s_BlitMethod == 4) to colorize the source art with the same color as the object's radar blip.

VividTargetIndicator::OnDraw @ 0x004F62B0 (on-screen branch)

if disabled, no selected id, or selected part was not reported by this world draw:
    hide indicator
else:
    colorize the four DAT corner images with the selected radar-blip color
    obtain the selected object's SmartBox screen rectangle
    use the actual DAT corner width and height
    place the corner images just outside the rectangle
    clamp the assembled indicator to an 8-pixel viewport margin
    show the on-screen root and hide the off-screen root

There is no selected-world-mesh tint or luminosity mutation in CPhysicsPart::Draw, ACCWeenieObject::SetSelected, or the world mesh draw path. The visible notification is the colorized four-corner DAT indicator. A mesh shader tint would therefore be a new effect, not a retail port.