acdream/docs/reports/2026-07-28-campaign-v-report.md

# Campaign V — OpenGL → Vulkan migration, results so far

**Date:** 2026-07-28
**Plan:** [`docs/plans/2026-07-27-vulkan-campaign.md`](../plans/2026-07-27-vulkan-campaign.md)
**Issues filed:** [`docs/ISSUES.md`](../ISSUES.md) #248–#253

acdream's mandatory modern renderer was built on GL 4.3 core +
`ARB_bindless_texture` + `ARB_shader_draw_parameters`, an API that is no longer
evolving and whose hardware floor is narrow — bindless textures are absent on
Intel integrated GPUs and on every Mesa software/D3D12 stack, which is exactly
what parked the Linux graphical slice (Slice L) at its L1 checkpoint. Campaign
V replaces it with a single Vulkan 1.3 backend, keeping the pinned Vulkan-shaped
RHI contract that a GL implementation satisfied first. It is **a compatibility
and efficiency campaign, not a rescue** — the GL path worked and was fast;
nothing about what the game looks like was supposed to change, and the campaign
spent most of its effort proving that it didn't. As of this report: every
production renderer draws on both backends, the GL-versus-Vulkan differential
is inside its acceptance threshold on deterministic content, the user has run
one full connected session on Vulkan and signed off on it, and the performance
gate (V8) is running.

---

## The screenshots

**V6c — first content on Vulkan.** The synthetic bring-up scene: colour
quadrants, a chequerboard, a gradient circle. This is the first frame the
Vulkan RHI ever produced — proof that the SPIR-V toolchain, pipelines, passes,
push constants and swapchain present all work together, before any production
renderer touched the backend.

![V6c: first Vulkan content](assets/campaign-v/v6c-vulkan-first-scene.png)

**V6d — the retained UI on Vulkan.** Same synthetic scene, now composited
underneath acdream's real `TextRenderer`/`DebugLineRenderer` output ("acdream —
retained UI on Vulkan / top-left origin — slice V6d"). This is the first
acdream-authored draw call on the new backend, proving the UI's fill+border+
glyph path was already backend-neutral.

![V6d: retained UI on Vulkan](assets/campaign-v/v6d-vulkan-retained-ui.png)

**V6j — Dereth draws on Vulkan (the centerpiece).** `ACDREAM_RENDER_BACKEND=vulkan`
rendering the full offline scene: blended terrain with road overlays, the water
edge and its shoreline, static world meshes with correct silhouettes and closed
shells, procedural scenery, per-instance lighting, and the complete retained UI
including the radar. This is the slice where the world arm landed — three
renderers' RHI submission arms, a backend-neutral pass-executor pair, and a
winding-inversion bug (below) fixed on the same commit. Sky is still flat fog
here; `SkyRenderer` doesn't cross until V6k.

![V6j: Dereth on Vulkan](assets/campaign-v/v6j-dereth-vulkan.png)

**V6k — the sky lands.** Same view, now with the dome, the horizon band, the
cloud sheet and the fog gradient rendered by the Vulkan arm — the same place
and colours as the GL capture of the same scene. This slice also discharged
§5.4's `BeginPass`-on-a-null-target divergence and moved the paperdoll/appraisal
viewports onto `IGpuRenderTarget`.

![V6k: sky rendered on Vulkan](assets/campaign-v/v6k-sky-vulkan.png)

**V6m — the retail portal tunnel on Vulkan.** One frame from a ten-frame transit
capture: the same tube, the same blue-grey shading and drifting purple motes,
the retained UI composited above it, with an opaque black surround and no world
bleeding through (the pass clears colour rather than loading it — a Vulkan-forced
decision documented in §5.5.18). `PortalTunnelPresentation` was the last raw-GL
world-adjacent renderer; this frame is its Vulkan arm.

![V6m: portal tunnel on Vulkan](assets/campaign-v/v6m-portal-tunnel-vulkan.png)

**V6l — the paperdoll on Vulkan.** The inventory/paperdoll panel open on the
Vulkan arm, rendered through the private entity viewport with correct pose,
orientation and alpha. This exercises the offscreen-render-target path the
backbuffer-only captures above don't reach.

![V6l: paperdoll on Vulkan](assets/campaign-v/v6l-paperdoll-vulkan.png)

**The GL-versus-Vulkan difference map.** Red channel encodes per-pixel delta
between a GL and a Vulkan capture of the same Holtburg scene (log scale, mostly
black). The visible structure is exactly the two populations the differential
work isolated: a thin band across the treeline/roofline near the top, and a
scatter of isolated dots at building corners and cloud edges — not a diffuse,
whole-frame shift. That shape is what let the investigation rule out a global
cause (gamma, half-pixel offset, depth precision) and converge on anisotropic
filtering as the remaining explanation (see the verdict tables below).

![GL vs Vulkan difference map](assets/campaign-v/v6k-diff-gl-vs-vulkan.png)

---

## The blank-world investigation

V4c and V4d — the slices that were meant to move the world draw path onto the
RHI on GL first — shipped, passed their offline pixel gate, and then produced a
**blank world** in the first connected sessions: UI rendered, sounds played, the
log was clean, `world-reveal` reported `visible=True`, and the user's own screen
surfaced one AMD driver-level timeout during testing. Both slices were reverted.
What followed was six investigation cycles (§5.5.1–§5.5.6 of the plan), each one
correcting the cycle before it.

**Cycle 1** measured the defect honestly for the first time: interleaved A/B
runs of the same binary (to share whatever machine-state drift had been
confounding earlier attributions) found **4/5 blank in the V4c arm versus 0/5 in
the parent arm, p ≈ 0.024** — real, but intermittent. It also found the first
broken instrument: the client's own screenshot read a 4×-multisampled default
framebuffer with `glReadPixels`, which is undefined behavior per the GL spec.

**Cycle 2** tested every "shared 3-D state" theory that could explain "world
dies, UI survives" — depth plane, camera constants, clip distances, GPU context
reset — and found all four **bit-identical** between blank and rendered frames.
It also found a second broken instrument: the in-process capture of framebuffer
0 returned all-zero pixels on a blank run, even over screen regions the desktop
compositor plainly showed the UI drawing into. Screenshot bytes were reporting
the readback, not the renderer.

**Cycle 3** built an occlusion-query instrument to settle whether draws were
executing and being discarded, or never executing at all — and found a third and
fourth broken instrument in the process: a CPU-side query read
(`glGetQueryObject`) **deadlocked the render thread inside the driver** on every
attempt, and routing the same query through a GPU-timeline `GL_QUERY_BUFFER`
write showed the write itself **never landing** on a blank run. The verdict: the
GPU renders the first world frame identically to the working build — the same
1,692,830 terrain samples and 317,561 entity samples — and then every GPU→CPU
reporting channel dies at once, permanently, while presentation and fence
signalling keep running normally.

**Cycles 4–5** laid out three ways forward and picked the cheapest discriminator:
run the identical ported world path on Vulkan, on the same GPU. If it renders
clean there, the defect is the AMD GL driver's, not the ported code's — settled
in one measurement instead of a multi-hour bisect that would keep pinning the
user's machine.

**Cycle 6** ran that measurement — first as a cross-vendor check while Vulkan
bring-up was still in flight: the exact V4c binary, ten repeat-connected runs,
on a separate NVIDIA machine against the same server, same account, same
worst-case cell. **10/10 rendered**, bounding the NVIDIA failure rate below ~4%
against a measured 30%+ on the AMD box. Same binary, same server, same scene,
two drivers — only AMD's failed, and it failed below the API in four
independent instruments. The driver-defect conclusion was adopted: the GL
backend keeps the legacy raw-GL world path as a scoped, documented exception to
the no-fork rule, and V4c/V4d's content returns as the Vulkan world path
instead (landed at V6j — see the screenshots above).

**Falsified along the way:** the frame ring's `glBufferSubData` hazard, a
capture-FBO binding leak, reveal ordering, shared 3-D state, and CPU-side
visibility — five mechanisms, none of them the cause. **Broken along the way:**
the multisampled `glReadPixels`, the in-process backbuffer capture, the
CPU-blocking query read, and the GPU-timeline query write — four instruments
that were measuring the wrong thing, all on the same driver.

---

## The verdict tables

**V7's per-stop differential**, full connected route, both backends, tolerance
2 / fraction 0.001, MSAA off, day group 0, world time pinned to 0.5, sky phase
pinned to 0 (`artifacts/v7-diff-c2`):

| Stop | Differing px | Fraction | Max Δ | Verdict |
|---|---|---|---|---|
| `holtburg_town` | 26,330 | 2.86e-02 | 255 | **EXCEPTION** — phase + AD-46 |
| `facility_hub_interior` | 7,176 | 7.79e-03 | 255 | **EXCEPTION** — phase |
| `aerlinthe_island` | 62,892 | 6.82e-02 | 255 | **EXCEPTION** — AD-46 + dark-scene floor |

No connected stop passes yet, and none of the three exceptions is a renderer
defect: `holtburg_town`'s residual is an animated portal, two wandering NPCs, a
smoke plume, and vitals that genuinely regenerated between two logins minutes
apart, plus the treeline; `facility_hub_interior`'s entire residual is the local
player's idle pose and its lighting — walls, floor and doorway are pixel-black
in the difference map; `aerlinthe_island`'s residual is half AD-46 shrubbery and
half an artifact of an *absolute* tolerance against a scene averaging 28/255
luminance, where half the differing pixels are exactly one step over threshold.

**The offline GL-versus-Vulkan pair**, both clocks pinned, MSAA off — the
instrument the investigation converged on, because the offline scene has no
session, no server, no entities, no camera settle and no wandering NPCs, so a
difference is the renderer or it is nothing:

| Pair | Differing px | Fraction |
|---|---|---|
| GL vs GL, same commit (control) | 1,966 | 2.13e-03 |
| Vulkan vs Vulkan, same commit (control) | 1,039 | 1.13e-03 |
| **GL vs Vulkan, whole frame** | 28,807 | 3.13e-02 |
| **GL vs Vulkan, below the tree band (rows 280–720)** | **497** | **8.82e-04 — inside the 0.001 threshold** |

Terrain, terrain blending, roads, the water edge, fog, statics, scenery below
the horizon, and the entire retained UI are at parity. Everything outside that
threshold lives in one population: the treeline. Four hypotheses were tested —
sub-pixel offset (refuted: best alignment is (0,0)), sharpness/LOD mismatch
(refuted: high-frequency energy matches within 2–5%), depth precision (refuted:
forcing GL's depth-range mapping onto Vulkan moved the number by only 3%), and
anisotropic filtering (confirmed: going from anisotropy 1 to 16 — GL's and
retail's own value — nearly halved the band, and no knob is left once both arms
request the same setting). What remains is the anisotropic tap pattern itself,
which both GL and Vulkan specs leave implementation-defined and AMD's own two
drivers don't agree about. Filed as **`AD-46`** in the
[divergence register](../architecture/retail-divergence-register.md), dormant
until V10.

---

## The defects the migration found in its own code

Each of these compiled clean, passed validation, and had a test — and each was
still wrong, caught only when a slice's pixel gate finally exercised the path
that mattered. That's the campaign's per-slice discipline working as intended:

- **Winding inversion (V6j).** `VulkanViewportMapping` had inverted the front
  face since V6c on the standard negative-viewport argument, but nothing before
  the world arm actually culled anything to test it. The world arm was its
  first culling consumer, and it was wrong twice on one frame: terrain vanished
  outright, and every closed building shell rendered inside-out with its front
  wall culled and its interior beams visible through the gap.
- **Stride mismatch (V6k).** The Vulkan sky's vertex layout declared a 32-byte
  stride (position, normal, texcoord) while the actual CPU vertex record is
  36 bytes — it carries a fourth field the sky shader never reads but the GL
  arm's `sizeof(Vertex)` always counted. The dome came up as a field of
  blue-white noise until the layout stated the record's real footprint.
- **Sampler overrides (V7).** The Vulkan world texture arrays asked for
  `MaxAnisotropy: 1` where the GL arm reads the driver's actual capability
  twice over — and retail itself sets every sampler stage to the device's
  reported maximum. Fixing it to request the device ceiling halved the
  treeline difference band and dropped everything below it by two-thirds.
- **A one-binding, two-renderer hazard (V6i-1).** A single descriptor binding
  would have let the GL particle draw clobber `WbDrawDispatcher`'s instance
  data mid-frame the moment both renderers were active in the same pass. Closed
  before the world arm could ever exercise it, with one descriptor-set pair per
  renderer scope derived from the descriptor state itself rather than declared
  by hand.
- **MSAA readback undefined behavior (the blank-world investigation).** Every
  pixel gate and every blank/rendered verdict in the campaign, until this was
  found, read a 4×-multisampled default framebuffer directly with
  `glReadPixels` — undefined per the GL spec. The capture now resolves to a
  single-sampled framebuffer first; re-measuring the noise band afterward
  showed the fix itself moved the offline gate by 23 pixels out of 563,200,
  indistinguishable from its own same-commit control.

---

## Current status

> **Updated 2026-07-29.** The campaign is **complete**. V8 through V11 have all
> landed; the tree is statically green and OpenGL is gone. What remains is
> runtime evidence for V11, which is blocked by a **machine fault rather than a
> campaign result** — see below and
> [`2026-07-29-overnight-report.md`](2026-07-29-overnight-report.md) §1.1.

- **V8 (performance gate)**, **V9 (Linux CI)** and **V10 (cutover)** are
  **complete**. V8's first commit landed the missing instrument: the Vulkan arm
  had been silently wired to `NullRenderFrameGpuMeasurement` and emitted no
  `[frame-prof]` line at all, so the campaign's own performance tooling had
  nothing to point at it. V9's `linux-vulkan` job runs the probe on lavapipe
  under Xvfb, asserts a clean accepting verdict, checks the forced-unsupported
  exit-4 path, and verifies committed `.spv` binaries are byte-identical to a
  fresh compile. V10 made Vulkan the default backend behind the user's visual
  sign-off; the one open finding from that session is #253 below. The physical
  Linux GPU row and Wayland stay deferred on the same precedent as Slice L.
- **V11 (closeout)** is **deleted and statically green; runtime gates
  outstanding.** Five commits (`844cf092` → `c265b52d`) removed **27,607 lines
  against 1,870 added** across 204 files. `Gpu/Gl`, `OpenGLGraphicsDevice`, the
  `ManagedGL*` stack, `GLHelpers`, `GLStateScope`, both render-state caches,
  `BindlessSupport`, `GraphicalGlFunctionProbe`, the ImGui project, the Studio
  tree, the `ui-studio` verb, `RenderBackendKind` and the
  `ACDREAM_RENDER_BACKEND` escape hatch are all gone. `Silk.NET.OpenGL` and
  `.Extensions.ARB` are dropped — **final reference count zero**. 9/9 shader
  pairs recompile.

**Three findings from the deletion.** *Chorizite could not be dropped*, and not
for the reason the risk register predicted: `TextureFormat` sits in the
`IWorldTextureArray.CreateClampedArray` signature the **Vulkan** path
implements, and `BoundingBox` is a **serialized type in the pak format**, so
removing it touches the on-disk format and is its own slice.
*`Studio/SampleData.cs` was production code*, not devtools —
`InteractionRetainedUiComposition` uses it as the character sheet's fallback —
so it was moved, not deleted. And *one real bug fell out*:
`WbMeshAdapter.Dispose()` was still pattern-matching the deleted
`GpuFrameFlightController` to decide whether to wait for submitted GPU work.
`VulkanFrameFlightController` replaced that type at V6a and the site was never
updated, so **the wait had been silently dead on every Vulkan run since V6a**.
Deleting the GL type is what turned a no-op into a compile error.

**Why the runtime gates did not run.** The offline pixel gate died at startup in
`vkGetPhysicalDeviceSurfaceCapabilitiesKHR`. Bisecting put the fault outside the
tree — the client fails identically at a pre-V11 commit whose offline capture had
succeeded three hours earlier — and `vulkaninfo --summary`, a Khronos tool
containing no acdream code, fails at the same call. Win32 surface creation is
broken process-wide on this machine ([#259](../ISSUES.md)); Vulkan itself is
healthy and both adapters report 1.4. Nothing was relaxed to manufacture a pass
and nothing was declared green on a prediction — a gate that could not run is not
a gate that passed. The outstanding commands are listed in the overnight report;
the pre-deletion baseline for the self-differential is already on disk at
`artifacts/v11-pre`.

---

## Filed issues (#248–#253)

- **[#248](../ISSUES.md)** — `FrustumCuller` extracts its near plane with the
  GL `[-1,1]`-NDC formula against a `[0,1]`-NDC projection; harmless today
  (makes culling strictly more permissive near the eye) but wrong hygiene.
  **DONE 2026-07-29** — `near = Normalize(col3)`, pinned by a theory asserting
  the extracted near distance equals the camera's near value across four
  near/far pairs; the old formula fails all four. The gate half of its
  acceptance (offline pixel gate + connected route) is outstanding under #259.
- **#249** — Released bindless texture slots never call `MakeNonResident`
  before the texture is deleted, which `GL_ARB_bindless_texture` leaves
  undefined, and leaks a resident handle for the process lifetime per release.
  OPEN.
- **#250** — Four "zero managed allocation" tests fail intermittently, roughly
  one run in three, from tiered-JIT/GC measurement noise rather than a real
  regression — but an acceptance-gating test that flakes trains everyone to
  re-run until green. **DONE 2026-07-29**, and it was not noise: in all four the
  measured window was never the warmed path — two measured a thousand-iteration
  loop written inline (which on-stack replacement rewrites mid-flight, on the
  measuring thread), and two measured a code path the warmup had never taken.
  Fixed with a shared `ZeroAllocationProbe` that measures already-warmed batches
  and reports their minimum; **the zero bound is unchanged and the apparatus has
  its own tests proving it can still fail.** The family turned out to be ten
  tests, not four. 20/20 clean App-suite runs.
- **#251** — `glClientWaitSync` returned a fence status of `0` (not a value
  the API is specified to return) and crashed the render loop once in nine
  connected runs; possibly the same below-the-API driver family the
  blank-world investigation documented four other instances of. OPEN.
- **#252** — Ten App test classes raced on process-global `CameraDiagnostics`/
  `RenderingDiagnostics`/`Console.Out` statics under xUnit's parallel-by-default
  class execution. **DONE** — serialized via a `CameraDiagnosticsCollection`
  marker.
- **#253** — Attribute/skill icons in the character sheet sit off-center in
  their cells and render fully opaque, where retail uses vitae-window-like
  translucency; found during the user's mid-campaign Vulkan visual session,
  suspected pre-existing (the retained UI compares identical between backends
  at 3.94e-04) rather than a Vulkan regression. OPEN.