acdream/docs/reports/2026-07-28-campaign-v-report.md

Campaign V — OpenGL → Vulkan migration, results so far

Date: 2026-07-28 Plan: docs/plans/2026-07-27-vulkan-campaign.md Issues filed: docs/ISSUES.md #248–#253

acdream's mandatory modern renderer was built on GL 4.3 core + ARB_bindless_texture + ARB_shader_draw_parameters, an API that is no longer evolving and whose hardware floor is narrow — bindless textures are absent on Intel integrated GPUs and on every Mesa software/D3D12 stack, which is exactly what parked the Linux graphical slice (Slice L) at its L1 checkpoint. Campaign V replaces it with a single Vulkan 1.3 backend, keeping the pinned Vulkan-shaped RHI contract that a GL implementation satisfied first. It is a compatibility and efficiency campaign, not a rescue — the GL path worked and was fast; nothing about what the game looks like was supposed to change, and the campaign spent most of its effort proving that it didn't. As of this report: every production renderer draws on both backends, the GL-versus-Vulkan differential is inside its acceptance threshold on deterministic content, the user has run one full connected session on Vulkan and signed off on it, and the performance gate (V8) is running.

---

The screenshots

V6c — first content on Vulkan. The synthetic bring-up scene: colour quadrants, a chequerboard, a gradient circle. This is the first frame the Vulkan RHI ever produced — proof that the SPIR-V toolchain, pipelines, passes, push constants and swapchain present all work together, before any production renderer touched the backend.

V6d — the retained UI on Vulkan. Same synthetic scene, now composited underneath acdream's real TextRenderer/DebugLineRenderer output ("acdream — retained UI on Vulkan / top-left origin — slice V6d"). This is the first acdream-authored draw call on the new backend, proving the UI's fill+border+ glyph path was already backend-neutral.

V6j — Dereth draws on Vulkan (the centerpiece). ACDREAM_RENDER_BACKEND=vulkan rendering the full offline scene: blended terrain with road overlays, the water edge and its shoreline, static world meshes with correct silhouettes and closed shells, procedural scenery, per-instance lighting, and the complete retained UI including the radar. This is the slice where the world arm landed — three renderers' RHI submission arms, a backend-neutral pass-executor pair, and a winding-inversion bug (below) fixed on the same commit. Sky is still flat fog here; SkyRenderer doesn't cross until V6k.

V6k — the sky lands. Same view, now with the dome, the horizon band, the cloud sheet and the fog gradient rendered by the Vulkan arm — the same place and colours as the GL capture of the same scene. This slice also discharged §5.4's BeginPass-on-a-null-target divergence and moved the paperdoll/appraisal viewports onto IGpuRenderTarget.

V6m — the retail portal tunnel on Vulkan. One frame from a ten-frame transit capture: the same tube, the same blue-grey shading and drifting purple motes, the retained UI composited above it, with an opaque black surround and no world bleeding through (the pass clears colour rather than loading it — a Vulkan-forced decision documented in §5.5.18). PortalTunnelPresentation was the last raw-GL world-adjacent renderer; this frame is its Vulkan arm.

V6l — the paperdoll on Vulkan. The inventory/paperdoll panel open on the Vulkan arm, rendered through the private entity viewport with correct pose, orientation and alpha. This exercises the offscreen-render-target path the backbuffer-only captures above don't reach.

The GL-versus-Vulkan difference map. Red channel encodes per-pixel delta between a GL and a Vulkan capture of the same Holtburg scene (log scale, mostly black). The visible structure is exactly the two populations the differential work isolated: a thin band across the treeline/roofline near the top, and a scatter of isolated dots at building corners and cloud edges — not a diffuse, whole-frame shift. That shape is what let the investigation rule out a global cause (gamma, half-pixel offset, depth precision) and converge on anisotropic filtering as the remaining explanation (see the verdict tables below).

---

The blank-world investigation

V4c and V4d — the slices that were meant to move the world draw path onto the RHI on GL first — shipped, passed their offline pixel gate, and then produced a blank world in the first connected sessions: UI rendered, sounds played, the log was clean, world-reveal reported visible=True, and the user's own screen surfaced one AMD driver-level timeout during testing. Both slices were reverted. What followed was six investigation cycles (§5.5.1–§5.5.6 of the plan), each one correcting the cycle before it.

Cycle 1 measured the defect honestly for the first time: interleaved A/B runs of the same binary (to share whatever machine-state drift had been confounding earlier attributions) found 4/5 blank in the V4c arm versus 0/5 in the parent arm, p ≈ 0.024 — real, but intermittent. It also found the first broken instrument: the client's own screenshot read a 4×-multisampled default framebuffer with glReadPixels, which is undefined behavior per the GL spec.

Cycle 2 tested every "shared 3-D state" theory that could explain "world dies, UI survives" — depth plane, camera constants, clip distances, GPU context reset — and found all four bit-identical between blank and rendered frames. It also found a second broken instrument: the in-process capture of framebuffer 0 returned all-zero pixels on a blank run, even over screen regions the desktop compositor plainly showed the UI drawing into. Screenshot bytes were reporting the readback, not the renderer.

Cycle 3 built an occlusion-query instrument to settle whether draws were executing and being discarded, or never executing at all — and found a third and fourth broken instrument in the process: a CPU-side query read (glGetQueryObject) deadlocked the render thread inside the driver on every attempt, and routing the same query through a GPU-timeline GL_QUERY_BUFFER write showed the write itself never landing on a blank run. The verdict: the GPU renders the first world frame identically to the working build — the same 1,692,830 terrain samples and 317,561 entity samples — and then every GPU→CPU reporting channel dies at once, permanently, while presentation and fence signalling keep running normally.

Cycles 4–5 laid out three ways forward and picked the cheapest discriminator: run the identical ported world path on Vulkan, on the same GPU. If it renders clean there, the defect is the AMD GL driver's, not the ported code's — settled in one measurement instead of a multi-hour bisect that would keep pinning the user's machine.

Cycle 6 ran that measurement — first as a cross-vendor check while Vulkan bring-up was still in flight: the exact V4c binary, ten repeat-connected runs, on a separate NVIDIA machine against the same server, same account, same worst-case cell. 10/10 rendered, bounding the NVIDIA failure rate below ~4% against a measured 30%+ on the AMD box. Same binary, same server, same scene, two drivers — only AMD's failed, and it failed below the API in four independent instruments. The driver-defect conclusion was adopted: the GL backend keeps the legacy raw-GL world path as a scoped, documented exception to the no-fork rule, and V4c/V4d's content returns as the Vulkan world path instead (landed at V6j — see the screenshots above).

Falsified along the way: the frame ring's glBufferSubData hazard, a capture-FBO binding leak, reveal ordering, shared 3-D state, and CPU-side visibility — five mechanisms, none of them the cause. Broken along the way: the multisampled glReadPixels, the in-process backbuffer capture, the CPU-blocking query read, and the GPU-timeline query write — four instruments that were measuring the wrong thing, all on the same driver.

---

The verdict tables

V7's per-stop differential, full connected route, both backends, tolerance 2 / fraction 0.001, MSAA off, day group 0, world time pinned to 0.5, sky phase pinned to 0 (artifacts/v7-diff-c2):

Stop	Differing px	Fraction	Max Δ	Verdict
holtburg_town	26,330	2.86e-02	255	EXCEPTION — phase + AD-46
facility_hub_interior	7,176	7.79e-03	255	EXCEPTION — phase
aerlinthe_island	62,892	6.82e-02	255	EXCEPTION — AD-46 + dark-scene floor

No connected stop passes yet, and none of the three exceptions is a renderer defect: holtburg_town's residual is an animated portal, two wandering NPCs, a smoke plume, and vitals that genuinely regenerated between two logins minutes apart, plus the treeline; facility_hub_interior's entire residual is the local player's idle pose and its lighting — walls, floor and doorway are pixel-black in the difference map; aerlinthe_island's residual is half AD-46 shrubbery and half an artifact of an absolute tolerance against a scene averaging 28/255 luminance, where half the differing pixels are exactly one step over threshold.

The offline GL-versus-Vulkan pair, both clocks pinned, MSAA off — the instrument the investigation converged on, because the offline scene has no session, no server, no entities, no camera settle and no wandering NPCs, so a difference is the renderer or it is nothing:

Pair	Differing px	Fraction
GL vs GL, same commit (control)	1,966	2.13e-03
Vulkan vs Vulkan, same commit (control)	1,039	1.13e-03
GL vs Vulkan, whole frame	28,807	3.13e-02
GL vs Vulkan, below the tree band (rows 280–720)	497	8.82e-04 — inside the 0.001 threshold

Terrain, terrain blending, roads, the water edge, fog, statics, scenery below the horizon, and the entire retained UI are at parity. Everything outside that threshold lives in one population: the treeline. Four hypotheses were tested — sub-pixel offset (refuted: best alignment is (0,0)), sharpness/LOD mismatch (refuted: high-frequency energy matches within 2–5%), depth precision (refuted: forcing GL's depth-range mapping onto Vulkan moved the number by only 3%), and anisotropic filtering (confirmed: going from anisotropy 1 to 16 — GL's and retail's own value — nearly halved the band, and no knob is left once both arms request the same setting). What remains is the anisotropic tap pattern itself, which both GL and Vulkan specs leave implementation-defined and AMD's own two drivers don't agree about. Filed as AD-46 in the divergence register, dormant until V10.

---

The defects the migration found in its own code

Each of these compiled clean, passed validation, and had a test — and each was still wrong, caught only when a slice's pixel gate finally exercised the path that mattered. That's the campaign's per-slice discipline working as intended:

• Winding inversion (V6j). VulkanViewportMapping had inverted the front face since V6c on the standard negative-viewport argument, but nothing before the world arm actually culled anything to test it. The world arm was its first culling consumer, and it was wrong twice on one frame: terrain vanished outright, and every closed building shell rendered inside-out with its front wall culled and its interior beams visible through the gap.
• Stride mismatch (V6k). The Vulkan sky's vertex layout declared a 32-byte stride (position, normal, texcoord) while the actual CPU vertex record is 36 bytes — it carries a fourth field the sky shader never reads but the GL arm's sizeof(Vertex) always counted. The dome came up as a field of blue-white noise until the layout stated the record's real footprint.
• Sampler overrides (V7). The Vulkan world texture arrays asked for MaxAnisotropy: 1 where the GL arm reads the driver's actual capability twice over — and retail itself sets every sampler stage to the device's reported maximum. Fixing it to request the device ceiling halved the treeline difference band and dropped everything below it by two-thirds.
• A one-binding, two-renderer hazard (V6i-1). A single descriptor binding would have let the GL particle draw clobber WbDrawDispatcher's instance data mid-frame the moment both renderers were active in the same pass. Closed before the world arm could ever exercise it, with one descriptor-set pair per renderer scope derived from the descriptor state itself rather than declared by hand.
• MSAA readback undefined behavior (the blank-world investigation). Every pixel gate and every blank/rendered verdict in the campaign, until this was found, read a 4×-multisampled default framebuffer directly with glReadPixels — undefined per the GL spec. The capture now resolves to a single-sampled framebuffer first; re-measuring the noise band afterward showed the fix itself moved the offline gate by 23 pixels out of 563,200, indistinguishable from its own same-commit control.

---

Current status

Updated 2026-07-29. The campaign is complete. V8 through V11 have all landed; the tree is statically green and OpenGL is gone. What remains is runtime evidence for V11, which is blocked by a machine fault rather than a campaign result — see below and 2026-07-29-overnight-report.md §1.1.

• V8 (performance gate), V9 (Linux CI) and V10 (cutover) are complete. V8's first commit landed the missing instrument: the Vulkan arm had been silently wired to NullRenderFrameGpuMeasurement and emitted no [frame-prof] line at all, so the campaign's own performance tooling had nothing to point at it. V9's linux-vulkan job runs the probe on lavapipe under Xvfb, asserts a clean accepting verdict, checks the forced-unsupported exit-4 path, and verifies committed .spv binaries are byte-identical to a fresh compile. V10 made Vulkan the default backend behind the user's visual sign-off; the one open finding from that session is #253 below. The physical Linux GPU row and Wayland stay deferred on the same precedent as Slice L.
• V11 (closeout) is deleted and statically green; runtime gates outstanding. Five commits (844cf092 → c265b52d) removed 27,607 lines against 1,870 added across 204 files. Gpu/Gl, OpenGLGraphicsDevice, the ManagedGL* stack, GLHelpers, GLStateScope, both render-state caches, BindlessSupport, GraphicalGlFunctionProbe, the ImGui project, the Studio tree, the ui-studio verb, RenderBackendKind and the ACDREAM_RENDER_BACKEND escape hatch are all gone. Silk.NET.OpenGL and .Extensions.ARB are dropped — final reference count zero. 9/9 shader pairs recompile.

Three findings from the deletion. Chorizite could not be dropped, and not for the reason the risk register predicted: TextureFormat sits in the IWorldTextureArray.CreateClampedArray signature the Vulkan path implements, and BoundingBox is a serialized type in the pak format, so removing it touches the on-disk format and is its own slice. Studio/SampleData.cs was production code, not devtools — InteractionRetainedUiComposition uses it as the character sheet's fallback — so it was moved, not deleted. And one real bug fell out: WbMeshAdapter.Dispose() was still pattern-matching the deleted GpuFrameFlightController to decide whether to wait for submitted GPU work. VulkanFrameFlightController replaced that type at V6a and the site was never updated, so the wait had been silently dead on every Vulkan run since V6a. Deleting the GL type is what turned a no-op into a compile error.

Why the runtime gates did not run. The offline pixel gate died at startup in vkGetPhysicalDeviceSurfaceCapabilitiesKHR. Bisecting put the fault outside the tree — the client fails identically at a pre-V11 commit whose offline capture had succeeded three hours earlier — and vulkaninfo --summary, a Khronos tool containing no acdream code, fails at the same call. Win32 surface creation is broken process-wide on this machine (#259); Vulkan itself is healthy and both adapters report 1.4. Nothing was relaxed to manufacture a pass and nothing was declared green on a prediction — a gate that could not run is not a gate that passed. The outstanding commands are listed in the overnight report; the pre-deletion baseline for the self-differential is already on disk at artifacts/v11-pre.

---

Filed issues (#248–#253)

• #248 — FrustumCuller extracts its near plane with the GL [-1,1]-NDC formula against a [0,1]-NDC projection; harmless today (makes culling strictly more permissive near the eye) but wrong hygiene. DONE 2026-07-29 — near = Normalize(col3), pinned by a theory asserting the extracted near distance equals the camera's near value across four near/far pairs; the old formula fails all four. The gate half of its acceptance (offline pixel gate + connected route) is outstanding under #259.
• #249 — Released bindless texture slots never call MakeNonResident before the texture is deleted, which GL_ARB_bindless_texture leaves undefined, and leaks a resident handle for the process lifetime per release. OPEN.
• #250 — Four "zero managed allocation" tests fail intermittently, roughly one run in three, from tiered-JIT/GC measurement noise rather than a real regression — but an acceptance-gating test that flakes trains everyone to re-run until green. DONE 2026-07-29, and it was not noise: in all four the measured window was never the warmed path — two measured a thousand-iteration loop written inline (which on-stack replacement rewrites mid-flight, on the measuring thread), and two measured a code path the warmup had never taken. Fixed with a shared ZeroAllocationProbe that measures already-warmed batches and reports their minimum; the zero bound is unchanged and the apparatus has its own tests proving it can still fail. The family turned out to be ten tests, not four. 20/20 clean App-suite runs.
• #251 — glClientWaitSync returned a fence status of 0 (not a value the API is specified to return) and crashed the render loop once in nine connected runs; possibly the same below-the-API driver family the blank-world investigation documented four other instances of. OPEN.
• #252 — Ten App test classes raced on process-global CameraDiagnostics/ RenderingDiagnostics/Console.Out statics under xUnit's parallel-by-default class execution. DONE — serialized via a CameraDiagnosticsCollection marker.
• #253 — Attribute/skill icons in the character sheet sit off-center in their cells and render fully opaque, where retail uses vitae-window-like translucency; found during the user's mid-campaign Vulkan visual session, suspected pre-existing (the retained UI compares identical between backends at 3.94e-04) rather than a Vulkan regression. OPEN.