docs: publish the Campaign V report
Synthesizes the OpenGL-to-Vulkan migration for viewing on github.com: the screenshot sequence from first Vulkan content through the Dereth world, sky, portal tunnel and paperdoll; the six-cycle blank-world investigation that falsified five mechanisms and broke four instruments before the interleaved A/B probe and the 10/10 NVIDIA cross-vendor run pinned the defect to the AMD GL driver; the V7 differential verdict tables, including the offline GL-versus-Vulkan pair at 8.82e-04 inside threshold; the five compiled-clean- but-still-wrong defects the migration's own pixel gates caught; and the current V8/V9/V10/V11 status. Images are copied and renamed from the untracked artifacts/ tree under docs/reports/assets/campaign-v/. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
295
docs/reports/2026-07-28-campaign-v-report.md
Normal file
|
|
@ -0,0 +1,295 @@
|
|||
# Campaign V — OpenGL → Vulkan migration, results so far
|
||||
|
||||
**Date:** 2026-07-28
|
||||
**Plan:** [`docs/plans/2026-07-27-vulkan-campaign.md`](../plans/2026-07-27-vulkan-campaign.md)
|
||||
**Issues filed:** [`docs/ISSUES.md`](../ISSUES.md) #248–#253
|
||||
|
||||
acdream's mandatory modern renderer was built on GL 4.3 core +
|
||||
`ARB_bindless_texture` + `ARB_shader_draw_parameters`, an API that is no longer
|
||||
evolving and whose hardware floor is narrow — bindless textures are absent on
|
||||
Intel integrated GPUs and on every Mesa software/D3D12 stack, which is exactly
|
||||
what parked the Linux graphical slice (Slice L) at its L1 checkpoint. Campaign
|
||||
V replaces it with a single Vulkan 1.3 backend, keeping the pinned Vulkan-shaped
|
||||
RHI contract that a GL implementation satisfied first. It is **a compatibility
|
||||
and efficiency campaign, not a rescue** — the GL path worked and was fast;
|
||||
nothing about what the game looks like was supposed to change, and the campaign
|
||||
spent most of its effort proving that it didn't. As of this report: every
|
||||
production renderer draws on both backends, the GL-versus-Vulkan differential
|
||||
is inside its acceptance threshold on deterministic content, the user has run
|
||||
one full connected session on Vulkan and signed off on it, and the performance
|
||||
gate (V8) is running.
|
||||
|
||||
---
|
||||
|
||||
## The screenshots
|
||||
|
||||
**V6c — first content on Vulkan.** The synthetic bring-up scene: colour
|
||||
quadrants, a chequerboard, a gradient circle. This is the first frame the
|
||||
Vulkan RHI ever produced — proof that the SPIR-V toolchain, pipelines, passes,
|
||||
push constants and swapchain present all work together, before any production
|
||||
renderer touched the backend.
|
||||
|
||||

|
||||
|
||||
**V6d — the retained UI on Vulkan.** Same synthetic scene, now composited
|
||||
underneath acdream's real `TextRenderer`/`DebugLineRenderer` output ("acdream —
|
||||
retained UI on Vulkan / top-left origin — slice V6d"). This is the first
|
||||
acdream-authored draw call on the new backend, proving the UI's fill+border+
|
||||
glyph path was already backend-neutral.
|
||||
|
||||

|
||||
|
||||
**V6j — Dereth draws on Vulkan (the centerpiece).** `ACDREAM_RENDER_BACKEND=vulkan`
|
||||
rendering the full offline scene: blended terrain with road overlays, the water
|
||||
edge and its shoreline, static world meshes with correct silhouettes and closed
|
||||
shells, procedural scenery, per-instance lighting, and the complete retained UI
|
||||
including the radar. This is the slice where the world arm landed — three
|
||||
renderers' RHI submission arms, a backend-neutral pass-executor pair, and a
|
||||
winding-inversion bug (below) fixed on the same commit. Sky is still flat fog
|
||||
here; `SkyRenderer` doesn't cross until V6k.
|
||||
|
||||

|
||||
|
||||
**V6k — the sky lands.** Same view, now with the dome, the horizon band, the
|
||||
cloud sheet and the fog gradient rendered by the Vulkan arm — the same place
|
||||
and colours as the GL capture of the same scene. This slice also discharged
|
||||
§5.4's `BeginPass`-on-a-null-target divergence and moved the paperdoll/appraisal
|
||||
viewports onto `IGpuRenderTarget`.
|
||||
|
||||

|
||||
|
||||
**V6m — the retail portal tunnel on Vulkan.** One frame from a ten-frame transit
|
||||
capture: the same tube, the same blue-grey shading and drifting purple motes,
|
||||
the retained UI composited above it, with an opaque black surround and no world
|
||||
bleeding through (the pass clears colour rather than loading it — a Vulkan-forced
|
||||
decision documented in §5.5.18). `PortalTunnelPresentation` was the last raw-GL
|
||||
world-adjacent renderer; this frame is its Vulkan arm.
|
||||
|
||||

|
||||
|
||||
**V6l — the paperdoll on Vulkan.** The inventory/paperdoll panel open on the
|
||||
Vulkan arm, rendered through the private entity viewport with correct pose,
|
||||
orientation and alpha. This exercises the offscreen-render-target path the
|
||||
backbuffer-only captures above don't reach.
|
||||
|
||||

|
||||
|
||||
**The GL-versus-Vulkan difference map.** Red channel encodes per-pixel delta
|
||||
between a GL and a Vulkan capture of the same Holtburg scene (log scale, mostly
|
||||
black). The visible structure is exactly the two populations the differential
|
||||
work isolated: a thin band across the treeline/roofline near the top, and a
|
||||
scatter of isolated dots at building corners and cloud edges — not a diffuse,
|
||||
whole-frame shift. That shape is what let the investigation rule out a global
|
||||
cause (gamma, half-pixel offset, depth precision) and converge on anisotropic
|
||||
filtering as the remaining explanation (see the verdict tables below).
|
||||
|
||||

|
||||
|
||||
---
|
||||
|
||||
## The blank-world investigation
|
||||
|
||||
V4c and V4d — the slices that were meant to move the world draw path onto the
|
||||
RHI on GL first — shipped, passed their offline pixel gate, and then produced a
|
||||
**blank world** in the first connected sessions: UI rendered, sounds played, the
|
||||
log was clean, `world-reveal` reported `visible=True`, and the user's own screen
|
||||
surfaced one AMD driver-level timeout during testing. Both slices were reverted.
|
||||
What followed was six investigation cycles (§5.5.1–§5.5.6 of the plan), each one
|
||||
correcting the cycle before it.
|
||||
|
||||
**Cycle 1** measured the defect honestly for the first time: interleaved A/B
|
||||
runs of the same binary (to share whatever machine-state drift had been
|
||||
confounding earlier attributions) found **4/5 blank in the V4c arm versus 0/5 in
|
||||
the parent arm, p ≈ 0.024** — real, but intermittent. It also found the first
|
||||
broken instrument: the client's own screenshot read a 4×-multisampled default
|
||||
framebuffer with `glReadPixels`, which is undefined behavior per the GL spec.
|
||||
|
||||
**Cycle 2** tested every "shared 3-D state" theory that could explain "world
|
||||
dies, UI survives" — depth plane, camera constants, clip distances, GPU context
|
||||
reset — and found all four **bit-identical** between blank and rendered frames.
|
||||
It also found a second broken instrument: the in-process capture of framebuffer
|
||||
0 returned all-zero pixels on a blank run, even over screen regions the desktop
|
||||
compositor plainly showed the UI drawing into. Screenshot bytes were reporting
|
||||
the readback, not the renderer.
|
||||
|
||||
**Cycle 3** built an occlusion-query instrument to settle whether draws were
|
||||
executing and being discarded, or never executing at all — and found a third and
|
||||
fourth broken instrument in the process: a CPU-side query read
|
||||
(`glGetQueryObject`) **deadlocked the render thread inside the driver** on every
|
||||
attempt, and routing the same query through a GPU-timeline `GL_QUERY_BUFFER`
|
||||
write showed the write itself **never landing** on a blank run. The verdict: the
|
||||
GPU renders the first world frame identically to the working build — the same
|
||||
1,692,830 terrain samples and 317,561 entity samples — and then every GPU→CPU
|
||||
reporting channel dies at once, permanently, while presentation and fence
|
||||
signalling keep running normally.
|
||||
|
||||
**Cycles 4–5** laid out three ways forward and picked the cheapest discriminator:
|
||||
run the identical ported world path on Vulkan, on the same GPU. If it renders
|
||||
clean there, the defect is the AMD GL driver's, not the ported code's — settled
|
||||
in one measurement instead of a multi-hour bisect that would keep pinning the
|
||||
user's machine.
|
||||
|
||||
**Cycle 6** ran that measurement — first as a cross-vendor check while Vulkan
|
||||
bring-up was still in flight: the exact V4c binary, ten repeat-connected runs,
|
||||
on a separate NVIDIA machine against the same server, same account, same
|
||||
worst-case cell. **10/10 rendered**, bounding the NVIDIA failure rate below ~4%
|
||||
against a measured 30%+ on the AMD box. Same binary, same server, same scene,
|
||||
two drivers — only AMD's failed, and it failed below the API in four
|
||||
independent instruments. The driver-defect conclusion was adopted: the GL
|
||||
backend keeps the legacy raw-GL world path as a scoped, documented exception to
|
||||
the no-fork rule, and V4c/V4d's content returns as the Vulkan world path
|
||||
instead (landed at V6j — see the screenshots above).
|
||||
|
||||
**Falsified along the way:** the frame ring's `glBufferSubData` hazard, a
|
||||
capture-FBO binding leak, reveal ordering, shared 3-D state, and CPU-side
|
||||
visibility — five mechanisms, none of them the cause. **Broken along the way:**
|
||||
the multisampled `glReadPixels`, the in-process backbuffer capture, the
|
||||
CPU-blocking query read, and the GPU-timeline query write — four instruments
|
||||
that were measuring the wrong thing, all on the same driver.
|
||||
|
||||
---
|
||||
|
||||
## The verdict tables
|
||||
|
||||
**V7's per-stop differential**, full connected route, both backends, tolerance
|
||||
2 / fraction 0.001, MSAA off, day group 0, world time pinned to 0.5, sky phase
|
||||
pinned to 0 (`artifacts/v7-diff-c2`):
|
||||
|
||||
| Stop | Differing px | Fraction | Max Δ | Verdict |
|
||||
|---|---|---|---|---|
|
||||
| `holtburg_town` | 26,330 | 2.86e-02 | 255 | **EXCEPTION** — phase + AD-46 |
|
||||
| `facility_hub_interior` | 7,176 | 7.79e-03 | 255 | **EXCEPTION** — phase |
|
||||
| `aerlinthe_island` | 62,892 | 6.82e-02 | 255 | **EXCEPTION** — AD-46 + dark-scene floor |
|
||||
|
||||
No connected stop passes yet, and none of the three exceptions is a renderer
|
||||
defect: `holtburg_town`'s residual is an animated portal, two wandering NPCs, a
|
||||
smoke plume, and vitals that genuinely regenerated between two logins minutes
|
||||
apart, plus the treeline; `facility_hub_interior`'s entire residual is the local
|
||||
player's idle pose and its lighting — walls, floor and doorway are pixel-black
|
||||
in the difference map; `aerlinthe_island`'s residual is half AD-46 shrubbery and
|
||||
half an artifact of an *absolute* tolerance against a scene averaging 28/255
|
||||
luminance, where half the differing pixels are exactly one step over threshold.
|
||||
|
||||
**The offline GL-versus-Vulkan pair**, both clocks pinned, MSAA off — the
|
||||
instrument the investigation converged on, because the offline scene has no
|
||||
session, no server, no entities, no camera settle and no wandering NPCs, so a
|
||||
difference is the renderer or it is nothing:
|
||||
|
||||
| Pair | Differing px | Fraction |
|
||||
|---|---|---|
|
||||
| GL vs GL, same commit (control) | 1,966 | 2.13e-03 |
|
||||
| Vulkan vs Vulkan, same commit (control) | 1,039 | 1.13e-03 |
|
||||
| **GL vs Vulkan, whole frame** | 28,807 | 3.13e-02 |
|
||||
| **GL vs Vulkan, below the tree band (rows 280–720)** | **497** | **8.82e-04 — inside the 0.001 threshold** |
|
||||
|
||||
Terrain, terrain blending, roads, the water edge, fog, statics, scenery below
|
||||
the horizon, and the entire retained UI are at parity. Everything outside that
|
||||
threshold lives in one population: the treeline. Four hypotheses were tested —
|
||||
sub-pixel offset (refuted: best alignment is (0,0)), sharpness/LOD mismatch
|
||||
(refuted: high-frequency energy matches within 2–5%), depth precision (refuted:
|
||||
forcing GL's depth-range mapping onto Vulkan moved the number by only 3%), and
|
||||
anisotropic filtering (confirmed: going from anisotropy 1 to 16 — GL's and
|
||||
retail's own value — nearly halved the band, and no knob is left once both arms
|
||||
request the same setting). What remains is the anisotropic tap pattern itself,
|
||||
which both GL and Vulkan specs leave implementation-defined and AMD's own two
|
||||
drivers don't agree about. Filed as **`AD-46`** in the
|
||||
[divergence register](../architecture/retail-divergence-register.md), dormant
|
||||
until V10.
|
||||
|
||||
---
|
||||
|
||||
## The defects the migration found in its own code
|
||||
|
||||
Each of these compiled clean, passed validation, and had a test — and each was
|
||||
still wrong, caught only when a slice's pixel gate finally exercised the path
|
||||
that mattered. That's the campaign's per-slice discipline working as intended:
|
||||
|
||||
- **Winding inversion (V6j).** `VulkanViewportMapping` had inverted the front
|
||||
face since V6c on the standard negative-viewport argument, but nothing before
|
||||
the world arm actually culled anything to test it. The world arm was its
|
||||
first culling consumer, and it was wrong twice on one frame: terrain vanished
|
||||
outright, and every closed building shell rendered inside-out with its front
|
||||
wall culled and its interior beams visible through the gap.
|
||||
- **Stride mismatch (V6k).** The Vulkan sky's vertex layout declared a 32-byte
|
||||
stride (position, normal, texcoord) while the actual CPU vertex record is
|
||||
36 bytes — it carries a fourth field the sky shader never reads but the GL
|
||||
arm's `sizeof(Vertex)` always counted. The dome came up as a field of
|
||||
blue-white noise until the layout stated the record's real footprint.
|
||||
- **Sampler overrides (V7).** The Vulkan world texture arrays asked for
|
||||
`MaxAnisotropy: 1` where the GL arm reads the driver's actual capability
|
||||
twice over — and retail itself sets every sampler stage to the device's
|
||||
reported maximum. Fixing it to request the device ceiling halved the
|
||||
treeline difference band and dropped everything below it by two-thirds.
|
||||
- **A one-binding, two-renderer hazard (V6i-1).** A single descriptor binding
|
||||
would have let the GL particle draw clobber `WbDrawDispatcher`'s instance
|
||||
data mid-frame the moment both renderers were active in the same pass. Closed
|
||||
before the world arm could ever exercise it, with one descriptor-set pair per
|
||||
renderer scope derived from the descriptor state itself rather than declared
|
||||
by hand.
|
||||
- **MSAA readback undefined behavior (the blank-world investigation).** Every
|
||||
pixel gate and every blank/rendered verdict in the campaign, until this was
|
||||
found, read a 4×-multisampled default framebuffer directly with
|
||||
`glReadPixels` — undefined per the GL spec. The capture now resolves to a
|
||||
single-sampled framebuffer first; re-measuring the noise band afterward
|
||||
showed the fix itself moved the offline gate by 23 pixels out of 563,200,
|
||||
indistinguishable from its own same-commit control.
|
||||
|
||||
---
|
||||
|
||||
## Current status
|
||||
|
||||
- **V8 (performance gate)** is **running**. Its first commit landed the
|
||||
missing instrument: the Vulkan arm had been silently wired to
|
||||
`NullRenderFrameGpuMeasurement` and emitted no `[frame-prof]` line at all, so
|
||||
the campaign's own performance tooling had nothing to point at it. GPU
|
||||
timestamp measurement now brackets identically on both arms. The acceptance
|
||||
table (CPU frame p50 ≤ 1.60 ms, GPU p50 ≤ 1.00 ms, working set ≤ 600 MiB,
|
||||
private set ≤ 860 MiB, 0 B/frame managed allocation, CPU/GPU p99 ≤ GL) is the
|
||||
next thing this slice measures against the RX 9070 XT.
|
||||
- **V9 (Linux CI)** is **implemented, first CI run pending**. A `linux-vulkan`
|
||||
job now runs the probe on lavapipe under Xvfb, asserts a clean accepting
|
||||
verdict, checks the forced-unsupported exit-4 path, and verifies committed
|
||||
`.spv` binaries are byte-identical to a fresh compile. lavapipe satisfies
|
||||
every one of the seventeen gated features, including `samplerAnisotropy`,
|
||||
which V7 made load-bearing. The physical Linux GPU row and Wayland stay
|
||||
deferred on the same precedent as Slice L.
|
||||
- **V10 (cutover)** is **ahead**: Vulkan becomes the default backend, GL stays
|
||||
reachable by environment variable for one further slice, and **user visual
|
||||
sign-off** is the gate — the same kind of connected session the user already
|
||||
ran once mid-campaign (six targeted checks, all passed with merits: the AD-46
|
||||
treeline judged invisible in practice, dyed-gear composites, doorway
|
||||
look-out/threshold crossing, stacked translucency, icon/text crispness, and
|
||||
motion feel — the one open finding from that session is #253 below).
|
||||
- **V11 (closeout)** deletes `Gpu/Gl`, `OpenGLGraphicsDevice`, the
|
||||
`ManagedGL*` stack, the ImGui developer stack, and UI Studio, and re-measures
|
||||
working set one final time.
|
||||
|
||||
---
|
||||
|
||||
## Filed issues (#248–#253)
|
||||
|
||||
- **[#248](../ISSUES.md)** — `FrustumCuller` extracts its near plane with the
|
||||
GL `[-1,1]`-NDC formula against a `[0,1]`-NDC projection; harmless today
|
||||
(makes culling strictly more permissive near the eye) but wrong hygiene.
|
||||
OPEN.
|
||||
- **#249** — Released bindless texture slots never call `MakeNonResident`
|
||||
before the texture is deleted, which `GL_ARB_bindless_texture` leaves
|
||||
undefined, and leaks a resident handle for the process lifetime per release.
|
||||
OPEN.
|
||||
- **#250** — Four "zero managed allocation" tests fail intermittently, roughly
|
||||
one run in three, from tiered-JIT/GC measurement noise rather than a real
|
||||
regression — but an acceptance-gating test that flakes trains everyone to
|
||||
re-run until green. OPEN.
|
||||
- **#251** — `glClientWaitSync` returned a fence status of `0` (not a value
|
||||
the API is specified to return) and crashed the render loop once in nine
|
||||
connected runs; possibly the same below-the-API driver family the
|
||||
blank-world investigation documented four other instances of. OPEN.
|
||||
- **#252** — Ten App test classes raced on process-global `CameraDiagnostics`/
|
||||
`RenderingDiagnostics`/`Console.Out` statics under xUnit's parallel-by-default
|
||||
class execution. **DONE** — serialized via a `CameraDiagnosticsCollection`
|
||||
marker.
|
||||
- **#253** — Attribute/skill icons in the character sheet sit off-center in
|
||||
their cells and render fully opaque, where retail uses vitae-window-like
|
||||
translucency; found during the user's mid-campaign Vulkan visual session,
|
||||
suspected pre-existing (the retained UI compares identical between backends
|
||||
at 3.94e-04) rather than a Vulkan regression. OPEN.
|
||||
BIN
docs/reports/assets/campaign-v/v6c-vulkan-first-scene.png
Normal file
|
After Width: | Height: | Size: 15 KiB |
BIN
docs/reports/assets/campaign-v/v6d-vulkan-retained-ui.png
Normal file
|
After Width: | Height: | Size: 36 KiB |
BIN
docs/reports/assets/campaign-v/v6j-dereth-vulkan.png
Normal file
|
After Width: | Height: | Size: 1.7 MiB |
BIN
docs/reports/assets/campaign-v/v6k-diff-gl-vs-vulkan.png
Normal file
|
After Width: | Height: | Size: 93 KiB |
BIN
docs/reports/assets/campaign-v/v6k-sky-vulkan.png
Normal file
|
After Width: | Height: | Size: 1.7 MiB |
BIN
docs/reports/assets/campaign-v/v6l-paperdoll-vulkan.png
Normal file
|
After Width: | Height: | Size: 1.2 MiB |
BIN
docs/reports/assets/campaign-v/v6m-portal-tunnel-vulkan.png
Normal file
|
After Width: | Height: | Size: 751 KiB |