From fb9c6693dc9c5f10a2bf2524b567d9b53a3fd148 Mon Sep 17 00:00:00 2001 From: Erik Date: Tue, 28 Jul 2026 07:36:15 +0200 Subject: [PATCH] feat(render): Campaign V slice V6a - Vulkan memory, buffers, rings and the frame timeline The first of V6's three commits, and the half of the Vulkan backend that has nothing to do with drawing: where memory comes from, how per-frame data reaches the GPU, and what makes it safe to reuse either. Plan sections: 4.2 (bindings layer and the no-VMA decision), 4.3 (memory: arena, staging ring, per-frame data), 4.8 (sync and the frame). The allocator is hand-rolled, roughly as 4.2 sizes it. Silk ships no VMA, and a third-party binding would be a native binary to carry across win-x64, linux-x64 and CI lavapipe for an allocation profile that is genuinely tame: two mesh arena buffers, one staging ring, a per-flight ring buffer each, a few render targets and a texture pool. What a custom allocator buys instead is exact accounting - every byte is attributable to a memory type and a block - which is what GpuMemoryTracker will want and what VMA would obscure. Placement, block policy and heap choice are pure types with no Vulkan handle in sight: VulkanMemoryBlockFreeList is first-fit with coalescing on release, VulkanMemoryTypePool decides when a request is large enough to warrant a block of its own, and VulkanMemoryTypeSelection maps each GpuMemoryResidency onto a preference order of property masks. VulkanDeviceMemoryAllocator turns their answers into vkAllocateMemory and one persistent vkMapMemory per host-visible block. That split is deliberate: an allocator's real failure modes are arithmetic - a mis-coalesced neighbour, an alignment that eats a block's tail, a double release that quietly corrupts the used-byte count - and arithmetic does not need a GPU to be wrong. Twenty-two tests cover exactly those. The HostWritable row of the selection table is the campaign's CPU win stated as data. It prefers a memory type that is both DEVICE_LOCAL and HOST_VISIBLE - resizable BAR, present on the RX 9070 XT - so per-frame data is written once, straight into memory the GPU reads, and falls back to ordinary host-visible coherent memory when no such type exists. GpuCapabilityRecord's SupportsPersistentlyMappedRings is the first capability that is true on this backend and false on GL. Mapping is per block, never per allocation, because Vulkan permits a memory object to be mapped once - mapping per buffer would need one VkDeviceMemory per buffer, which is precisely the allocation-count explosion the design exists to avoid. VulkanRingBufferState is markedly simpler than its GL sibling, and the difference IS the point. GlRingBufferState has to track a dirty watermark and prove its upload never overlaps an in-flight read, because a ring allocation there writes into a managed array that is later copied into a GL buffer. Here the allocation hands back memory the GPU reads directly: there is no upload step to track. What is left is a cursor. VulkanUploadQueue accumulates transfers rather than issuing them, for two reasons that both come from Vulkan rather than from taste: copies must be recorded into a command buffer, and they must be recorded outside a dynamic-rendering block. So requests queue and drain at the one moment both hold - immediately before a pass begins - which is the direct analogue of the GL backend's flush-before-every-draw discipline at the granularity Vulkan needs. The drain emits one batched buffer barrier for the whole batch, one of the four to six 4.8 budgets per frame. Staging exhaustion falls back to a temporary dedicated buffer retired through the ledger. Section 4.3 already specifies that for oversized uploads; extending it to "the ring is full of unretired frames" is the same shape and is a policy rather than a workaround - the transfer stays correct and ordered, it just costs one allocation. VulkanFrameFlightController is the mechanical port 4.8 promised. GL's array of fences becomes one timeline semaphore whose value is the frame serial, "has this slot retired?" becomes "is the counter at least serial minus two?", and the SortedDictionary retirement ledger keeps its keys because those keys were already frame serials. One subtlety is worth stating: a release is filed against the frame currently being RECORDED, not the last one completed, because commands already recorded into the open frame may still read the resource. A test pins that, since getting it wrong frees memory a pending command buffer reads and the symptom would appear somewhere else entirely. Frame acquire ordering is the other subtlety. TryBeginFrame waits on the flight slot BEFORE acquiring its swapchain image, so the slot's acquire semaphore is provably idle - signalling a semaphore a pending submit still waits on is the classic Vulkan deadlock. When the acquire fails the serial is still signalled through an empty submit, because a serial that never completes makes every later frame wait forever. The device is a partial class split along the V6 commit boundary: everything here is memory and frames, while textures and the descriptor table (V6b) and pipelines, passes and readback (V6c) throw with the slice named rather than returning something that fails later and further away. Nothing constructs this device yet - VulkanBringUpHost still presents its clear colour - so the GL path executes not one new statement. VK_EXT_debug_utils naming arrives with the allocator rather than at V6c, because every resource wants a name from birth and the campaign has already spent days on defects only visible from outside the API. It stays optional: absent extension means every call is a no-op and no call site checks. Gates: Release build clean, App suite 4014 passed / 3 skipped (3981 baseline plus 33 new). One Issue181WallPressEquilibriumTests failure in the full run is the known #250 zero-allocation flake and passes on a single run. Offline pixel gate against the parent is a tripwire here - the backend is dark and no GL code path changed - and is reported with the slice. Co-Authored-By: Claude Fable 5 --- src/AcDream.App/AcDream.App.csproj | 4 + .../Rendering/Gpu/Vk/VulkanDebugNames.cs | 136 +++++ .../Gpu/Vk/VulkanDeviceMemoryAllocator.cs | 242 ++++++++ .../Gpu/Vk/VulkanFrameFlightController.cs | 221 +++++++ .../Rendering/Gpu/Vk/VulkanGpuBuffer.cs | 211 +++++++ .../Gpu/Vk/VulkanGpuDevice.Resources.cs | 67 ++ .../Rendering/Gpu/Vk/VulkanGpuDevice.cs | 572 ++++++++++++++++++ .../Rendering/Gpu/Vk/VulkanGpuFrame.cs | 66 ++ .../Rendering/Gpu/Vk/VulkanMemoryModel.cs | 411 +++++++++++++ .../Rendering/Gpu/Vk/VulkanRingBufferState.cs | 189 ++++++ .../Rendering/Gpu/Vk/VulkanUploadQueue.cs | 392 ++++++++++++ .../Gpu/Vk/VulkanFrameFlightTests.cs | 281 +++++++++ .../Gpu/Vk/VulkanMemoryModelTests.cs | 248 ++++++++ 13 files changed, 3040 insertions(+) create mode 100644 src/AcDream.App/Rendering/Gpu/Vk/VulkanDebugNames.cs create mode 100644 src/AcDream.App/Rendering/Gpu/Vk/VulkanDeviceMemoryAllocator.cs create mode 100644 src/AcDream.App/Rendering/Gpu/Vk/VulkanFrameFlightController.cs create mode 100644 src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuBuffer.cs create mode 100644 src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuDevice.Resources.cs create mode 100644 src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuDevice.cs create mode 100644 src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuFrame.cs create mode 100644 src/AcDream.App/Rendering/Gpu/Vk/VulkanMemoryModel.cs create mode 100644 src/AcDream.App/Rendering/Gpu/Vk/VulkanRingBufferState.cs create mode 100644 src/AcDream.App/Rendering/Gpu/Vk/VulkanUploadQueue.cs create mode 100644 tests/AcDream.App.Tests/Rendering/Gpu/Vk/VulkanFrameFlightTests.cs create mode 100644 tests/AcDream.App.Tests/Rendering/Gpu/Vk/VulkanMemoryModelTests.cs diff --git a/src/AcDream.App/AcDream.App.csproj b/src/AcDream.App/AcDream.App.csproj index 513360c5..e47fb6ec 100644 --- a/src/AcDream.App/AcDream.App.csproj +++ b/src/AcDream.App/AcDream.App.csproj @@ -32,6 +32,10 @@ the rest of the Silk family so the shared Silk.NET.Core does not fork. --> + + diff --git a/src/AcDream.App/Rendering/Gpu/Vk/VulkanDebugNames.cs b/src/AcDream.App/Rendering/Gpu/Vk/VulkanDebugNames.cs new file mode 100644 index 00000000..0a3b048c --- /dev/null +++ b/src/AcDream.App/Rendering/Gpu/Vk/VulkanDebugNames.cs @@ -0,0 +1,136 @@ +using Silk.NET.Core.Native; +using Silk.NET.Vulkan; +using Silk.NET.Vulkan.Extensions.EXT; +using Buffer = Silk.NET.Vulkan.Buffer; + +namespace AcDream.App.Rendering.Gpu.Vk; + +/// +/// Campaign V slice V6, plan §4.1: publishes acdream's own object names through +/// VK_EXT_debug_utils when the extension is present. +/// +/// Naming is why GpuBufferDescription.Name is documented as "not +/// cosmetic". A RenderDoc capture or a validation-layer message that says +/// mesh-arena-vertex instead of VkBuffer 0x7f… is the difference +/// between reading a frame and guessing at one — and the campaign has already +/// spent days on defects that were only ever visible from outside the API. +/// +/// The extension is optional and never required: when it is absent every +/// method here is a cheap no-op, so callers name everything unconditionally and +/// no call site needs a capability check. +/// +internal sealed unsafe class VulkanDebugNames : IDisposable +{ + private readonly Device _device; + private readonly ExtDebugUtils? _api; + private bool _disposed; + + private VulkanDebugNames(Device device, ExtDebugUtils? api) + { + _device = device; + _api = api; + } + + /// A naming sink that does nothing — used when the extension is absent and by tests. + internal static VulkanDebugNames Disabled { get; } = new(default, null); + + internal bool IsEnabled => _api is not null; + + /// + /// Loads the extension's entry points when + /// contains it. Failure to load is treated exactly like absence: naming is a + /// diagnostic, never a dependency. + /// + internal static VulkanDebugNames Create( + Silk.NET.Vulkan.Vk vk, + Instance instance, + Device device, + IReadOnlyCollection enabledInstanceExtensions) + { + ArgumentNullException.ThrowIfNull(vk); + ArgumentNullException.ThrowIfNull(enabledInstanceExtensions); + if (!enabledInstanceExtensions.Contains(VulkanExtensionSelection.DebugUtilsExtension)) + return Disabled; + if (!vk.TryGetInstanceExtension(instance, out ExtDebugUtils api)) + return Disabled; + return new VulkanDebugNames(device, api); + } + + internal void NameBuffer(Buffer buffer, string name) => + Name(ObjectType.Buffer, buffer.Handle, name); + + internal void NameImage(Image image, string name) => + Name(ObjectType.Image, image.Handle, name); + + internal void NameImageView(ImageView view, string name) => + Name(ObjectType.ImageView, view.Handle, name); + + internal void NameSampler(Sampler sampler, string name) => + Name(ObjectType.Sampler, sampler.Handle, name); + + internal void NamePipeline(Pipeline pipeline, string name) => + Name(ObjectType.Pipeline, pipeline.Handle, name); + + internal void NameDeviceMemory(DeviceMemory memory, string name) => + Name(ObjectType.DeviceMemory, memory.Handle, name); + + /// Opens a labelled region in the command stream — one per RHI pass. + internal void BeginLabel(CommandBuffer commands, string label) + { + if (_api is null || _disposed) + return; + + nint text = SilkMarshal.StringToPtr(label); + try + { + var info = new DebugUtilsLabelEXT + { + SType = StructureType.DebugUtilsLabelExt, + PLabelName = (byte*)text, + }; + _api.CmdBeginDebugUtilsLabel(commands, &info); + } + finally + { + SilkMarshal.Free(text); + } + } + + internal void EndLabel(CommandBuffer commands) + { + if (_api is null || _disposed) + return; + _api.CmdEndDebugUtilsLabel(commands); + } + + private void Name(ObjectType type, ulong handle, string name) + { + if (_api is null || _disposed || handle == 0 || string.IsNullOrEmpty(name)) + return; + + nint text = SilkMarshal.StringToPtr(name); + try + { + var info = new DebugUtilsObjectNameInfoEXT + { + SType = StructureType.DebugUtilsObjectNameInfoExt, + ObjectType = type, + ObjectHandle = handle, + PObjectName = (byte*)text, + }; + _api.SetDebugUtilsObjectName(_device, &info); + } + finally + { + SilkMarshal.Free(text); + } + } + + public void Dispose() + { + if (_disposed) + return; + _disposed = true; + _api?.Dispose(); + } +} diff --git a/src/AcDream.App/Rendering/Gpu/Vk/VulkanDeviceMemoryAllocator.cs b/src/AcDream.App/Rendering/Gpu/Vk/VulkanDeviceMemoryAllocator.cs new file mode 100644 index 00000000..207eb51b --- /dev/null +++ b/src/AcDream.App/Rendering/Gpu/Vk/VulkanDeviceMemoryAllocator.cs @@ -0,0 +1,242 @@ +using Silk.NET.Vulkan; + +namespace AcDream.App.Rendering.Gpu.Vk; + +/// +/// A live suballocation: the device memory it sits in, where, and — for +/// host-visible memory — a pointer to its first byte inside the block's +/// persistent mapping. +/// +internal readonly unsafe struct VulkanAllocation( + DeviceMemory memory, + ulong offsetBytes, + ulong sizeBytes, + uint memoryTypeIndex, + VulkanMemoryRange range, + void* mapped) +{ + internal DeviceMemory Memory { get; } = memory; + internal ulong OffsetBytes { get; } = offsetBytes; + internal ulong SizeBytes { get; } = sizeBytes; + internal uint MemoryTypeIndex { get; } = memoryTypeIndex; + internal VulkanMemoryRange Range { get; } = range; + + /// First mapped byte of this allocation, or null on device-local memory. + internal void* Mapped { get; } = mapped; + + internal bool IsMapped => Mapped is not null; + + internal Span AsSpan() => IsMapped + ? new Span(Mapped, checked((int)SizeBytes)) + : throw new InvalidOperationException( + "This allocation lives in device-local memory and has no CPU mapping."); +} + +/// +/// Campaign V slice V6a, plan §4.2: acdream's Vulkan memory allocator. +/// +/// No VMA, deliberately. Silk does not ship it; third-party .NET +/// bindings are a native-binary and maintenance liability across win-x64, +/// linux-x64 and CI lavapipe; and acdream's allocation profile is tame — two +/// mesh arena buffers, one staging ring, a few per-flight buffers, a handful of +/// render targets and a texture pool. What this buys instead is exact +/// accounting: every byte is attributable to a memory type and a block, which +/// VMA would obscure. +/// +/// The interesting decisions are not here. Placement inside a block is +/// , the block/dedicated policy is +/// , and the heap choice is +/// — all pure and unit-tested. This +/// class turns their answers into vkAllocateMemory and one persistent +/// vkMapMemory per host-visible block. +/// +/// One mapping per block, never per allocation. Vulkan permits a +/// memory object to be mapped once; mapping per buffer would therefore need one +/// VkDeviceMemory per buffer, which is the allocation-count explosion the +/// whole design exists to avoid. Blocks are mapped when they are created and +/// stay mapped for their lifetime, and a suballocation's pointer is the block +/// base plus its offset. +/// +internal sealed unsafe class VulkanDeviceMemoryAllocator : IDisposable +{ + private readonly Silk.NET.Vulkan.Vk _vk; + private readonly Device _device; + private readonly MemoryPropertyFlags[] _memoryTypeProperties; + private readonly ulong _blockSizeBytes; + private readonly ulong _dedicatedThresholdBytes; + + private readonly Dictionary _pools = []; + private readonly Dictionary<(uint TypeIndex, int BlockIndex), BlockMemory> _blockMemory = []; + + private bool _disposed; + + private readonly record struct BlockMemory(DeviceMemory Memory, nint Mapped, ulong CapacityBytes); + + internal VulkanDeviceMemoryAllocator( + Silk.NET.Vulkan.Vk vk, + PhysicalDevice physicalDevice, + Device device, + ulong blockSizeBytes = VulkanMemoryTypePool.DefaultBlockSizeBytes, + ulong dedicatedThresholdBytes = VulkanMemoryTypePool.DefaultDedicatedThresholdBytes) + { + _vk = vk ?? throw new ArgumentNullException(nameof(vk)); + _device = device; + _blockSizeBytes = blockSizeBytes; + _dedicatedThresholdBytes = dedicatedThresholdBytes; + + vk.GetPhysicalDeviceMemoryProperties(physicalDevice, out PhysicalDeviceMemoryProperties properties); + _memoryTypeProperties = new MemoryPropertyFlags[properties.MemoryTypeCount]; + for (uint i = 0; i < properties.MemoryTypeCount && i < 32; i++) + _memoryTypeProperties[i] = properties.MemoryTypes[(int)i].PropertyFlags; + } + + /// Live vkAllocateMemory objects. Kept two orders of magnitude below the device limit by design. + internal int DeviceMemoryObjectCount => _blockMemory.Count; + + /// Total bytes handed to callers, excluding block headroom. + internal ulong AllocatedBytes { get; private set; } + + /// Total bytes committed to vkAllocateMemory, including headroom in shared blocks. + internal ulong CommittedBytes { get; private set; } + + internal IReadOnlyList MemoryTypeProperties => _memoryTypeProperties; + + /// + /// Allocates memory satisfying for the given + /// residency class, creating a block when no existing one fits. + /// + internal VulkanAllocation Allocate( + in MemoryRequirements requirements, + GpuMemoryResidency residency, + string ownerName) + { + ObjectDisposedException.ThrowIf(_disposed, this); + + uint typeIndex = VulkanMemoryTypeSelection.Choose( + _memoryTypeProperties, + requirements.MemoryTypeBits, + residency) + ?? throw new NotSupportedException( + $"No Vulkan memory type satisfies {residency} for '{ownerName}'. " + + $"Allowed type bits 0x{requirements.MemoryTypeBits:X8}; the device exposes " + + $"{_memoryTypeProperties.Length} memory types."); + + if (!_pools.TryGetValue(typeIndex, out VulkanMemoryTypePool? pool)) + { + pool = new VulkanMemoryTypePool(typeIndex, _blockSizeBytes, _dedicatedThresholdBytes); + _pools.Add(typeIndex, pool); + } + + ulong size = requirements.Size; + ulong alignment = Math.Max(requirements.Alignment, 1); + if (!pool.TryAllocate(size, alignment, out VulkanMemoryRange range)) + { + bool dedicated = pool.IsDedicatedSize(size); + ulong capacity = Math.Max(pool.BlockCapacityFor(size), size); + int blockIndex = pool.AddBlock(capacity, dedicated); + CreateBlockMemory(typeIndex, blockIndex, capacity, ownerName); + + range = dedicated + ? pool.AllocateWholeBlock(blockIndex, size) + : pool.TryAllocate(size, alignment, out VulkanMemoryRange placed) + ? placed + : throw new InvalidOperationException( + $"A freshly created {capacity}-byte block could not satisfy a {size}-byte " + + $"allocation at alignment {alignment} for '{ownerName}'."); + } + + BlockMemory block = _blockMemory[(typeIndex, range.BlockIndex)]; + AllocatedBytes += range.SizeBytes; + void* mapped = block.Mapped == 0 + ? null + : (void*)(block.Mapped + (nint)range.OffsetBytes); + return new VulkanAllocation( + block.Memory, + range.OffsetBytes, + range.SizeBytes, + typeIndex, + range, + mapped); + } + + /// Returns an allocation's bytes to its pool, freeing the block when a dedicated one empties. + internal void Free(in VulkanAllocation allocation) + { + if (_disposed || allocation.SizeBytes == 0) + return; + if (!_pools.TryGetValue(allocation.MemoryTypeIndex, out VulkanMemoryTypePool? pool)) + return; + + AllocatedBytes -= Math.Min(AllocatedBytes, allocation.Range.SizeBytes); + if (!pool.Free(allocation.Range)) + return; + + var key = (allocation.MemoryTypeIndex, allocation.Range.BlockIndex); + if (!_blockMemory.Remove(key, out BlockMemory block)) + return; + + if (block.Mapped != 0) + _vk.UnmapMemory(_device, block.Memory); + _vk.FreeMemory(_device, block.Memory, null); + CommittedBytes -= Math.Min(CommittedBytes, block.CapacityBytes); + } + + private void CreateBlockMemory(uint typeIndex, int blockIndex, ulong capacityBytes, string ownerName) + { + var allocate = new MemoryAllocateInfo + { + SType = StructureType.MemoryAllocateInfo, + AllocationSize = capacityBytes, + MemoryTypeIndex = typeIndex, + }; + Result result = _vk.AllocateMemory(_device, &allocate, null, out DeviceMemory memory); + if (result != Result.Success) + { + throw new VulkanCallException( + $"vkAllocateMemory ({capacityBytes} bytes on memory type {typeIndex} for '{ownerName}')", + result); + } + + nint mapped = 0; + if (_memoryTypeProperties[(int)typeIndex].HasFlag(MemoryPropertyFlags.HostVisibleBit)) + { + void* pointer = null; + Result mapResult = _vk.MapMemory(_device, memory, 0, capacityBytes, 0, &pointer); + if (mapResult != Result.Success) + { + _vk.FreeMemory(_device, memory, null); + throw new VulkanCallException($"vkMapMemory (block for '{ownerName}')", mapResult); + } + + mapped = (nint)pointer; + } + + _blockMemory[(typeIndex, blockIndex)] = new BlockMemory(memory, mapped, capacityBytes); + CommittedBytes += capacityBytes; + } + + /// Human-readable accounting for the diagnostics report and for teardown assertions. + internal string Describe() => + $"{DeviceMemoryObjectCount} device-memory object(s), " + + $"{CommittedBytes / (1024 * 1024)} MiB committed, " + + $"{AllocatedBytes / (1024 * 1024)} MiB allocated"; + + public void Dispose() + { + if (_disposed) + return; + _disposed = true; + + foreach (BlockMemory block in _blockMemory.Values) + { + if (block.Mapped != 0) + _vk.UnmapMemory(_device, block.Memory); + _vk.FreeMemory(_device, block.Memory, null); + } + + _blockMemory.Clear(); + _pools.Clear(); + AllocatedBytes = 0; + CommittedBytes = 0; + } +} diff --git a/src/AcDream.App/Rendering/Gpu/Vk/VulkanFrameFlightController.cs b/src/AcDream.App/Rendering/Gpu/Vk/VulkanFrameFlightController.cs new file mode 100644 index 00000000..5ef2a7b6 --- /dev/null +++ b/src/AcDream.App/Rendering/Gpu/Vk/VulkanFrameFlightController.cs @@ -0,0 +1,221 @@ +using AcDream.App.Rendering; +using Silk.NET.Vulkan; +using Semaphore = Silk.NET.Vulkan.Semaphore; + +namespace AcDream.App.Rendering.Gpu.Vk; + +/// +/// The four timeline-semaphore operations +/// needs, behind an interface so the controller's ledger and flight arithmetic +/// run under a unit test with no device — the same role IGpuFenceApi +/// plays for the GL flight controller. +/// +internal interface IVulkanTimelineApi +{ + /// Highest value the timeline has signalled. + ulong CurrentValue { get; } + + /// Blocks until the timeline reaches . + void Wait(ulong value); +} + +/// Live implementation over one VkSemaphore of type timeline. +internal sealed unsafe class VulkanTimelineApi( + Silk.NET.Vulkan.Vk vk, + Device device, + Semaphore timeline) : IVulkanTimelineApi +{ + private readonly Silk.NET.Vulkan.Vk _vk = vk ?? throw new ArgumentNullException(nameof(vk)); + + public ulong CurrentValue + { + get + { + VulkanInterop.Check( + _vk.GetSemaphoreCounterValue(device, timeline, out ulong value), + "vkGetSemaphoreCounterValue"); + return value; + } + } + + public void Wait(ulong value) + { + Semaphore semaphore = timeline; + ulong target = value; + var wait = new SemaphoreWaitInfo + { + SType = StructureType.SemaphoreWaitInfo, + SemaphoreCount = 1, + PSemaphores = &semaphore, + PValues = &target, + }; + VulkanInterop.Check( + _vk.WaitSemaphores(device, &wait, ulong.MaxValue), + $"vkWaitSemaphores (frame flight, value {value})"); + } +} + +/// +/// Campaign V slice V6a, plan §4.8: frames in flight and retirement, on ONE +/// timeline semaphore whose value is the frame serial. +/// +/// This is the Vulkan sibling of GpuFrameFlightController, and the +/// port is close to mechanical because the plan chose the timeline for exactly +/// that reason. GL's array of fences becomes one monotonic counter; "has slot +/// s retired?" becomes "is the counter at least serial - 2?"; and +/// the SortedDictionary<long, List<Action>> retirement ledger +/// keeps its keys, because those keys were already frame serials. +/// +/// Retirement is keyed to the frame that has not been submitted +/// yet. A resource released now may still be referenced by commands already +/// recorded into the open frame, so its release is filed under the serial of the +/// frame currently being recorded (or the next one to open, when no frame is +/// open) and runs once the timeline passes that value. Releasing at the +/// last-completed serial would free memory a recorded-but-unsubmitted command +/// buffer still reads. +/// +internal sealed class VulkanFrameFlightController : IGpuResourceRetirementQueue, IDisposable +{ + /// Plan §4.8: two frames in flight. + internal const int DefaultFramesInFlight = 2; + + private readonly IVulkanTimelineApi _timeline; + private readonly SortedDictionary> _retirements = []; + + private long _openSerial; + private long _submittedSerial; + private bool _disposed; + + internal VulkanFrameFlightController( + IVulkanTimelineApi timeline, + int framesInFlight = DefaultFramesInFlight) + { + _timeline = timeline ?? throw new ArgumentNullException(nameof(timeline)); + ArgumentOutOfRangeException.ThrowIfLessThan(framesInFlight, 1); + SlotCount = framesInFlight; + } + + /// Frames in flight, and therefore ring/command-pool slots. + internal int SlotCount { get; } + + /// Serial of the frame currently being recorded, or 0 when none is open. + internal long OpenSerial => _openSerial; + + /// Highest serial handed to . + internal long SubmittedSerial => _submittedSerial; + + /// Flight slot index of the currently open frame. + internal int CurrentSlot => SlotIndexOf(_openSerial); + + internal int PendingRetirementCount => _retirements.Sum(entry => entry.Value.Count); + + /// Maps a frame serial onto its flight slot. Serials are 1-based. + internal int SlotIndexOf(long serial) => + serial <= 0 ? 0 : (int)((serial - 1) % SlotCount); + + /// + /// Opens the next frame: waits until its flight slot's previous occupant has + /// completed on the GPU, runs everything that retirement made due, and + /// returns the new serial. + /// + internal long BeginFrame() + { + ObjectDisposedException.ThrowIf(_disposed, this); + if (_openSerial != 0) + { + throw new InvalidOperationException( + $"Frame {_openSerial} is still open; call EndFrame before beginning another."); + } + + long serial = _submittedSerial + 1; + long mustComplete = serial - SlotCount; + if (mustComplete > 0) + _timeline.Wait((ulong)mustComplete); + + _openSerial = serial; + RunRetirements(); + return serial; + } + + /// Records that the open frame has been submitted with its serial as the timeline signal value. + internal void EndFrame() + { + if (_openSerial == 0) + return; + _submittedSerial = _openSerial; + _openSerial = 0; + } + + /// + /// Files to run once every frame that could still + /// reference the resource has completed. See the class remarks for why the + /// key is the open (or next) serial rather than the last completed one. + /// + public void Retire(Action release) + { + ArgumentNullException.ThrowIfNull(release); + if (_disposed) + { + // Teardown already drained the ledger; running immediately is the + // only way this release ever happens, and by then the device is idle. + release(); + return; + } + + long key = _openSerial != 0 ? _openSerial : _submittedSerial + 1; + if (!_retirements.TryGetValue(key, out List? actions)) + { + actions = []; + _retirements.Add(key, actions); + } + + actions.Add(release); + } + + /// Runs every retirement whose frame the GPU has completed. + internal void RunRetirements() + { + if (_retirements.Count == 0) + return; + + var completed = (long)_timeline.CurrentValue; + while (_retirements.Count > 0) + { + KeyValuePair> first = _retirements.First(); + if (first.Key > completed) + break; + + _retirements.Remove(first.Key); + foreach (Action release in first.Value) + release(); + } + } + + /// Blocks until every submitted frame has completed, then drains the whole ledger. + internal void WaitForSubmittedWork() + { + if (_submittedSerial > 0) + _timeline.Wait((ulong)_submittedSerial); + DrainAll(); + } + + /// Runs every pending retirement regardless of serial. Only legal when the device is idle. + internal void DrainAll() + { + while (_retirements.Count > 0) + { + KeyValuePair> first = _retirements.First(); + _retirements.Remove(first.Key); + foreach (Action release in first.Value) + release(); + } + } + + public void Dispose() + { + if (_disposed) + return; + DrainAll(); + _disposed = true; + } +} diff --git a/src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuBuffer.cs b/src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuBuffer.cs new file mode 100644 index 00000000..c2e9cad2 --- /dev/null +++ b/src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuBuffer.cs @@ -0,0 +1,211 @@ +using Silk.NET.Vulkan; +using Buffer = Silk.NET.Vulkan.Buffer; + +namespace AcDream.App.Rendering.Gpu.Vk; + +/// +/// Campaign V slice V6a: on Vulkan. +/// +/// Host-writable buffers are written, not uploaded. On a +/// buffer is +/// a memcpy into memory the GPU already reads — the driver validation, +/// the copy and the rename tracking that glBufferSubData pays for simply +/// do not happen. On a device-local buffer it stages through +/// instead, which is the only case that costs a +/// transfer. +/// +/// Usage flags are named at creation, so they are named generously. +/// Vulkan requires every consumption to be declared up front and the RHI +/// contract puts and on every buffer, +/// so transfer source and destination are always included. They cost nothing on +/// any implementation acdream targets and the alternative — inferring them from +/// how a buffer happens to be used later — is how a driver error appears months +/// after the code that caused it. +/// +/// Disposal routes through the device's retirement queue rather than +/// destroying immediately, so a submitted frame that still reads this buffer +/// keeps valid memory. +/// +internal sealed unsafe class VulkanGpuBuffer : IGpuBuffer +{ + private readonly Silk.NET.Vulkan.Vk _vk; + private readonly Device _device; + private readonly VulkanDeviceMemoryAllocator _allocator; + private readonly VulkanUploadQueue _uploads; + private readonly IGpuResourceRetirementQueue _retirement; + private readonly VulkanAllocation _allocation; + private bool _disposed; + + internal VulkanGpuBuffer( + Silk.NET.Vulkan.Vk vk, + Device device, + VulkanDeviceMemoryAllocator allocator, + VulkanUploadQueue uploads, + IGpuResourceRetirementQueue retirement, + VulkanDebugNames debugNames, + in GpuBufferDescription description) + { + _vk = vk ?? throw new ArgumentNullException(nameof(vk)); + _device = device; + _allocator = allocator ?? throw new ArgumentNullException(nameof(allocator)); + _uploads = uploads ?? throw new ArgumentNullException(nameof(uploads)); + _retirement = retirement ?? throw new ArgumentNullException(nameof(retirement)); + ArgumentException.ThrowIfNullOrWhiteSpace(description.Name); + ArgumentOutOfRangeException.ThrowIfNegativeOrZero(description.SizeBytes); + + Name = description.Name; + SizeBytes = description.SizeBytes; + Usage = description.Usage; + Residency = description.Residency; + + var create = new BufferCreateInfo + { + SType = StructureType.BufferCreateInfo, + Size = (ulong)description.SizeBytes, + Usage = UsageFlagsOf(description.Usage), + SharingMode = SharingMode.Exclusive, + }; + VulkanInterop.Check( + _vk.CreateBuffer(_device, &create, null, out Buffer handle), + $"vkCreateBuffer ('{description.Name}')"); + Handle = handle; + + try + { + _vk.GetBufferMemoryRequirements(_device, handle, out MemoryRequirements requirements); + _allocation = _allocator.Allocate(requirements, description.Residency, description.Name); + VulkanInterop.Check( + _vk.BindBufferMemory(_device, handle, _allocation.Memory, _allocation.OffsetBytes), + $"vkBindBufferMemory ('{description.Name}')"); + } + catch + { + _vk.DestroyBuffer(_device, handle, null); + throw; + } + + debugNames.NameBuffer(handle, description.Name); + } + + public string Name { get; } + public long SizeBytes { get; } + public GpuBufferUsage Usage { get; } + public GpuMemoryResidency Residency { get; } + + internal Buffer Handle { get; } + + /// True when this buffer's memory is persistently mapped and directly writable. + internal bool IsMapped => _allocation.IsMapped; + + /// The buffer's whole mapped range. Only valid on host-visible residency. + internal Span MappedSpan => _allocation.AsSpan()[..checked((int)SizeBytes)]; + + public void Upload(long offsetBytes, ReadOnlySpan data) + { + ThrowIfDisposed(); + ArgumentOutOfRangeException.ThrowIfNegative(offsetBytes); + if (data.IsEmpty) + return; + if (offsetBytes + data.Length > SizeBytes) + { + throw new ArgumentOutOfRangeException( + nameof(data), + $"Writing {data.Length} bytes at offset {offsetBytes} exceeds " + + $"'{Name}' ({SizeBytes} bytes)."); + } + + if (_allocation.IsMapped) + { + data.CopyTo(MappedSpan.Slice((int)offsetBytes, data.Length)); + return; + } + + _uploads.StageBufferWrite(Handle, (ulong)offsetBytes, data, Name); + } + + public void CopyTo( + IGpuBuffer destination, + long sourceOffsetBytes, + long destinationOffsetBytes, + long byteCount) + { + ThrowIfDisposed(); + ArgumentNullException.ThrowIfNull(destination); + if (destination is not VulkanGpuBuffer target) + { + throw new ArgumentException( + "The Vulkan backend can only copy into a Vulkan buffer.", + nameof(destination)); + } + + ArgumentOutOfRangeException.ThrowIfNegative(byteCount); + if (byteCount == 0) + return; + if (sourceOffsetBytes + byteCount > SizeBytes) + throw new ArgumentOutOfRangeException(nameof(byteCount), $"The copy reads past the end of '{Name}'."); + if (destinationOffsetBytes + byteCount > target.SizeBytes) + throw new ArgumentOutOfRangeException(nameof(byteCount), $"The copy writes past the end of '{target.Name}'."); + + _uploads.EnqueueBufferCopy( + Handle, + target.Handle, + (ulong)sourceOffsetBytes, + (ulong)destinationOffsetBytes, + (ulong)byteCount); + } + + public void Read(long offsetBytes, Span destination) + { + ThrowIfDisposed(); + if (Residency != GpuMemoryResidency.HostReadable) + { + throw new InvalidOperationException( + $"'{Name}' has {Residency} residency; only HostReadable buffers can be read back. " + + "This path is diagnostics-only by design."); + } + + if (destination.IsEmpty) + return; + if (offsetBytes + destination.Length > SizeBytes) + throw new ArgumentOutOfRangeException(nameof(destination), $"The read runs past the end of '{Name}'."); + + MappedSpan.Slice((int)offsetBytes, destination.Length).CopyTo(destination); + } + + /// + /// Maps onto Vulkan usage bits. Transfer source + /// and destination are unconditional — see the class remarks. + /// + internal static BufferUsageFlags UsageFlagsOf(GpuBufferUsage usage) + { + BufferUsageFlags flags = BufferUsageFlags.TransferSrcBit | BufferUsageFlags.TransferDstBit; + if (usage.HasFlag(GpuBufferUsage.Vertex)) + flags |= BufferUsageFlags.VertexBufferBit; + if (usage.HasFlag(GpuBufferUsage.Index)) + flags |= BufferUsageFlags.IndexBufferBit; + if (usage.HasFlag(GpuBufferUsage.Storage)) + flags |= BufferUsageFlags.StorageBufferBit; + if (usage.HasFlag(GpuBufferUsage.Uniform)) + flags |= BufferUsageFlags.UniformBufferBit; + if (usage.HasFlag(GpuBufferUsage.Indirect)) + flags |= BufferUsageFlags.IndirectBufferBit; + return flags; + } + + public void Dispose() + { + if (_disposed) + return; + _disposed = true; + + Buffer handle = Handle; + VulkanAllocation allocation = _allocation; + _retirement.Retire(() => + { + _vk.DestroyBuffer(_device, handle, null); + _allocator.Free(allocation); + }); + } + + private void ThrowIfDisposed() => ObjectDisposedException.ThrowIf(_disposed, this); +} diff --git a/src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuDevice.Resources.cs b/src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuDevice.Resources.cs new file mode 100644 index 00000000..ea6842fb --- /dev/null +++ b/src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuDevice.Resources.cs @@ -0,0 +1,67 @@ +using Silk.NET.Vulkan; + +namespace AcDream.App.Rendering.Gpu.Vk; + +/// +/// The half of that owns sampled resources, +/// pipelines and passes. +/// +/// Split into its own file because the three V6 commits divide along +/// exactly this line: V6a lands memory, buffers, rings and the frame timeline — +/// everything in VulkanGpuDevice.cs — while textures, the descriptor +/// table and render targets arrive at V6b and pipelines, passes and readback at +/// V6c. Until each lands, the corresponding contract member throws with the +/// slice named, rather than returning something that would fail later and +/// further away. +/// +internal sealed unsafe partial class VulkanGpuDevice +{ + private void InitialiseResources(string? shaderSpirvDirectory, string? pipelineCacheDirectory) + { + _ = shaderSpirvDirectory; + _ = pipelineCacheDirectory; + } + + private void BeginFrameResources(int slotIndex) => _ = slotIndex; + + private void EndFrameResources(int slotIndex, CommandBuffer commands) + { + _ = slotIndex; + _ = commands; + } + + private void DisposeResources() + { + } + + public IGpuTimerPool Timers => throw NotYet("GPU timer scopes", "V6c"); + + public GpuTextureSlot DefaultTextureSlot => throw NotYet("the default 1x1 white table slot", "V6b"); + + public IGpuTexture CreateTexture(in GpuTextureDescription description) => + throw NotYet($"texture creation ('{description.Name}')", "V6b"); + + public IGpuSampler CreateSampler(in GpuSamplerDescription description) => + throw NotYet("sampler creation", "V6b"); + + public IGpuRenderTarget CreateRenderTarget(in GpuRenderTargetDescription description) => + throw NotYet($"render targets ('{description.Name}')", "V6b"); + + public GpuTextureSlot RegisterTexture(IGpuTexture texture, IGpuSampler sampler) => + throw NotYet("the global texture table", "V6b"); + + public void ReleaseTextureSlot(GpuTextureSlot slot) => + throw NotYet("the global texture table", "V6b"); + + public IGpuPipeline CreatePipeline(GpuPipelineDescription description) => + throw NotYet($"pipelines ('{description?.Name}')", "V6c"); + + internal IGpuPassEncoder BeginPass(VulkanGpuFrame frame, GpuPassDescription description) => + throw NotYet($"render passes ('{description.Name}')", "V6c"); + + public byte[] CaptureBackbuffer(int width, int height) => + throw NotYet("backbuffer capture", "V6c"); + + private static NotSupportedException NotYet(string what, string slice) => + new($"The Vulkan backend does not implement {what} yet; it lands at Campaign V slice {slice}."); +} diff --git a/src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuDevice.cs b/src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuDevice.cs new file mode 100644 index 00000000..1ddaee37 --- /dev/null +++ b/src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuDevice.cs @@ -0,0 +1,572 @@ +using Silk.NET.Vulkan; +using Semaphore = Silk.NET.Vulkan.Semaphore; + +namespace AcDream.App.Rendering.Gpu.Vk; + +/// +/// Raised when the swapchain must be rebuilt before another frame can be +/// recorded. Thrown rather than returned by +/// because the pinned contract has no "try" shape; hosts that own the swapchain +/// call and never see it. +/// +internal sealed class VulkanSwapchainOutOfDateException(string message) : InvalidOperationException(message); + +/// +/// What needs from whoever owns the swapchain. +/// The device deliberately does not own presentation: format/extent/present-mode +/// selection and the OUT_OF_DATE/SUBOPTIMAL policy are slice V5's, already pure +/// and unit-tested, and duplicating that judgement inside the RHI would fork it. +/// +internal interface IVulkanBackbuffer +{ + Format ImageFormat { get; } + + uint Width { get; } + + uint Height { get; } + + /// Acquires the next image. False means the caller must recreate and skip the frame. + bool TryAcquire(Semaphore acquired, out uint imageIndex); + + Image ImageAt(uint imageIndex); + + ImageView ViewAt(uint imageIndex); + + /// The semaphore a submit rendering into must signal. + Semaphore RenderCompleteAt(uint imageIndex); + + /// Presents. False means the caller should recreate before the next frame. + bool Present(uint imageIndex); +} + +/// +/// Campaign V slice V6: on Vulkan 1.3. +/// +/// The second implementation of the contract pinned at V0. Where the GL +/// backend is deliberately behaviour-preserving — it keeps +/// BufferSubData, it emulates the texture table with a buffer of bindless +/// handles — this one is where the campaign's efficiency actually lands: +/// per-frame data is written straight into persistently mapped memory the GPU +/// reads, the texture table is a real update-after-bind descriptor array with no +/// per-frame writes at all, and every pipeline is built once at startup so no +/// frame ever compiles a shader. +/// +/// Target: null means the swapchain image, literally. Plan §5.4: +/// the GL backend currently inherits whatever framebuffer is bound for a +/// null-target pass, which is a transitional behaviour that exists because +/// PrivateEntityViewportRenderer and PortalTunnelPresentation bind +/// their own FBO and then call into a ported renderer. Vulkan has no ambient +/// framebuffer to inherit and this backend never pretends otherwise: a null +/// target is the acquired swapchain image, or the multisampled scratch that +/// resolves into it. The two backends are never live in the same process, and +/// V4h owes the removal of the GL divergence. +/// +/// Frame skeleton (plan §4.8): wait the timeline back to the flight +/// window, run retirements, reset the slot's command pool, acquire, record +/// [uploads → offscreen passes → main pass], barrier to PRESENT_SRC, submit +/// signalling both the per-image render-complete semaphore and the timeline at +/// this frame's serial, present. +/// +internal sealed unsafe partial class VulkanGpuDevice : IGpuDevice +{ + /// Per-flight-slot ring capacity, matching the GL backend's 16 MiB. + internal const int DefaultRingCapacityBytesPerSlot = 16 * 1024 * 1024; + + private readonly Silk.NET.Vulkan.Vk _vk; + private readonly PhysicalDevice _physicalDevice; + private readonly Device _device; + private readonly Queue _graphicsQueue; + private readonly Queue _presentQueue; + private readonly uint _graphicsFamily; + private readonly IVulkanBackbuffer? _backbuffer; + + private readonly VulkanDeviceMemoryAllocator _allocator; + private readonly VulkanUploadQueue _uploads; + private readonly VulkanFrameFlightController _flights; + private readonly VulkanDebugNames _debugNames; + private readonly Semaphore _timeline; + + private readonly CommandPool[] _commandPools; + private readonly CommandBuffer[] _commandBuffers; + private readonly Semaphore[] _imageAcquired; + + private readonly VulkanRingBufferState[] _ringStates; + private readonly VulkanGpuBuffer[] _ringBuffers; + + private readonly List _queuedActions = []; + + private VulkanGpuFrame? _openFrame; + private uint? _acquiredImageIndex; + private bool _disposed; + + internal VulkanGpuDevice( + Silk.NET.Vulkan.Vk vk, + PhysicalDevice physicalDevice, + Device device, + Queue graphicsQueue, + Queue presentQueue, + uint graphicsFamily, + VulkanDeviceFeatureSupport features, + VulkanDeviceLimitSupport limits, + VulkanFormatSupport formats, + string deviceName, + string driverInfo, + string apiVersion, + VulkanDebugNames debugNames, + IVulkanBackbuffer? backbuffer = null, + string? shaderSpirvDirectory = null, + string? pipelineCacheDirectory = null, + int ringCapacityBytesPerSlot = DefaultRingCapacityBytesPerSlot, + int framesInFlight = VulkanFrameFlightController.DefaultFramesInFlight) + { + _vk = vk ?? throw new ArgumentNullException(nameof(vk)); + ArgumentNullException.ThrowIfNull(features); + ArgumentNullException.ThrowIfNull(limits); + ArgumentNullException.ThrowIfNull(formats); + _physicalDevice = physicalDevice; + _device = device; + _graphicsQueue = graphicsQueue; + _presentQueue = presentQueue; + _graphicsFamily = graphicsFamily; + _backbuffer = backbuffer; + _debugNames = debugNames ?? throw new ArgumentNullException(nameof(debugNames)); + DepthStencilFormat = formats.DepthStencilFormat; + ArgumentOutOfRangeException.ThrowIfNegativeOrZero(ringCapacityBytesPerSlot); + + Capabilities = new GpuCapabilityRecord + { + Backend = GpuBackendKind.Vulkan, + DeviceName = deviceName, + DriverInfo = driverInfo, + ApiVersion = apiVersion, + MaxTextureTableSlots = Math.Min( + limits.MaxDescriptorSetUpdateAfterBindSampledImages, + limits.MaxPerStageDescriptorUpdateAfterBindSampledImages), + MaxStorageBufferBindings = GpuBindingModel.StorageBindingCount, + MaxPushConstantBytes = limits.MaxPushConstantsSize, + MinStorageBufferOffsetAlignment = Math.Max(limits.MinStorageBufferOffsetAlignment, 1), + MinUniformBufferOffsetAlignment = Math.Max(limits.MinUniformBufferOffsetAlignment, 1), + MaxClipDistances = limits.MaxClipDistances, + MaxSampleCount = limits.MaxColorSampleCount, + SupportsMultiDrawIndirect = features.MultiDrawIndirect, + SupportsDrawParameters = features.ShaderDrawParameters, + SupportsTextureCompressionBc = + features.TextureCompressionBc && formats.Bc1Sampled && formats.Bc2Sampled && formats.Bc3Sampled, + SupportsTimestampQueries = limits.TimestampComputeAndGraphics, + // The one capability that is true here and false on GL, and the + // mechanism behind the campaign's CPU-cost target. + SupportsPersistentlyMappedRings = true, + }; + + var timelineType = new SemaphoreTypeCreateInfo + { + SType = StructureType.SemaphoreTypeCreateInfo, + SemaphoreType = SemaphoreType.Timeline, + InitialValue = 0, + }; + var timelineCreate = new SemaphoreCreateInfo + { + SType = StructureType.SemaphoreCreateInfo, + PNext = &timelineType, + }; + VulkanInterop.Check( + _vk.CreateSemaphore(_device, &timelineCreate, null, out _timeline), + "vkCreateSemaphore (RHI frame timeline)"); + + _flights = new VulkanFrameFlightController( + new VulkanTimelineApi(_vk, _device, _timeline), + framesInFlight); + _allocator = new VulkanDeviceMemoryAllocator(_vk, physicalDevice, _device); + _uploads = new VulkanUploadQueue(_vk, _device, _allocator, _flights, _debugNames); + + int slots = _flights.SlotCount; + _commandPools = new CommandPool[slots]; + _commandBuffers = new CommandBuffer[slots]; + _imageAcquired = new Semaphore[slots]; + _ringStates = new VulkanRingBufferState[slots]; + _ringBuffers = new VulkanGpuBuffer[slots]; + for (int slot = 0; slot < slots; slot++) + { + var poolCreate = new CommandPoolCreateInfo + { + SType = StructureType.CommandPoolCreateInfo, + QueueFamilyIndex = graphicsFamily, + }; + VulkanInterop.Check( + _vk.CreateCommandPool(_device, &poolCreate, null, out CommandPool pool), + "vkCreateCommandPool (RHI flight slot)"); + _commandPools[slot] = pool; + + var allocate = new CommandBufferAllocateInfo + { + SType = StructureType.CommandBufferAllocateInfo, + CommandPool = pool, + Level = CommandBufferLevel.Primary, + CommandBufferCount = 1, + }; + VulkanInterop.Check( + _vk.AllocateCommandBuffers(_device, &allocate, out CommandBuffer commands), + "vkAllocateCommandBuffers (RHI flight slot)"); + _commandBuffers[slot] = commands; + + var semaphoreCreate = new SemaphoreCreateInfo { SType = StructureType.SemaphoreCreateInfo }; + VulkanInterop.Check( + _vk.CreateSemaphore(_device, &semaphoreCreate, null, out Semaphore acquired), + "vkCreateSemaphore (RHI image acquired)"); + _imageAcquired[slot] = acquired; + + _ringStates[slot] = new VulkanRingBufferState((ulong)ringCapacityBytesPerSlot); + _ringBuffers[slot] = new VulkanGpuBuffer( + _vk, + _device, + _allocator, + _uploads, + _flights, + _debugNames, + new GpuBufferDescription( + $"vk-ring-slot-{slot}", + ringCapacityBytesPerSlot, + GpuBufferUsage.Storage + | GpuBufferUsage.Uniform + | GpuBufferUsage.Indirect + | GpuBufferUsage.Vertex + | GpuBufferUsage.Index, + GpuMemoryResidency.HostWritable)); + + if (!_ringBuffers[slot].IsMapped) + { + throw new NotSupportedException( + "The Vulkan per-frame ring must live in host-visible memory; this device " + + "offered no host-visible type for a HostWritable buffer."); + } + } + + InitialiseResources(shaderSpirvDirectory, pipelineCacheDirectory); + } + + public GpuBackendKind Backend => GpuBackendKind.Vulkan; + + public GpuCapabilityRecord Capabilities { get; } + + public IGpuResourceRetirementQueue Retirement => _flights; + + /// Depth+stencil format chosen by the V5 gate: D32_SFLOAT_S8_UINT, falling back to D24_UNORM_S8_UINT. + internal Format DepthStencilFormat { get; } + + internal Silk.NET.Vulkan.Vk Api => _vk; + + internal Device Handle => _device; + + internal VulkanDeviceMemoryAllocator Allocator => _allocator; + + internal VulkanUploadQueue Uploads => _uploads; + + internal VulkanDebugNames DebugNames => _debugNames; + + internal VulkanFrameFlightController Flights => _flights; + + /// Command buffer of the open frame. Only valid between BeginFrame and End. + internal CommandBuffer CurrentCommands => _commandBuffers[_flights.CurrentSlot]; + + public IGpuBuffer CreateBuffer(in GpuBufferDescription description) + { + ThrowIfDisposed(); + return new VulkanGpuBuffer(_vk, _device, _allocator, _uploads, _flights, _debugNames, description); + } + + public void QueueDeviceAction(Action action) + { + ArgumentNullException.ThrowIfNull(action); + _queuedActions.Add(action); + } + + public void ProcessDeviceActions() + { + Action[] pending = [.. _queuedActions]; + _queuedActions.Clear(); + foreach (Action action in pending) + action(); + } + + /// + /// Opens the next frame. Throws when the swapchain must be recreated first; + /// hosts that own the swapchain use instead. + /// + public IGpuFrame BeginFrame() => + TryBeginFrame(out IGpuFrame? frame) && frame is not null + ? frame + : throw new VulkanSwapchainOutOfDateException( + "The swapchain is out of date and must be recreated before another frame is recorded."); + + /// + /// Opens the next frame, returning false when the swapchain image could not + /// be acquired and the caller must recreate and skip. The flight slot is + /// waited on BEFORE the acquire, so the slot's acquire semaphore is provably + /// idle — signalling a semaphore that a pending submit still waits on is the + /// classic way to deadlock a Vulkan renderer. + /// + internal bool TryBeginFrame(out IGpuFrame? frame) + { + ThrowIfDisposed(); + frame = null; + if (_openFrame is not null) + throw new InvalidOperationException("A frame is already open; end it before beginning another."); + + long serial = _flights.BeginFrame(); + int slot = _flights.CurrentSlot; + + _uploads.ReleaseCompleted(CompletedSerial()); + _ringStates[slot].Reset(); + + _acquiredImageIndex = null; + if (_backbuffer is not null) + { + if (!_backbuffer.TryAcquire(_imageAcquired[slot], out uint imageIndex)) + { + // Release the serial by submitting nothing: the timeline must + // still reach this value or the next BeginFrame waits forever. + SignalTimelineWithoutWork(serial); + _flights.EndFrame(); + return false; + } + + _acquiredImageIndex = imageIndex; + } + + VulkanInterop.Check( + _vk.ResetCommandPool(_device, _commandPools[slot], 0), + "vkResetCommandPool"); + var begin = new CommandBufferBeginInfo + { + SType = StructureType.CommandBufferBeginInfo, + Flags = CommandBufferUsageFlags.OneTimeSubmitBit, + }; + VulkanInterop.Check( + _vk.BeginCommandBuffer(_commandBuffers[slot], &begin), + "vkBeginCommandBuffer (frame)"); + + BeginFrameResources(slot); + + var opened = new VulkanGpuFrame(this, slot, serial); + _openFrame = opened; + frame = opened; + return true; + } + + private long CompletedSerial() + { + VulkanInterop.Check( + _vk.GetSemaphoreCounterValue(_device, _timeline, out ulong value), + "vkGetSemaphoreCounterValue (upload release)"); + return (long)value; + } + + internal GpuRingAllocation AllocateRing(int slotIndex, int byteCount, GpuRingUsage usage) + { + ThrowIfDisposed(); + ulong alignment = usage switch + { + GpuRingUsage.Storage => Capabilities.MinStorageBufferOffsetAlignment, + GpuRingUsage.Uniform => Capabilities.MinUniformBufferOffsetAlignment, + GpuRingUsage.Indirect => 4, + // 16 covers every vertex stride and index type acdream uses and is + // what the staging path already aligns to, so one number rather + // than a per-format table. + _ => 16, + }; + + ulong offset = _ringStates[slotIndex].Allocate(byteCount, alignment); + VulkanGpuBuffer buffer = _ringBuffers[slotIndex]; + Span data = byteCount == 0 + ? Span.Empty + : buffer.MappedSpan.Slice((int)offset, byteCount); + return new GpuRingAllocation(buffer, (uint)offset, data); + } + + /// + /// Closes the frame: finishes any open pass work, records outstanding + /// uploads, submits with the timeline signalled at this frame's serial, and + /// presents when a backbuffer image was acquired. + /// + internal void EndFrame(VulkanGpuFrame frame) + { + if (!ReferenceEquals(_openFrame, frame)) + return; + + int slot = frame.SlotIndex; + CommandBuffer commands = _commandBuffers[slot]; + + EndFrameResources(slot, commands); + + // Anything queued after the last pass still belongs in this submission: + // it is correct, ordered, and saves it waiting a whole frame. + _uploads.Record(commands); + + if (_acquiredImageIndex is { } imageIndex && _backbuffer is not null) + { + TransitionBackbufferForPresent(commands, _backbuffer.ImageAt(imageIndex)); + } + + VulkanInterop.Check(_vk.EndCommandBuffer(commands), "vkEndCommandBuffer (frame)"); + + var commandSubmit = new CommandBufferSubmitInfo + { + SType = StructureType.CommandBufferSubmitInfo, + CommandBuffer = commands, + }; + var waitSemaphore = new SemaphoreSubmitInfo + { + SType = StructureType.SemaphoreSubmitInfo, + Semaphore = _imageAcquired[slot], + StageMask = PipelineStageFlags2.ColorAttachmentOutputBit, + }; + SemaphoreSubmitInfo* signals = stackalloc SemaphoreSubmitInfo[2]; + int signalCount = 0; + if (_acquiredImageIndex is { } presented && _backbuffer is not null) + { + signals[signalCount++] = new SemaphoreSubmitInfo + { + SType = StructureType.SemaphoreSubmitInfo, + Semaphore = _backbuffer.RenderCompleteAt(presented), + StageMask = PipelineStageFlags2.AllCommandsBit, + }; + } + + signals[signalCount++] = new SemaphoreSubmitInfo + { + SType = StructureType.SemaphoreSubmitInfo, + Semaphore = _timeline, + Value = (ulong)frame.Serial, + StageMask = PipelineStageFlags2.AllCommandsBit, + }; + + var submit = new SubmitInfo2 + { + SType = StructureType.SubmitInfo2, + WaitSemaphoreInfoCount = _acquiredImageIndex is null ? 0u : 1u, + PWaitSemaphoreInfos = _acquiredImageIndex is null ? null : &waitSemaphore, + CommandBufferInfoCount = 1, + PCommandBufferInfos = &commandSubmit, + SignalSemaphoreInfoCount = (uint)signalCount, + PSignalSemaphoreInfos = signals, + }; + VulkanInterop.Check( + _vk.QueueSubmit2(_graphicsQueue, 1, &submit, default), + "vkQueueSubmit2 (frame)"); + + _flights.EndFrame(); + _openFrame = null; + + if (_acquiredImageIndex is { } toPresent && _backbuffer is not null) + { + PresentSucceeded = _backbuffer.Present(toPresent); + _acquiredImageIndex = null; + } + } + + /// False after a present that reported the swapchain should be rebuilt. + internal bool PresentSucceeded { get; private set; } = true; + + private void TransitionBackbufferForPresent(CommandBuffer commands, Image image) + { + var barrier = new ImageMemoryBarrier2 + { + SType = StructureType.ImageMemoryBarrier2, + SrcStageMask = PipelineStageFlags2.ColorAttachmentOutputBit, + SrcAccessMask = AccessFlags2.ColorAttachmentWriteBit, + DstStageMask = PipelineStageFlags2.BottomOfPipeBit, + DstAccessMask = AccessFlags2.None, + OldLayout = ImageLayout.ColorAttachmentOptimal, + NewLayout = ImageLayout.PresentSrcKhr, + SrcQueueFamilyIndex = Silk.NET.Vulkan.Vk.QueueFamilyIgnored, + DstQueueFamilyIndex = Silk.NET.Vulkan.Vk.QueueFamilyIgnored, + Image = image, + SubresourceRange = new ImageSubresourceRange + { + AspectMask = ImageAspectFlags.ColorBit, + BaseMipLevel = 0, + LevelCount = 1, + BaseArrayLayer = 0, + LayerCount = 1, + }, + }; + var dependency = new DependencyInfo + { + SType = StructureType.DependencyInfo, + ImageMemoryBarrierCount = 1, + PImageMemoryBarriers = &barrier, + }; + _vk.CmdPipelineBarrier2(commands, &dependency); + } + + /// + /// Submits nothing but the timeline signal. Used when a frame is abandoned + /// because its swapchain image could not be acquired: the serial must still + /// complete or every later frame waits on a value nothing signals. + /// + private void SignalTimelineWithoutWork(long serial) + { + var signal = new SemaphoreSubmitInfo + { + SType = StructureType.SemaphoreSubmitInfo, + Semaphore = _timeline, + Value = (ulong)serial, + StageMask = PipelineStageFlags2.AllCommandsBit, + }; + var submit = new SubmitInfo2 + { + SType = StructureType.SubmitInfo2, + SignalSemaphoreInfoCount = 1, + PSignalSemaphoreInfos = &signal, + }; + VulkanInterop.Check( + _vk.QueueSubmit2(_graphicsQueue, 1, &submit, default), + "vkQueueSubmit2 (abandoned frame timeline signal)"); + } + + public void WaitIdle() + { + if (_disposed) + return; + VulkanInterop.Check(_vk.DeviceWaitIdle(_device), "vkDeviceWaitIdle"); + _flights.WaitForSubmittedWork(); + } + + public void Dispose() + { + if (_disposed) + return; + _disposed = true; + + _vk.DeviceWaitIdle(_device); + DisposeResources(); + _flights.DrainAll(); + + foreach (VulkanGpuBuffer ring in _ringBuffers) + ring.Dispose(); + _flights.DrainAll(); + + foreach (Semaphore semaphore in _imageAcquired) + { + if (semaphore.Handle != 0) + _vk.DestroySemaphore(_device, semaphore, null); + } + + foreach (CommandPool pool in _commandPools) + { + if (pool.Handle != 0) + _vk.DestroyCommandPool(_device, pool, null); + } + + _uploads.Dispose(); + _flights.DrainAll(); + + if (_timeline.Handle != 0) + _vk.DestroySemaphore(_device, _timeline, null); + + _flights.Dispose(); + _allocator.Dispose(); + } + + private void ThrowIfDisposed() => ObjectDisposedException.ThrowIf(_disposed, this); +} diff --git a/src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuFrame.cs b/src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuFrame.cs new file mode 100644 index 00000000..3ec2eec2 --- /dev/null +++ b/src/AcDream.App/Rendering/Gpu/Vk/VulkanGpuFrame.cs @@ -0,0 +1,66 @@ +namespace AcDream.App.Rendering.Gpu.Vk; + +/// +/// Campaign V slice V6a: one frame's recording context on Vulkan. +/// +/// Thin on purpose. The frame owns no Vulkan objects: the command buffer, +/// the ring and the timeline all belong to , which +/// keeps them alive across frames and hands this object the slot index they are +/// addressed by. What is here is the contract's shape — one open pass at a time, +/// idempotent . +/// +/// being idempotent with Dispose is +/// load-bearing rather than tidy: a renderer that throws mid-frame must still +/// close its flight slot, or the ring never rewinds and the next +/// BeginFrame waits on a timeline value nothing will ever signal. +/// +internal sealed class VulkanGpuFrame : IGpuFrame +{ + private readonly VulkanGpuDevice _device; + private IGpuPassEncoder? _openPass; + private bool _ended; + + internal VulkanGpuFrame(VulkanGpuDevice device, int slotIndex, long serial) + { + _device = device ?? throw new ArgumentNullException(nameof(device)); + SlotIndex = slotIndex; + Serial = serial; + } + + public int SlotIndex { get; } + + public long Serial { get; } + + public GpuRingAllocation AllocateRing(int byteCount, GpuRingUsage usage) => + _device.AllocateRing(SlotIndex, byteCount, usage); + + public IGpuPassEncoder BeginPass(GpuPassDescription description) + { + ArgumentNullException.ThrowIfNull(description); + if (_openPass is not null) + { + throw new InvalidOperationException( + "A pass is already open on this frame; dispose it before beginning another."); + } + + IGpuPassEncoder encoder = _device.BeginPass(this, description); + _openPass = encoder; + return encoder; + } + + internal void ClosePass(IGpuPassEncoder encoder) + { + if (ReferenceEquals(_openPass, encoder)) + _openPass = null; + } + + public void End() + { + if (_ended) + return; + _ended = true; + _device.EndFrame(this); + } + + public void Dispose() => End(); +} diff --git a/src/AcDream.App/Rendering/Gpu/Vk/VulkanMemoryModel.cs b/src/AcDream.App/Rendering/Gpu/Vk/VulkanMemoryModel.cs new file mode 100644 index 00000000..53b53e97 --- /dev/null +++ b/src/AcDream.App/Rendering/Gpu/Vk/VulkanMemoryModel.cs @@ -0,0 +1,411 @@ +using Silk.NET.Vulkan; + +namespace AcDream.App.Rendering.Gpu.Vk; + +/// +/// One suballocation: which block it came from and where inside it. A +/// range owns its whole block, which is how +/// allocations at or above the dedicated threshold are expressed without a +/// second code path. +/// +internal readonly record struct VulkanMemoryRange( + int BlockIndex, + ulong OffsetBytes, + ulong SizeBytes, + bool IsDedicated); + +/// +/// Campaign V slice V6a, plan §4.2: the free list inside one device-memory +/// block. +/// +/// A first-fit list of free ranges, coalescing on release. It is +/// deliberately pure — no VkDeviceMemory, no driver call — because the +/// interesting failure modes of an allocator (fragmentation, a mis-coalesced +/// neighbour, an alignment that eats the tail of a block) are arithmetic, and +/// arithmetic can be tested with no GPU in the room. The driver-facing half +/// lives in and does nothing this +/// class has not already decided. +/// +/// First fit rather than best fit is a considered choice, not laziness: +/// acdream's allocation profile is a handful of long-lived giants (two mesh +/// arena buffers, the staging ring, the per-flight ring buffers) plus a texture +/// pool whose members are all similar sizes. Best fit buys its reduced external +/// fragmentation by leaving many tiny unusable holes, which is the worse trade +/// for exactly that profile. +/// +internal sealed class VulkanMemoryBlockFreeList +{ + private readonly List _free = []; + + private struct Range(ulong offset, ulong size) + { + public ulong Offset = offset; + public ulong Size = size; + public readonly ulong End => Offset + Size; + } + + internal VulkanMemoryBlockFreeList(ulong capacityBytes) + { + if (capacityBytes == 0) + throw new ArgumentOutOfRangeException(nameof(capacityBytes), "A memory block cannot be empty."); + CapacityBytes = capacityBytes; + _free.Add(new Range(0, capacityBytes)); + } + + internal ulong CapacityBytes { get; } + + /// Bytes currently handed out, including alignment padding left in front of a range. + internal ulong UsedBytes { get; private set; } + + internal ulong FreeBytes => CapacityBytes - UsedBytes; + + /// Largest single allocation this block could still satisfy at alignment 1. + internal ulong LargestFreeBytes + { + get + { + ulong largest = 0; + foreach (Range range in _free) + largest = Math.Max(largest, range.Size); + return largest; + } + } + + internal int FreeRangeCount => _free.Count; + + /// + /// First-fit placement. The aligned offset can sit above a free range's + /// start, in which case the padding in front stays part of the allocation + /// (it is released with it) rather than becoming an unreachable hole. + /// + internal bool TryAllocate(ulong sizeBytes, ulong alignmentBytes, out ulong offsetBytes) + { + offsetBytes = 0; + if (sizeBytes == 0) + return false; + + for (int i = 0; i < _free.Count; i++) + { + Range range = _free[i]; + ulong aligned = AlignUp(range.Offset, alignmentBytes); + ulong padding = aligned - range.Offset; + if (padding > range.Size || range.Size - padding < sizeBytes) + continue; + + // The allocation owns [range.Offset, aligned + size) so releasing it + // returns the alignment padding too. + ulong consumed = padding + sizeBytes; + if (consumed == range.Size) + { + _free.RemoveAt(i); + } + else + { + range.Offset += consumed; + range.Size -= consumed; + _free[i] = range; + } + + UsedBytes += consumed; + offsetBytes = aligned; + AllocatedPaddingByOffset[aligned] = padding; + return true; + } + + return false; + } + + /// + /// Alignment padding carried in front of each live allocation, so + /// can return the same bytes + /// took. Keyed by the aligned offset the caller was handed, which is the + /// only value the caller ever sees. + /// + private Dictionary AllocatedPaddingByOffset { get; } = []; + + /// Returns at to the free list, coalescing neighbours. + internal void Free(ulong offsetBytes, ulong sizeBytes) + { + if (sizeBytes == 0) + return; + + ulong padding = 0; + if (AllocatedPaddingByOffset.Remove(offsetBytes, out ulong recorded)) + padding = recorded; + + ulong start = offsetBytes - padding; + ulong end = offsetBytes + sizeBytes; + ulong length = end - start; + if (length > UsedBytes) + { + throw new InvalidOperationException( + $"Releasing [{start}, {end}) would free more than the {UsedBytes} bytes this block " + + "has handed out — the same range was probably released twice."); + } + + UsedBytes -= length; + + int insertAt = _free.FindIndex(range => range.Offset > start); + if (insertAt < 0) + insertAt = _free.Count; + _free.Insert(insertAt, new Range(start, length)); + Coalesce(insertAt); + } + + private void Coalesce(int index) + { + if (index > 0 && _free[index - 1].End == _free[index].Offset) + { + Range merged = _free[index - 1]; + merged.Size += _free[index].Size; + _free[index - 1] = merged; + _free.RemoveAt(index); + index--; + } + + if (index + 1 < _free.Count && _free[index].End == _free[index + 1].Offset) + { + Range merged = _free[index]; + merged.Size += _free[index + 1].Size; + _free[index] = merged; + _free.RemoveAt(index + 1); + } + } + + internal static ulong AlignUp(ulong value, ulong alignmentBytes) => + alignmentBytes <= 1 ? value : (value + alignmentBytes - 1) / alignmentBytes * alignmentBytes; +} + +/// +/// Campaign V slice V6a, plan §4.2: every block belonging to one Vulkan memory +/// type, and the policy that decides when a request gets a block of its own. +/// +/// Pure by the same argument as : +/// answers "no" rather than allocating, and the +/// driver-facing owner responds by calling with real +/// device memory and asking again. That keeps every placement decision +/// reachable from a unit test. +/// +internal sealed class VulkanMemoryTypePool +{ + /// Plan §4.2's block size: 128 MiB device-local blocks per memory type. + internal const ulong DefaultBlockSizeBytes = 128UL * 1024 * 1024; + + /// Plan §4.2: allocations at or above this size take a block of their own. + internal const ulong DefaultDedicatedThresholdBytes = 32UL * 1024 * 1024; + + private readonly List _blocks = []; + private readonly HashSet _dedicatedBlocks = []; + + internal VulkanMemoryTypePool( + uint memoryTypeIndex, + ulong blockSizeBytes = DefaultBlockSizeBytes, + ulong dedicatedThresholdBytes = DefaultDedicatedThresholdBytes) + { + ArgumentOutOfRangeException.ThrowIfZero(blockSizeBytes); + ArgumentOutOfRangeException.ThrowIfZero(dedicatedThresholdBytes); + MemoryTypeIndex = memoryTypeIndex; + BlockSizeBytes = blockSizeBytes; + DedicatedThresholdBytes = dedicatedThresholdBytes; + } + + internal uint MemoryTypeIndex { get; } + + internal ulong BlockSizeBytes { get; } + + internal ulong DedicatedThresholdBytes { get; } + + internal int BlockCount => _blocks.Count; + + /// Blocks that still exist, i.e. have been added and not retired. + internal int LiveBlockCount => _blocks.Count(block => block is not null); + + internal bool IsDedicatedSize(ulong sizeBytes) => sizeBytes >= DedicatedThresholdBytes; + + /// Capacity a new block must have to satisfy . + internal ulong BlockCapacityFor(ulong sizeBytes) => + IsDedicatedSize(sizeBytes) ? sizeBytes : BlockSizeBytes; + + /// + /// Places in an existing block. Returns false + /// when the caller must create a new block of + /// bytes and retry. + /// + internal bool TryAllocate(ulong sizeBytes, ulong alignmentBytes, out VulkanMemoryRange range) + { + range = default; + if (sizeBytes == 0) + return false; + + // A dedicated-size request never shares: hunting for a hole big enough + // in a shared block would find one only in a nearly empty block, and + // taking it would strand the rest. + if (IsDedicatedSize(sizeBytes)) + return false; + + for (int i = 0; i < _blocks.Count; i++) + { + if (_blocks[i] is not { } block || _dedicatedBlocks.Contains(i)) + continue; + if (block.TryAllocate(sizeBytes, alignmentBytes, out ulong offset)) + { + range = new VulkanMemoryRange(i, offset, sizeBytes, IsDedicated: false); + return true; + } + } + + return false; + } + + /// Registers a newly created block and returns its index. + internal int AddBlock(ulong capacityBytes, bool dedicated) + { + _blocks.Add(new VulkanMemoryBlockFreeList(capacityBytes)); + int index = _blocks.Count - 1; + if (dedicated) + _dedicatedBlocks.Add(index); + return index; + } + + /// Claims the whole of a block that was created for one dedicated allocation. + internal VulkanMemoryRange AllocateWholeBlock(int blockIndex, ulong sizeBytes) + { + VulkanMemoryBlockFreeList block = BlockAt(blockIndex); + if (!block.TryAllocate(sizeBytes, 1, out ulong offset) || offset != 0) + { + throw new InvalidOperationException( + $"Block {blockIndex} was created for a dedicated {sizeBytes}-byte allocation " + + "but could not satisfy it at offset 0."); + } + + return new VulkanMemoryRange(blockIndex, 0, sizeBytes, IsDedicated: true); + } + + /// + /// Releases a range. Returns true when the block became completely empty + /// AND is dedicated, which is the caller's signal to free the underlying + /// VkDeviceMemory; shared blocks are kept for reuse. + /// + internal bool Free(in VulkanMemoryRange range) + { + VulkanMemoryBlockFreeList block = BlockAt(range.BlockIndex); + block.Free(range.OffsetBytes, range.SizeBytes); + if (block.UsedBytes != 0 || !_dedicatedBlocks.Contains(range.BlockIndex)) + return false; + + _blocks[range.BlockIndex] = null; + _dedicatedBlocks.Remove(range.BlockIndex); + return true; + } + + internal ulong UsedBytes + { + get + { + ulong total = 0; + foreach (VulkanMemoryBlockFreeList? block in _blocks) + total += block?.UsedBytes ?? 0; + return total; + } + } + + internal ulong CapacityBytes + { + get + { + ulong total = 0; + foreach (VulkanMemoryBlockFreeList? block in _blocks) + total += block?.CapacityBytes ?? 0; + return total; + } + } + + private VulkanMemoryBlockFreeList BlockAt(int index) => + index >= 0 && index < _blocks.Count && _blocks[index] is { } block + ? block + : throw new ArgumentOutOfRangeException( + nameof(index), + $"Memory block {index} does not exist in the pool for memory type {MemoryTypeIndex}."); +} + +/// +/// Campaign V slice V6a, plan §4.3: which Vulkan memory type backs each +/// . +/// +/// The row is the one that +/// earns the campaign's CPU win. It prefers a type that is BOTH device-local and +/// host-visible — resizable BAR, present on the RX 9070 XT, on RADV and on +/// modern NVIDIA — so per-frame data is written once, straight into memory the +/// GPU reads. When no such type exists the fallback is ordinary host-visible +/// coherent memory, which still beats BufferSubData but gives up the +/// device-local read. +/// +/// Pure and table-driven so the preference order is a fact a test can +/// assert against a synthesised VkPhysicalDeviceMemoryProperties, rather +/// than something only the RX 9070 XT can confirm. +/// +internal static class VulkanMemoryTypeSelection +{ + /// The property masks tried in order for a residency class. First match wins. + internal static IReadOnlyList PreferenceOrder(GpuMemoryResidency residency) => + residency switch + { + GpuMemoryResidency.DeviceLocal => + [ + MemoryPropertyFlags.DeviceLocalBit, + 0, + ], + GpuMemoryResidency.HostWritable => + [ + // ReBAR: device-local AND host-visible. The whole point of §4.3. + MemoryPropertyFlags.DeviceLocalBit + | MemoryPropertyFlags.HostVisibleBit + | MemoryPropertyFlags.HostCoherentBit, + MemoryPropertyFlags.HostVisibleBit | MemoryPropertyFlags.HostCoherentBit, + MemoryPropertyFlags.HostVisibleBit, + ], + GpuMemoryResidency.HostReadable => + [ + // Cached memory is dramatically faster to read back on the CPU; + // coherent-only is correct but slow, so it is the fallback. + MemoryPropertyFlags.HostVisibleBit + | MemoryPropertyFlags.HostCoherentBit + | MemoryPropertyFlags.HostCachedBit, + MemoryPropertyFlags.HostVisibleBit | MemoryPropertyFlags.HostCoherentBit, + MemoryPropertyFlags.HostVisibleBit, + ], + _ => throw new ArgumentOutOfRangeException(nameof(residency), residency, "Unknown residency class."), + }; + + /// Whether a residency class must end up on memory the CPU can map. + internal static bool RequiresMapping(GpuMemoryResidency residency) => + residency is GpuMemoryResidency.HostWritable or GpuMemoryResidency.HostReadable; + + /// + /// Chooses a memory type index for out of the + /// types permits. Returns null when the + /// device offers nothing usable, which the caller turns into a named + /// exception rather than a silent wrong-heap allocation. + /// + internal static uint? Choose( + IReadOnlyList memoryTypeProperties, + uint allowedTypeBits, + GpuMemoryResidency residency) + { + ArgumentNullException.ThrowIfNull(memoryTypeProperties); + + foreach (MemoryPropertyFlags required in PreferenceOrder(residency)) + { + for (int i = 0; i < memoryTypeProperties.Count && i < 32; i++) + { + if ((allowedTypeBits & (1u << i)) == 0) + continue; + if ((memoryTypeProperties[i] & required) != required) + continue; + return (uint)i; + } + } + + return null; + } +} diff --git a/src/AcDream.App/Rendering/Gpu/Vk/VulkanRingBufferState.cs b/src/AcDream.App/Rendering/Gpu/Vk/VulkanRingBufferState.cs new file mode 100644 index 00000000..55095648 --- /dev/null +++ b/src/AcDream.App/Rendering/Gpu/Vk/VulkanRingBufferState.cs @@ -0,0 +1,189 @@ +namespace AcDream.App.Rendering.Gpu.Vk; + +/// +/// Campaign V slice V6a: one flight slot's per-frame upload ring, as pure +/// bookkeeping. +/// +/// Deliberately simpler than its GL sibling +/// (GlRingBufferState), and the difference is the campaign's whole +/// point. On GL a ring allocation writes into a managed staging array which is +/// later copied into a GL buffer, so that class has to track a dirty watermark +/// and prove the copy never overlaps an in-flight read. Here the allocation +/// hands back a span of memory the GPU reads directly, so there is no upload +/// step to track and no dirty range to flush: what is left is a cursor. +/// +/// The forward-only invariant still matters and is still enforced. The +/// slot's buffer is being read by the frame that was submitted from this slot +/// two frames ago; the timeline wait in +/// is what makes rewinding +/// the cursor safe, and may only be called after it. +/// +internal sealed class VulkanRingBufferState +{ + private ulong _cursor; + + internal VulkanRingBufferState(ulong capacityBytes) + { + ArgumentOutOfRangeException.ThrowIfZero(capacityBytes); + CapacityBytes = capacityBytes; + } + + internal ulong CapacityBytes { get; } + + /// Bytes handed out since the last , including alignment padding. + internal ulong AllocatedBytes => _cursor; + + /// High-water mark across the ring's lifetime — the number that says whether the capacity is right. + internal ulong PeakAllocatedBytes { get; private set; } + + /// + /// Reserves bytes at the required alignment. + /// Throws rather than truncating: a silently shortened allocation would + /// corrupt the frame invisibly, and the exception names the capacity to set. + /// + internal ulong Allocate(int byteCount, ulong alignmentBytes) + { + ArgumentOutOfRangeException.ThrowIfNegative(byteCount); + ulong aligned = VulkanMemoryBlockFreeList.AlignUp(_cursor, alignmentBytes); + ulong end = aligned + (ulong)byteCount; + if (end > CapacityBytes) + { + throw new InvalidOperationException( + $"Ring allocation of {byteCount} bytes at aligned offset {aligned} needs {end} bytes; " + + $"this flight slot's ring is {CapacityBytes} bytes. Increase the per-slot ring " + + "capacity (VulkanGpuDevice's ringCapacityBytesPerSlot)."); + } + + _cursor = end; + PeakAllocatedBytes = Math.Max(PeakAllocatedBytes, end); + return aligned; + } + + /// + /// Rewinds for a new frame. Only legal once the GPU has finished the frame + /// that last used this slot — the caller proves that with a timeline wait. + /// + internal void Reset() => _cursor = 0; +} + +/// +/// Campaign V slice V6a, plan §4.3: the staging ring's bookkeeping. +/// +/// One persistently mapped host-visible buffer carries every upload +/// destined for device-local memory. Allocations are handed out linearly and +/// released in frame order: a region taken while recording frame N is reclaimed +/// when frame N retires, which is exactly the guarantee the timeline semaphore +/// already provides. +/// +/// It is a genuine ring — the cursor wraps — because uploads are bursty +/// (a landblock arriving is thousands of small copies) and a linear allocator +/// would need a capacity sized for the worst burst rather than the worst +/// in-flight window. Wrapping only happens when the bytes being overwritten +/// belong to a frame the GPU has finished, which is what +/// establishes. +/// +/// When the ring cannot satisfy a request — because it is larger than the +/// whole ring, or because everything still in flight is holding the space — the +/// allocator answers no and the caller takes a temporary dedicated staging +/// buffer retired through the ledger. That is the §4.3 escape for oversized +/// uploads, applied to the one other case that has the same shape. It is not a +/// fallback that hides a problem: the upload is still correct, still ordered, +/// and still freed on retirement; it just costs one extra allocation. +/// +internal sealed class VulkanStagingRingState +{ + private readonly List _pending = []; + private ulong _head; + private ulong _tail; + private ulong _live; + + private readonly record struct PendingSegment(long Serial, ulong SizeBytes); + + internal VulkanStagingRingState(ulong capacityBytes) + { + ArgumentOutOfRangeException.ThrowIfZero(capacityBytes); + CapacityBytes = capacityBytes; + } + + internal ulong CapacityBytes { get; } + + /// Bytes reserved by frames that have not yet retired. + internal ulong LiveBytes => _live; + + internal int PendingSegmentCount => _pending.Count; + + /// + /// Reserves bytes for the frame identified by + /// . Returns false when the ring cannot serve the + /// request without overwriting bytes an unretired frame still owns. + /// + internal bool TryAllocate(int byteCount, ulong alignmentBytes, long serial, out ulong offsetBytes) + { + ArgumentOutOfRangeException.ThrowIfNegative(byteCount); + offsetBytes = 0; + if (byteCount == 0) + return true; + + var size = (ulong)byteCount; + if (size > CapacityBytes) + return false; + + ulong aligned = VulkanMemoryBlockFreeList.AlignUp(_head, alignmentBytes); + ulong consumed; + if (aligned + size > CapacityBytes) + { + // Wrapping costs the tail of the ring; charge it to this segment so + // Release returns it with the rest. + ulong wasted = CapacityBytes - _head; + aligned = 0; + consumed = wasted + size; + } + else + { + consumed = (aligned - _head) + size; + } + + if (_live + consumed > CapacityBytes) + return false; + + _head = (aligned + size) % CapacityBytes; + _live += consumed; + offsetBytes = aligned; + + if (_pending.Count > 0 && _pending[^1].Serial == serial) + { + PendingSegment last = _pending[^1]; + _pending[^1] = last with { SizeBytes = last.SizeBytes + consumed }; + } + else + { + _pending.Add(new PendingSegment(serial, consumed)); + } + + return true; + } + + /// Reclaims every segment taken by a frame at or below . + internal void Release(long completedSerial) + { + int released = 0; + while (released < _pending.Count && _pending[released].Serial <= completedSerial) + { + _tail = (_tail + _pending[released].SizeBytes) % CapacityBytes; + _live -= Math.Min(_live, _pending[released].SizeBytes); + released++; + } + + if (released > 0) + _pending.RemoveRange(0, released); + } + + /// Drops all bookkeeping. Only legal when the device is idle. + internal void Reset() + { + _pending.Clear(); + _head = 0; + _tail = 0; + _live = 0; + } +} diff --git a/src/AcDream.App/Rendering/Gpu/Vk/VulkanUploadQueue.cs b/src/AcDream.App/Rendering/Gpu/Vk/VulkanUploadQueue.cs new file mode 100644 index 00000000..6c384391 --- /dev/null +++ b/src/AcDream.App/Rendering/Gpu/Vk/VulkanUploadQueue.cs @@ -0,0 +1,392 @@ +using Silk.NET.Vulkan; +using Buffer = Silk.NET.Vulkan.Buffer; + +namespace AcDream.App.Rendering.Gpu.Vk; + +/// +/// Campaign V slice V6a, plan §4.3 and §4.8: every transfer that has been asked +/// for but not yet recorded, plus the staging memory those transfers read from. +/// +/// Why a queue rather than an immediate copy. Uploads arrive from +/// the streaming and texture-cache code at arbitrary moments, including with no +/// frame open at all. Vulkan copies must be recorded into a command buffer, and +/// they must be recorded OUTSIDE a dynamic-rendering block. So requests +/// accumulate here and are drained into the frame's command buffer at the one +/// moment both conditions hold: immediately before a pass begins. That is the +/// direct analogue of the GL backend's "flush immediately before every draw" +/// discipline, moved to the coarser granularity Vulkan actually needs. +/// +/// One batched barrier, not one per copy. The drain emits a single +/// buffer memory barrier covering every copy it recorded, moving the whole batch +/// from transfer writes to the vertex/index/indirect/shader reads that follow. +/// Plan §4.8 budgets four to six barriers per frame; this is one of them. +/// +/// Staging exhaustion is not an error. When the ring cannot serve a +/// request — the payload is larger than the whole ring, or unretired frames hold +/// the space — the upload takes a temporary dedicated staging buffer that is +/// retired through the ledger. The transfer is equally correct and equally +/// ordered; it just costs one allocation. See +/// for why that is a policy rather than a +/// workaround. +/// +internal sealed unsafe class VulkanUploadQueue : IDisposable +{ + /// Plan §4.3: a 48 MiB persistently mapped staging ring. + internal const ulong DefaultStagingCapacityBytes = 48UL * 1024 * 1024; + + private readonly Silk.NET.Vulkan.Vk _vk; + private readonly Device _device; + private readonly VulkanDeviceMemoryAllocator _allocator; + private readonly VulkanFrameFlightController _flights; + private readonly VulkanStagingRingState _ringState; + private readonly VulkanDebugNames _debugNames; + + private readonly Buffer _stagingBuffer; + private readonly VulkanAllocation _stagingAllocation; + + private readonly List _bufferCopies = []; + private readonly List _imageCopies = []; + private readonly List _temporaries = []; + + private bool _disposed; + + private readonly record struct BufferCopy2(Buffer Source, Buffer Destination, ulong SourceOffset, ulong DestinationOffset, ulong SizeBytes); + + private readonly record struct ImageCopy2( + Buffer Source, + ulong SourceOffset, + Image Destination, + uint MipLevel, + uint Layer, + uint Width, + uint Height, + uint MipLevelCount, + uint LayerCount); + + private readonly record struct TemporaryStaging(Buffer Buffer, VulkanAllocation Allocation); + + internal VulkanUploadQueue( + Silk.NET.Vulkan.Vk vk, + Device device, + VulkanDeviceMemoryAllocator allocator, + VulkanFrameFlightController flights, + VulkanDebugNames debugNames, + ulong stagingCapacityBytes = DefaultStagingCapacityBytes) + { + _vk = vk ?? throw new ArgumentNullException(nameof(vk)); + _device = device; + _allocator = allocator ?? throw new ArgumentNullException(nameof(allocator)); + _flights = flights ?? throw new ArgumentNullException(nameof(flights)); + _debugNames = debugNames ?? throw new ArgumentNullException(nameof(debugNames)); + _ringState = new VulkanStagingRingState(stagingCapacityBytes); + + (_stagingBuffer, _stagingAllocation) = CreateHostBuffer( + stagingCapacityBytes, + BufferUsageFlags.TransferSrcBit, + "vk-staging-ring"); + } + + /// Images whose layout must be moved to TRANSFER_DST before the drain and to SHADER_READ after it. + private readonly HashSet _imagesNeedingBarrier = []; + + internal int PendingBufferCopyCount => _bufferCopies.Count; + + internal int PendingImageCopyCount => _imageCopies.Count; + + internal ulong StagingLiveBytes => _ringState.LiveBytes; + + /// Stages and queues a copy into . + internal void StageBufferWrite( + Buffer destination, + ulong destinationOffsetBytes, + ReadOnlySpan data, + string ownerName) + { + ObjectDisposedException.ThrowIf(_disposed, this); + if (data.IsEmpty) + return; + + (Buffer source, ulong sourceOffset) = Stage(data, ownerName); + _bufferCopies.Add(new BufferCopy2( + source, + destination, + sourceOffset, + destinationOffsetBytes, + (ulong)data.Length)); + } + + /// Stages and queues a copy into one mip level of one array layer. + internal void StageImageWrite( + Image destination, + int mipLevel, + int layer, + int width, + int height, + int mipLevelCount, + int layerCount, + ReadOnlySpan data, + string ownerName) + { + ObjectDisposedException.ThrowIf(_disposed, this); + if (data.IsEmpty) + return; + + // Vulkan requires a buffer-to-image copy's source offset to be a + // multiple of 4 and of the texel/block size; 16 covers every format + // acdream uses (BC3's 16-byte block is the largest). + (Buffer source, ulong sourceOffset) = Stage(data, ownerName, alignmentBytes: 16); + _imageCopies.Add(new ImageCopy2( + source, + sourceOffset, + destination, + (uint)mipLevel, + (uint)layer, + (uint)width, + (uint)height, + (uint)mipLevelCount, + (uint)layerCount)); + _imagesNeedingBarrier.Add(destination); + } + + /// Queues a device-side buffer copy — the mesh arena's grow-and-copy migration. + internal void EnqueueBufferCopy( + Buffer source, + Buffer destination, + ulong sourceOffsetBytes, + ulong destinationOffsetBytes, + ulong byteCount) + { + ObjectDisposedException.ThrowIf(_disposed, this); + if (byteCount == 0) + return; + _bufferCopies.Add(new BufferCopy2( + source, + destination, + sourceOffsetBytes, + destinationOffsetBytes, + byteCount)); + } + + /// + /// Records every pending transfer into and + /// clears the queue. Must be called outside a dynamic-rendering block. + /// Returns false when there was nothing to do. + /// + internal bool Record(CommandBuffer commands) + { + if (_bufferCopies.Count == 0 && _imageCopies.Count == 0) + return false; + + if (_imagesNeedingBarrier.Count > 0) + TransitionImages(commands, ImageLayout.Undefined, ImageLayout.TransferDstOptimal, toTransfer: true); + + foreach (BufferCopy2 copy in _bufferCopies) + { + var region = new BufferCopy + { + SrcOffset = copy.SourceOffset, + DstOffset = copy.DestinationOffset, + Size = copy.SizeBytes, + }; + _vk.CmdCopyBuffer(commands, copy.Source, copy.Destination, 1, ®ion); + } + + foreach (ImageCopy2 copy in _imageCopies) + { + var region = new BufferImageCopy + { + BufferOffset = copy.SourceOffset, + BufferRowLength = 0, + BufferImageHeight = 0, + ImageSubresource = new ImageSubresourceLayers + { + AspectMask = ImageAspectFlags.ColorBit, + MipLevel = copy.MipLevel, + BaseArrayLayer = copy.Layer, + LayerCount = 1, + }, + ImageOffset = new Offset3D(0, 0, 0), + ImageExtent = new Extent3D(copy.Width, copy.Height, 1), + }; + _vk.CmdCopyBufferToImage( + commands, + copy.Source, + copy.Destination, + ImageLayout.TransferDstOptimal, + 1, + ®ion); + } + + if (_imagesNeedingBarrier.Count > 0) + TransitionImages(commands, ImageLayout.TransferDstOptimal, ImageLayout.ShaderReadOnlyOptimal, toTransfer: false); + + // One buffer barrier for the whole batch: transfer writes become + // readable by every consumer stage a copied buffer can feed. + if (_bufferCopies.Count > 0) + { + var barrier = new MemoryBarrier2 + { + SType = StructureType.MemoryBarrier2, + SrcStageMask = PipelineStageFlags2.AllTransferBit, + SrcAccessMask = AccessFlags2.TransferWriteBit, + DstStageMask = PipelineStageFlags2.VertexInputBit + | PipelineStageFlags2.VertexShaderBit + | PipelineStageFlags2.FragmentShaderBit + | PipelineStageFlags2.DrawIndirectBit, + DstAccessMask = AccessFlags2.VertexAttributeReadBit + | AccessFlags2.IndexReadBit + | AccessFlags2.ShaderReadBit + | AccessFlags2.UniformReadBit + | AccessFlags2.IndirectCommandReadBit, + }; + var dependency = new DependencyInfo + { + SType = StructureType.DependencyInfo, + MemoryBarrierCount = 1, + PMemoryBarriers = &barrier, + }; + _vk.CmdPipelineBarrier2(commands, &dependency); + } + + _bufferCopies.Clear(); + _imageCopies.Clear(); + _imagesNeedingBarrier.Clear(); + return true; + } + + private void TransitionImages( + CommandBuffer commands, + ImageLayout oldLayout, + ImageLayout newLayout, + bool toTransfer) + { + int count = _imagesNeedingBarrier.Count; + var barriers = new ImageMemoryBarrier2[count]; + int index = 0; + foreach (Image image in _imagesNeedingBarrier) + { + barriers[index++] = new ImageMemoryBarrier2 + { + SType = StructureType.ImageMemoryBarrier2, + SrcStageMask = toTransfer + ? PipelineStageFlags2.AllCommandsBit + : PipelineStageFlags2.AllTransferBit, + SrcAccessMask = toTransfer ? AccessFlags2.None : AccessFlags2.TransferWriteBit, + DstStageMask = toTransfer + ? PipelineStageFlags2.AllTransferBit + : PipelineStageFlags2.FragmentShaderBit | PipelineStageFlags2.VertexShaderBit, + DstAccessMask = toTransfer ? AccessFlags2.TransferWriteBit : AccessFlags2.ShaderReadBit, + // Undefined discards existing contents, which is right for the + // first upload into a fresh image and wrong for an incremental + // one; incremental array-layer fills therefore name the layout + // they are already in. + OldLayout = oldLayout, + NewLayout = newLayout, + SrcQueueFamilyIndex = Silk.NET.Vulkan.Vk.QueueFamilyIgnored, + DstQueueFamilyIndex = Silk.NET.Vulkan.Vk.QueueFamilyIgnored, + Image = image, + SubresourceRange = new ImageSubresourceRange + { + AspectMask = ImageAspectFlags.ColorBit, + BaseMipLevel = 0, + LevelCount = Silk.NET.Vulkan.Vk.RemainingMipLevels, + BaseArrayLayer = 0, + LayerCount = Silk.NET.Vulkan.Vk.RemainingArrayLayers, + }, + }; + } + + fixed (ImageMemoryBarrier2* first = barriers) + { + var dependency = new DependencyInfo + { + SType = StructureType.DependencyInfo, + ImageMemoryBarrierCount = (uint)count, + PImageMemoryBarriers = first, + }; + _vk.CmdPipelineBarrier2(commands, &dependency); + } + } + + /// Reclaims staging bytes and temporary buffers belonging to completed frames. + internal void ReleaseCompleted(long completedSerial) => _ringState.Release(completedSerial); + + private (Buffer Buffer, ulong Offset) Stage( + ReadOnlySpan data, + string ownerName, + ulong alignmentBytes = 4) + { + long serial = _flights.OpenSerial != 0 ? _flights.OpenSerial : _flights.SubmittedSerial + 1; + if (_ringState.TryAllocate(data.Length, alignmentBytes, serial, out ulong offset)) + { + data.CopyTo(_stagingAllocation.AsSpan().Slice((int)offset, data.Length)); + return (_stagingBuffer, offset); + } + + (Buffer temporary, VulkanAllocation allocation) = CreateHostBuffer( + (ulong)data.Length, + BufferUsageFlags.TransferSrcBit, + $"vk-staging-temp-{ownerName}"); + data.CopyTo(allocation.AsSpan()); + _temporaries.Add(new TemporaryStaging(temporary, allocation)); + + Buffer captured = temporary; + VulkanAllocation capturedAllocation = allocation; + _flights.Retire(() => + { + _vk.DestroyBuffer(_device, captured, null); + _allocator.Free(capturedAllocation); + _temporaries.RemoveAll(entry => entry.Buffer.Handle == captured.Handle); + }); + return (temporary, 0); + } + + private (Buffer Buffer, VulkanAllocation Allocation) CreateHostBuffer( + ulong sizeBytes, + BufferUsageFlags usage, + string name) + { + var create = new BufferCreateInfo + { + SType = StructureType.BufferCreateInfo, + Size = sizeBytes, + Usage = usage, + SharingMode = SharingMode.Exclusive, + }; + VulkanInterop.Check( + _vk.CreateBuffer(_device, &create, null, out Buffer buffer), + $"vkCreateBuffer ({name})"); + + _vk.GetBufferMemoryRequirements(_device, buffer, out MemoryRequirements requirements); + VulkanAllocation allocation = _allocator.Allocate( + requirements, + GpuMemoryResidency.HostWritable, + name); + VulkanInterop.Check( + _vk.BindBufferMemory(_device, buffer, allocation.Memory, allocation.OffsetBytes), + $"vkBindBufferMemory ({name})"); + _debugNames.NameBuffer(buffer, name); + return (buffer, allocation); + } + + public void Dispose() + { + if (_disposed) + return; + _disposed = true; + + foreach (TemporaryStaging temporary in _temporaries) + { + _vk.DestroyBuffer(_device, temporary.Buffer, null); + _allocator.Free(temporary.Allocation); + } + _temporaries.Clear(); + + _vk.DestroyBuffer(_device, _stagingBuffer, null); + _allocator.Free(_stagingAllocation); + _ringState.Reset(); + _bufferCopies.Clear(); + _imageCopies.Clear(); + _imagesNeedingBarrier.Clear(); + } +} diff --git a/tests/AcDream.App.Tests/Rendering/Gpu/Vk/VulkanFrameFlightTests.cs b/tests/AcDream.App.Tests/Rendering/Gpu/Vk/VulkanFrameFlightTests.cs new file mode 100644 index 00000000..e747b9d8 --- /dev/null +++ b/tests/AcDream.App.Tests/Rendering/Gpu/Vk/VulkanFrameFlightTests.cs @@ -0,0 +1,281 @@ +using System; +using System.Collections.Generic; +using AcDream.App.Rendering.Gpu.Vk; + +namespace AcDream.App.Tests.Rendering.Gpu.Vk; + +/// +/// Campaign V slice V6a — the frame timeline, the per-slot ring, and the +/// staging ring (plan §4.3 and §4.8). +/// +/// These three carry the invariant that everything else in the Vulkan backend +/// rests on: memory handed to the GPU is not reused, and memory freed by the CPU +/// is not destroyed, until the frame that could still be reading it has +/// completed. A timeline semaphore makes that one number, which is exactly what +/// makes it testable — is the number. +/// +public sealed class VulkanFrameFlightTests +{ + private sealed class FakeTimeline : IVulkanTimelineApi + { + public ulong Value { get; set; } + + public List Waits { get; } = []; + + public ulong CurrentValue => Value; + + public void Wait(ulong value) + { + Waits.Add(value); + // A real wait blocks until the GPU gets there; the fake models a + // GPU that is always just far enough ahead. + Value = Math.Max(Value, value); + } + } + + // ── flight controller ──────────────────────────────────────────────────── + + [Fact] + public void Flights_FirstFramesDoNotWaitBecauseNoSlotIsOccupiedYet() + { + var timeline = new FakeTimeline(); + var flights = new VulkanFrameFlightController(timeline, framesInFlight: 2); + + Assert.Equal(1, flights.BeginFrame()); + flights.EndFrame(); + Assert.Equal(2, flights.BeginFrame()); + flights.EndFrame(); + + Assert.Empty(timeline.Waits); + } + + [Fact] + public void Flights_ThirdFrameWaitsForTheFirstToComplete() + { + var timeline = new FakeTimeline(); + var flights = new VulkanFrameFlightController(timeline, framesInFlight: 2); + flights.BeginFrame(); + flights.EndFrame(); + flights.BeginFrame(); + flights.EndFrame(); + + Assert.Equal(3, flights.BeginFrame()); + + Assert.Equal([1ul], timeline.Waits); + } + + [Fact] + public void Flights_SerialsMapOntoSlotsRoundRobin() + { + var flights = new VulkanFrameFlightController(new FakeTimeline(), framesInFlight: 2); + + Assert.Equal(0, flights.SlotIndexOf(1)); + Assert.Equal(1, flights.SlotIndexOf(2)); + Assert.Equal(0, flights.SlotIndexOf(3)); + Assert.Equal(1, flights.SlotIndexOf(4)); + } + + [Fact] + public void Flights_RetirementIsKeyedToTheOpenFrameNotTheCompletedOne() + { + var timeline = new FakeTimeline(); + var flights = new VulkanFrameFlightController(timeline, framesInFlight: 2); + var released = new List(); + + flights.BeginFrame(); // serial 1 open + flights.Retire(() => released.Add("during-1")); + flights.EndFrame(); + + // Frame 1 has been submitted but the GPU has not finished it, so the + // release must not run: commands recorded into frame 1 may still read + // the resource. + timeline.Value = 0; + flights.BeginFrame(); // serial 2 open; retirements run here + Assert.Empty(released); + + flights.EndFrame(); + timeline.Value = 1; + flights.RunRetirements(); + + Assert.Equal(["during-1"], released); + } + + [Fact] + public void Flights_ResourcesReleasedBetweenFramesBelongToTheNextFrame() + { + var timeline = new FakeTimeline(); + var flights = new VulkanFrameFlightController(timeline, framesInFlight: 2); + var released = new List(); + + flights.BeginFrame(); + flights.EndFrame(); + flights.Retire(() => released.Add("between")); + + // Filed against serial 2, which has not even opened. + timeline.Value = 1; + flights.RunRetirements(); + Assert.Empty(released); + + timeline.Value = 2; + flights.RunRetirements(); + Assert.Equal(["between"], released); + } + + [Fact] + public void Flights_WaitForSubmittedWorkDrainsEverythingPending() + { + var timeline = new FakeTimeline(); + var flights = new VulkanFrameFlightController(timeline, framesInFlight: 2); + var released = new List(); + + flights.BeginFrame(); + flights.Retire(() => released.Add("a")); + flights.EndFrame(); + flights.Retire(() => released.Add("b")); + + flights.WaitForSubmittedWork(); + + Assert.Equal(["a", "b"], released); + Assert.Equal(0, flights.PendingRetirementCount); + } + + [Fact] + public void Flights_OpeningTwoFramesAtOnceIsRejected() + { + var flights = new VulkanFrameFlightController(new FakeTimeline(), framesInFlight: 2); + flights.BeginFrame(); + + Assert.Throws(() => flights.BeginFrame()); + } + + [Fact] + public void Flights_DisposeRunsRemainingReleasesSoNothingLeaks() + { + var flights = new VulkanFrameFlightController(new FakeTimeline(), framesInFlight: 2); + var released = new List(); + flights.BeginFrame(); + flights.Retire(() => released.Add("pending")); + + flights.Dispose(); + + Assert.Equal(["pending"], released); + } + + // ── per-frame ring ─────────────────────────────────────────────────────── + + [Fact] + public void Ring_AllocatesForwardWithAlignmentAndTracksThePeak() + { + var ring = new VulkanRingBufferState(1024); + + Assert.Equal(0ul, ring.Allocate(10, 1)); + Assert.Equal(256ul, ring.Allocate(16, 256)); + Assert.Equal(272ul, ring.AllocatedBytes); + Assert.Equal(272ul, ring.PeakAllocatedBytes); + + ring.Reset(); + + Assert.Equal(0ul, ring.AllocatedBytes); + // The peak survives the reset: it is the number that says whether the + // per-slot capacity is right. + Assert.Equal(272ul, ring.PeakAllocatedBytes); + } + + [Fact] + public void Ring_OverCapacityThrowsRatherThanTruncating() + { + var ring = new VulkanRingBufferState(64); + + InvalidOperationException error = Assert.Throws(() => ring.Allocate(65, 1)); + + // The message must name the capacity, because the fix is always + // "raise it" and the person reading has no other source for the number. + Assert.Contains("64 bytes", error.Message, StringComparison.Ordinal); + } + + [Fact] + public void Ring_ZeroSizedAllocationIsLegalAndMovesNothing() + { + var ring = new VulkanRingBufferState(64); + Assert.Equal(0ul, ring.Allocate(0, 16)); + Assert.Equal(0ul, ring.AllocatedBytes); + } + + // ── staging ring ───────────────────────────────────────────────────────── + + [Fact] + public void Staging_HandsOutBytesUntilTheRingIsFullOfUnretiredFrames() + { + var staging = new VulkanStagingRingState(1024); + + Assert.True(staging.TryAllocate(512, 4, serial: 1, out ulong first)); + Assert.True(staging.TryAllocate(512, 4, serial: 1, out ulong second)); + Assert.Equal(0ul, first); + Assert.Equal(512ul, second); + + // Nothing has retired, so there is genuinely nowhere to put this. + Assert.False(staging.TryAllocate(1, 4, serial: 1, out _)); + } + + [Fact] + public void Staging_ReclaimsSpaceWhenTheFrameThatTookItCompletes() + { + var staging = new VulkanStagingRingState(1024); + Assert.True(staging.TryAllocate(1024, 4, serial: 1, out _)); + Assert.False(staging.TryAllocate(16, 4, serial: 2, out _)); + + staging.Release(completedSerial: 1); + + Assert.Equal(0ul, staging.LiveBytes); + Assert.True(staging.TryAllocate(16, 4, serial: 2, out _)); + } + + [Fact] + public void Staging_ReleasesOnlyFramesTheGpuHasActuallyFinished() + { + var staging = new VulkanStagingRingState(1024); + Assert.True(staging.TryAllocate(256, 4, serial: 1, out _)); + Assert.True(staging.TryAllocate(256, 4, serial: 2, out _)); + + staging.Release(completedSerial: 1); + + Assert.Equal(256ul, staging.LiveBytes); + Assert.Equal(1, staging.PendingSegmentCount); + } + + [Fact] + public void Staging_WrapsRatherThanRefusingWhenTheTailIsFree() + { + var staging = new VulkanStagingRingState(1024); + Assert.True(staging.TryAllocate(768, 4, serial: 1, out _)); + staging.Release(completedSerial: 1); + + // 512 does not fit in the 256 bytes left at the end, so it wraps to 0 + // and the wasted tail is charged to this segment. + Assert.True(staging.TryAllocate(512, 4, serial: 2, out ulong wrapped)); + + Assert.Equal(0ul, wrapped); + Assert.Equal(768ul, staging.LiveBytes); + } + + [Fact] + public void Staging_RefusesAPayloadLargerThanTheWholeRing() + { + var staging = new VulkanStagingRingState(1024); + + // The caller's answer to this is a temporary dedicated staging buffer, + // not a bigger ring — see VulkanUploadQueue. + Assert.False(staging.TryAllocate(2048, 4, serial: 1, out _)); + } + + [Fact] + public void Staging_MergesConsecutiveRequestsFromTheSameFrame() + { + var staging = new VulkanStagingRingState(1024); + Assert.True(staging.TryAllocate(16, 4, serial: 1, out _)); + Assert.True(staging.TryAllocate(16, 4, serial: 1, out _)); + Assert.True(staging.TryAllocate(16, 4, serial: 2, out _)); + + Assert.Equal(2, staging.PendingSegmentCount); + } +} diff --git a/tests/AcDream.App.Tests/Rendering/Gpu/Vk/VulkanMemoryModelTests.cs b/tests/AcDream.App.Tests/Rendering/Gpu/Vk/VulkanMemoryModelTests.cs new file mode 100644 index 00000000..f25c6041 --- /dev/null +++ b/tests/AcDream.App.Tests/Rendering/Gpu/Vk/VulkanMemoryModelTests.cs @@ -0,0 +1,248 @@ +using System; +using System.Collections.Generic; +using AcDream.App.Rendering.Gpu; +using AcDream.App.Rendering.Gpu.Vk; +using Silk.NET.Vulkan; + +namespace AcDream.App.Tests.Rendering.Gpu.Vk; + +/// +/// Campaign V slice V6a — the allocator's arithmetic (plan §4.2/§4.3). +/// +/// Every decision the Vulkan allocator makes is reachable from here: where a +/// suballocation lands inside a block, whether releasing one lets the next reuse +/// its bytes, when a request gets a block of its own, and which heap a residency +/// class picks. That is the whole reason the free list and the pool carry no +/// Vulkan handles — an allocator's real failure modes are arithmetic, and +/// arithmetic does not need a GPU to be wrong. +/// +public sealed class VulkanMemoryModelTests +{ + // ── free list ──────────────────────────────────────────────────────────── + + [Fact] + public void FreeList_AllocatesFromTheFrontAndTracksUse() + { + var block = new VulkanMemoryBlockFreeList(1024); + + Assert.True(block.TryAllocate(256, 1, out ulong first)); + Assert.True(block.TryAllocate(256, 1, out ulong second)); + + Assert.Equal(0ul, first); + Assert.Equal(256ul, second); + Assert.Equal(512ul, block.UsedBytes); + Assert.Equal(512ul, block.FreeBytes); + } + + [Fact] + public void FreeList_HonoursAlignmentAndChargesThePaddingToTheAllocation() + { + var block = new VulkanMemoryBlockFreeList(1024); + Assert.True(block.TryAllocate(1, 1, out _)); + + Assert.True(block.TryAllocate(16, 256, out ulong aligned)); + + Assert.Equal(256ul, aligned); + // 1 byte of payload, then 255 bytes of padding plus 16 of payload. + Assert.Equal(272ul, block.UsedBytes); + } + + [Fact] + public void FreeList_ReleasingAnAlignedAllocationReturnsItsPaddingToo() + { + var block = new VulkanMemoryBlockFreeList(1024); + Assert.True(block.TryAllocate(1, 1, out _)); + Assert.True(block.TryAllocate(16, 256, out ulong aligned)); + + block.Free(aligned, 16); + + // Only the leading 1-byte allocation is still out. + Assert.Equal(1ul, block.UsedBytes); + Assert.Equal(1023ul, block.FreeBytes); + } + + [Fact] + public void FreeList_CoalescesNeighboursSoTheBlockDoesNotFragmentAway() + { + var block = new VulkanMemoryBlockFreeList(1024); + Assert.True(block.TryAllocate(256, 1, out ulong a)); + Assert.True(block.TryAllocate(256, 1, out ulong b)); + Assert.True(block.TryAllocate(256, 1, out ulong c)); + + block.Free(a, 256); + block.Free(c, 256); + // Two ranges, not three: releasing c already merged with the block's + // untouched tail. + Assert.Equal(2, block.FreeRangeCount); + + block.Free(b, 256); + + // Everything is back and it is ONE range, not three: a block that + // coalesced badly would still report the right free byte count while + // being unable to satisfy a large allocation ever again. + Assert.Equal(1, block.FreeRangeCount); + Assert.Equal(1024ul, block.LargestFreeBytes); + Assert.Equal(0ul, block.UsedBytes); + } + + [Fact] + public void FreeList_RefusesAnAllocationLargerThanTheLargestHole() + { + var block = new VulkanMemoryBlockFreeList(1024); + Assert.True(block.TryAllocate(600, 1, out _)); + + Assert.False(block.TryAllocate(600, 1, out _)); + Assert.True(block.TryAllocate(400, 1, out _)); + } + + [Fact] + public void FreeList_DoubleReleaseIsRejectedRatherThanCorruptingTheAccounting() + { + var block = new VulkanMemoryBlockFreeList(1024); + Assert.True(block.TryAllocate(256, 1, out ulong offset)); + block.Free(offset, 256); + + Assert.Throws(() => block.Free(offset, 256)); + } + + // ── pool policy ────────────────────────────────────────────────────────── + + [Fact] + public void Pool_AsksForANewBlockBeforeItHasOne() + { + var pool = new VulkanMemoryTypePool(memoryTypeIndex: 0, blockSizeBytes: 4096, dedicatedThresholdBytes: 2048); + + Assert.False(pool.TryAllocate(64, 1, out _)); + + int index = pool.AddBlock(pool.BlockCapacityFor(64), dedicated: false); + Assert.Equal(0, index); + Assert.True(pool.TryAllocate(64, 1, out VulkanMemoryRange range)); + Assert.Equal(0, range.BlockIndex); + Assert.False(range.IsDedicated); + } + + [Fact] + public void Pool_GivesALargeRequestAWholeBlockOfItsOwn() + { + var pool = new VulkanMemoryTypePool(memoryTypeIndex: 0, blockSizeBytes: 4096, dedicatedThresholdBytes: 2048); + int shared = pool.AddBlock(4096, dedicated: false); + Assert.Equal(0, shared); + + // 3000 >= the 2048 threshold, so it must not be placed in the shared + // block even though the shared block would fit it — taking that space + // would strand the remaining kilobyte. + Assert.False(pool.TryAllocate(3000, 1, out _)); + Assert.Equal(3000ul, pool.BlockCapacityFor(3000)); + + int dedicated = pool.AddBlock(3000, dedicated: true); + VulkanMemoryRange range = pool.AllocateWholeBlock(dedicated, 3000); + + Assert.True(range.IsDedicated); + Assert.Equal(0ul, range.OffsetBytes); + Assert.Equal(2, pool.LiveBlockCount); + } + + [Fact] + public void Pool_FreeingADedicatedRangeRetiresItsBlockButKeepsSharedOnes() + { + var pool = new VulkanMemoryTypePool(memoryTypeIndex: 0, blockSizeBytes: 4096, dedicatedThresholdBytes: 2048); + int shared = pool.AddBlock(4096, dedicated: false); + Assert.True(pool.TryAllocate(64, 1, out VulkanMemoryRange small)); + int dedicatedBlock = pool.AddBlock(3000, dedicated: true); + VulkanMemoryRange large = pool.AllocateWholeBlock(dedicatedBlock, 3000); + + Assert.True(pool.Free(large)); // dedicated block is now free device memory + Assert.False(pool.Free(small)); // shared block is kept for reuse + + Assert.Equal(0, shared); + Assert.Equal(1, pool.LiveBlockCount); + } + + [Fact] + public void Pool_ReusesASharedBlockAfterItsAllocationsAreReleased() + { + var pool = new VulkanMemoryTypePool(memoryTypeIndex: 3, blockSizeBytes: 1024, dedicatedThresholdBytes: 4096); + pool.AddBlock(1024, dedicated: false); + Assert.True(pool.TryAllocate(1024, 1, out VulkanMemoryRange whole)); + Assert.False(pool.TryAllocate(16, 1, out _)); + + pool.Free(whole); + + Assert.True(pool.TryAllocate(16, 1, out VulkanMemoryRange reused)); + Assert.Equal(0, reused.BlockIndex); + Assert.Equal(1, pool.LiveBlockCount); + } + + // ── heap selection ─────────────────────────────────────────────────────── + + private static readonly MemoryPropertyFlags DeviceLocal = MemoryPropertyFlags.DeviceLocalBit; + private static readonly MemoryPropertyFlags HostCoherent = + MemoryPropertyFlags.HostVisibleBit | MemoryPropertyFlags.HostCoherentBit; + private static readonly MemoryPropertyFlags ReBar = + MemoryPropertyFlags.DeviceLocalBit | MemoryPropertyFlags.HostVisibleBit | MemoryPropertyFlags.HostCoherentBit; + private static readonly MemoryPropertyFlags HostCached = + MemoryPropertyFlags.HostVisibleBit | MemoryPropertyFlags.HostCoherentBit | MemoryPropertyFlags.HostCachedBit; + + [Fact] + public void Selection_HostWritablePrefersResizableBarOverPlainHostMemory() + { + List types = [DeviceLocal, HostCoherent, ReBar]; + + uint? chosen = VulkanMemoryTypeSelection.Choose(types, 0b111, GpuMemoryResidency.HostWritable); + + // Index 2 is the device-local AND host-visible type: writing per-frame + // data straight into memory the GPU reads is the campaign's CPU win. + Assert.Equal(2u, chosen); + } + + [Fact] + public void Selection_HostWritableFallsBackWhenNoResizableBarTypeExists() + { + List types = [DeviceLocal, HostCoherent]; + + Assert.Equal(1u, VulkanMemoryTypeSelection.Choose(types, 0b11, GpuMemoryResidency.HostWritable)); + } + + [Fact] + public void Selection_RespectsTheResourcesAllowedTypeMask() + { + List types = [ReBar, HostCoherent]; + + // Bit 0 is masked out even though it is the preferred type. + Assert.Equal(1u, VulkanMemoryTypeSelection.Choose(types, 0b10, GpuMemoryResidency.HostWritable)); + } + + [Fact] + public void Selection_DeviceLocalPrefersDeviceLocalButAcceptsAnythingRatherThanFailing() + { + List deviceLocalPresent = [HostCoherent, DeviceLocal]; + Assert.Equal(1u, VulkanMemoryTypeSelection.Choose(deviceLocalPresent, 0b11, GpuMemoryResidency.DeviceLocal)); + + List unifiedMemory = [HostCoherent]; + Assert.Equal(0u, VulkanMemoryTypeSelection.Choose(unifiedMemory, 0b1, GpuMemoryResidency.DeviceLocal)); + } + + [Fact] + public void Selection_HostReadablePrefersCachedMemoryForTheReadbackItExistsFor() + { + List types = [HostCoherent, HostCached]; + + Assert.Equal(1u, VulkanMemoryTypeSelection.Choose(types, 0b11, GpuMemoryResidency.HostReadable)); + } + + [Fact] + public void Selection_AnswersNullWhenNothingIsUsableRatherThanGuessing() + { + List types = [DeviceLocal]; + + Assert.Null(VulkanMemoryTypeSelection.Choose(types, 0b1, GpuMemoryResidency.HostWritable)); + } + + [Fact] + public void Selection_NamesWhichResidenciesMustBeMappable() + { + Assert.False(VulkanMemoryTypeSelection.RequiresMapping(GpuMemoryResidency.DeviceLocal)); + Assert.True(VulkanMemoryTypeSelection.RequiresMapping(GpuMemoryResidency.HostWritable)); + Assert.True(VulkanMemoryTypeSelection.RequiresMapping(GpuMemoryResidency.HostReadable)); + } +}