feat(render): Campaign V slice V6a - Vulkan memory, buffers, rings and the frame timeline
The first of V6's three commits, and the half of the Vulkan backend that has nothing to do with drawing: where memory comes from, how per-frame data reaches the GPU, and what makes it safe to reuse either. Plan sections: 4.2 (bindings layer and the no-VMA decision), 4.3 (memory: arena, staging ring, per-frame data), 4.8 (sync and the frame). The allocator is hand-rolled, roughly as 4.2 sizes it. Silk ships no VMA, and a third-party binding would be a native binary to carry across win-x64, linux-x64 and CI lavapipe for an allocation profile that is genuinely tame: two mesh arena buffers, one staging ring, a per-flight ring buffer each, a few render targets and a texture pool. What a custom allocator buys instead is exact accounting - every byte is attributable to a memory type and a block - which is what GpuMemoryTracker will want and what VMA would obscure. Placement, block policy and heap choice are pure types with no Vulkan handle in sight: VulkanMemoryBlockFreeList is first-fit with coalescing on release, VulkanMemoryTypePool decides when a request is large enough to warrant a block of its own, and VulkanMemoryTypeSelection maps each GpuMemoryResidency onto a preference order of property masks. VulkanDeviceMemoryAllocator turns their answers into vkAllocateMemory and one persistent vkMapMemory per host-visible block. That split is deliberate: an allocator's real failure modes are arithmetic - a mis-coalesced neighbour, an alignment that eats a block's tail, a double release that quietly corrupts the used-byte count - and arithmetic does not need a GPU to be wrong. Twenty-two tests cover exactly those. The HostWritable row of the selection table is the campaign's CPU win stated as data. It prefers a memory type that is both DEVICE_LOCAL and HOST_VISIBLE - resizable BAR, present on the RX 9070 XT - so per-frame data is written once, straight into memory the GPU reads, and falls back to ordinary host-visible coherent memory when no such type exists. GpuCapabilityRecord's SupportsPersistentlyMappedRings is the first capability that is true on this backend and false on GL. Mapping is per block, never per allocation, because Vulkan permits a memory object to be mapped once - mapping per buffer would need one VkDeviceMemory per buffer, which is precisely the allocation-count explosion the design exists to avoid. VulkanRingBufferState is markedly simpler than its GL sibling, and the difference IS the point. GlRingBufferState has to track a dirty watermark and prove its upload never overlaps an in-flight read, because a ring allocation there writes into a managed array that is later copied into a GL buffer. Here the allocation hands back memory the GPU reads directly: there is no upload step to track. What is left is a cursor. VulkanUploadQueue accumulates transfers rather than issuing them, for two reasons that both come from Vulkan rather than from taste: copies must be recorded into a command buffer, and they must be recorded outside a dynamic-rendering block. So requests queue and drain at the one moment both hold - immediately before a pass begins - which is the direct analogue of the GL backend's flush-before-every-draw discipline at the granularity Vulkan needs. The drain emits one batched buffer barrier for the whole batch, one of the four to six 4.8 budgets per frame. Staging exhaustion falls back to a temporary dedicated buffer retired through the ledger. Section 4.3 already specifies that for oversized uploads; extending it to "the ring is full of unretired frames" is the same shape and is a policy rather than a workaround - the transfer stays correct and ordered, it just costs one allocation. VulkanFrameFlightController is the mechanical port 4.8 promised. GL's array of fences becomes one timeline semaphore whose value is the frame serial, "has this slot retired?" becomes "is the counter at least serial minus two?", and the SortedDictionary retirement ledger keeps its keys because those keys were already frame serials. One subtlety is worth stating: a release is filed against the frame currently being RECORDED, not the last one completed, because commands already recorded into the open frame may still read the resource. A test pins that, since getting it wrong frees memory a pending command buffer reads and the symptom would appear somewhere else entirely. Frame acquire ordering is the other subtlety. TryBeginFrame waits on the flight slot BEFORE acquiring its swapchain image, so the slot's acquire semaphore is provably idle - signalling a semaphore a pending submit still waits on is the classic Vulkan deadlock. When the acquire fails the serial is still signalled through an empty submit, because a serial that never completes makes every later frame wait forever. The device is a partial class split along the V6 commit boundary: everything here is memory and frames, while textures and the descriptor table (V6b) and pipelines, passes and readback (V6c) throw with the slice named rather than returning something that fails later and further away. Nothing constructs this device yet - VulkanBringUpHost still presents its clear colour - so the GL path executes not one new statement. VK_EXT_debug_utils naming arrives with the allocator rather than at V6c, because every resource wants a name from birth and the campaign has already spent days on defects only visible from outside the API. It stays optional: absent extension means every call is a no-op and no call site checks. Gates: Release build clean, App suite 4014 passed / 3 skipped (3981 baseline plus 33 new). One Issue181WallPressEquilibriumTests failure in the full run is the known #250 zero-allocation flake and passes on a single run. Offline pixel gate against the parent is a tripwire here - the backend is dark and no GL code path changed - and is reported with the slice. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Erik
parent
78b0e14214
commit
fb9c6693dc
13 changed files with 3040 additions and 0 deletions
392
src/AcDream.App/Rendering/Gpu/Vk/VulkanUploadQueue.cs
Normal file
392
src/AcDream.App/Rendering/Gpu/Vk/VulkanUploadQueue.cs
Normal file
|
|
@ -0,0 +1,392 @@
|
|||
using Silk.NET.Vulkan;
|
||||
using Buffer = Silk.NET.Vulkan.Buffer;
|
||||
|
||||
namespace AcDream.App.Rendering.Gpu.Vk;
|
||||
|
||||
/// <summary>
|
||||
/// Campaign V slice V6a, plan §4.3 and §4.8: every transfer that has been asked
|
||||
/// for but not yet recorded, plus the staging memory those transfers read from.
|
||||
///
|
||||
/// <para><b>Why a queue rather than an immediate copy.</b> Uploads arrive from
|
||||
/// the streaming and texture-cache code at arbitrary moments, including with no
|
||||
/// frame open at all. Vulkan copies must be recorded into a command buffer, and
|
||||
/// they must be recorded OUTSIDE a dynamic-rendering block. So requests
|
||||
/// accumulate here and are drained into the frame's command buffer at the one
|
||||
/// moment both conditions hold: immediately before a pass begins. That is the
|
||||
/// direct analogue of the GL backend's "flush immediately before every draw"
|
||||
/// discipline, moved to the coarser granularity Vulkan actually needs.</para>
|
||||
///
|
||||
/// <para><b>One batched barrier, not one per copy.</b> The drain emits a single
|
||||
/// buffer memory barrier covering every copy it recorded, moving the whole batch
|
||||
/// from transfer writes to the vertex/index/indirect/shader reads that follow.
|
||||
/// Plan §4.8 budgets four to six barriers per frame; this is one of them.</para>
|
||||
///
|
||||
/// <para><b>Staging exhaustion is not an error.</b> When the ring cannot serve a
|
||||
/// request — the payload is larger than the whole ring, or unretired frames hold
|
||||
/// the space — the upload takes a temporary dedicated staging buffer that is
|
||||
/// retired through the ledger. The transfer is equally correct and equally
|
||||
/// ordered; it just costs one allocation. See
|
||||
/// <see cref="VulkanStagingRingState"/> for why that is a policy rather than a
|
||||
/// workaround.</para>
|
||||
/// </summary>
|
||||
internal sealed unsafe class VulkanUploadQueue : IDisposable
|
||||
{
|
||||
/// <summary>Plan §4.3: a 48 MiB persistently mapped staging ring.</summary>
|
||||
internal const ulong DefaultStagingCapacityBytes = 48UL * 1024 * 1024;
|
||||
|
||||
private readonly Silk.NET.Vulkan.Vk _vk;
|
||||
private readonly Device _device;
|
||||
private readonly VulkanDeviceMemoryAllocator _allocator;
|
||||
private readonly VulkanFrameFlightController _flights;
|
||||
private readonly VulkanStagingRingState _ringState;
|
||||
private readonly VulkanDebugNames _debugNames;
|
||||
|
||||
private readonly Buffer _stagingBuffer;
|
||||
private readonly VulkanAllocation _stagingAllocation;
|
||||
|
||||
private readonly List<BufferCopy2> _bufferCopies = [];
|
||||
private readonly List<ImageCopy2> _imageCopies = [];
|
||||
private readonly List<TemporaryStaging> _temporaries = [];
|
||||
|
||||
private bool _disposed;
|
||||
|
||||
private readonly record struct BufferCopy2(Buffer Source, Buffer Destination, ulong SourceOffset, ulong DestinationOffset, ulong SizeBytes);
|
||||
|
||||
private readonly record struct ImageCopy2(
|
||||
Buffer Source,
|
||||
ulong SourceOffset,
|
||||
Image Destination,
|
||||
uint MipLevel,
|
||||
uint Layer,
|
||||
uint Width,
|
||||
uint Height,
|
||||
uint MipLevelCount,
|
||||
uint LayerCount);
|
||||
|
||||
private readonly record struct TemporaryStaging(Buffer Buffer, VulkanAllocation Allocation);
|
||||
|
||||
internal VulkanUploadQueue(
|
||||
Silk.NET.Vulkan.Vk vk,
|
||||
Device device,
|
||||
VulkanDeviceMemoryAllocator allocator,
|
||||
VulkanFrameFlightController flights,
|
||||
VulkanDebugNames debugNames,
|
||||
ulong stagingCapacityBytes = DefaultStagingCapacityBytes)
|
||||
{
|
||||
_vk = vk ?? throw new ArgumentNullException(nameof(vk));
|
||||
_device = device;
|
||||
_allocator = allocator ?? throw new ArgumentNullException(nameof(allocator));
|
||||
_flights = flights ?? throw new ArgumentNullException(nameof(flights));
|
||||
_debugNames = debugNames ?? throw new ArgumentNullException(nameof(debugNames));
|
||||
_ringState = new VulkanStagingRingState(stagingCapacityBytes);
|
||||
|
||||
(_stagingBuffer, _stagingAllocation) = CreateHostBuffer(
|
||||
stagingCapacityBytes,
|
||||
BufferUsageFlags.TransferSrcBit,
|
||||
"vk-staging-ring");
|
||||
}
|
||||
|
||||
/// <summary>Images whose layout must be moved to TRANSFER_DST before the drain and to SHADER_READ after it.</summary>
|
||||
private readonly HashSet<Image> _imagesNeedingBarrier = [];
|
||||
|
||||
internal int PendingBufferCopyCount => _bufferCopies.Count;
|
||||
|
||||
internal int PendingImageCopyCount => _imageCopies.Count;
|
||||
|
||||
internal ulong StagingLiveBytes => _ringState.LiveBytes;
|
||||
|
||||
/// <summary>Stages <paramref name="data"/> and queues a copy into <paramref name="destination"/>.</summary>
|
||||
internal void StageBufferWrite(
|
||||
Buffer destination,
|
||||
ulong destinationOffsetBytes,
|
||||
ReadOnlySpan<byte> data,
|
||||
string ownerName)
|
||||
{
|
||||
ObjectDisposedException.ThrowIf(_disposed, this);
|
||||
if (data.IsEmpty)
|
||||
return;
|
||||
|
||||
(Buffer source, ulong sourceOffset) = Stage(data, ownerName);
|
||||
_bufferCopies.Add(new BufferCopy2(
|
||||
source,
|
||||
destination,
|
||||
sourceOffset,
|
||||
destinationOffsetBytes,
|
||||
(ulong)data.Length));
|
||||
}
|
||||
|
||||
/// <summary>Stages <paramref name="data"/> and queues a copy into one mip level of one array layer.</summary>
|
||||
internal void StageImageWrite(
|
||||
Image destination,
|
||||
int mipLevel,
|
||||
int layer,
|
||||
int width,
|
||||
int height,
|
||||
int mipLevelCount,
|
||||
int layerCount,
|
||||
ReadOnlySpan<byte> data,
|
||||
string ownerName)
|
||||
{
|
||||
ObjectDisposedException.ThrowIf(_disposed, this);
|
||||
if (data.IsEmpty)
|
||||
return;
|
||||
|
||||
// Vulkan requires a buffer-to-image copy's source offset to be a
|
||||
// multiple of 4 and of the texel/block size; 16 covers every format
|
||||
// acdream uses (BC3's 16-byte block is the largest).
|
||||
(Buffer source, ulong sourceOffset) = Stage(data, ownerName, alignmentBytes: 16);
|
||||
_imageCopies.Add(new ImageCopy2(
|
||||
source,
|
||||
sourceOffset,
|
||||
destination,
|
||||
(uint)mipLevel,
|
||||
(uint)layer,
|
||||
(uint)width,
|
||||
(uint)height,
|
||||
(uint)mipLevelCount,
|
||||
(uint)layerCount));
|
||||
_imagesNeedingBarrier.Add(destination);
|
||||
}
|
||||
|
||||
/// <summary>Queues a device-side buffer copy — the mesh arena's grow-and-copy migration.</summary>
|
||||
internal void EnqueueBufferCopy(
|
||||
Buffer source,
|
||||
Buffer destination,
|
||||
ulong sourceOffsetBytes,
|
||||
ulong destinationOffsetBytes,
|
||||
ulong byteCount)
|
||||
{
|
||||
ObjectDisposedException.ThrowIf(_disposed, this);
|
||||
if (byteCount == 0)
|
||||
return;
|
||||
_bufferCopies.Add(new BufferCopy2(
|
||||
source,
|
||||
destination,
|
||||
sourceOffsetBytes,
|
||||
destinationOffsetBytes,
|
||||
byteCount));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Records every pending transfer into <paramref name="commands"/> and
|
||||
/// clears the queue. Must be called outside a dynamic-rendering block.
|
||||
/// Returns false when there was nothing to do.
|
||||
/// </summary>
|
||||
internal bool Record(CommandBuffer commands)
|
||||
{
|
||||
if (_bufferCopies.Count == 0 && _imageCopies.Count == 0)
|
||||
return false;
|
||||
|
||||
if (_imagesNeedingBarrier.Count > 0)
|
||||
TransitionImages(commands, ImageLayout.Undefined, ImageLayout.TransferDstOptimal, toTransfer: true);
|
||||
|
||||
foreach (BufferCopy2 copy in _bufferCopies)
|
||||
{
|
||||
var region = new BufferCopy
|
||||
{
|
||||
SrcOffset = copy.SourceOffset,
|
||||
DstOffset = copy.DestinationOffset,
|
||||
Size = copy.SizeBytes,
|
||||
};
|
||||
_vk.CmdCopyBuffer(commands, copy.Source, copy.Destination, 1, ®ion);
|
||||
}
|
||||
|
||||
foreach (ImageCopy2 copy in _imageCopies)
|
||||
{
|
||||
var region = new BufferImageCopy
|
||||
{
|
||||
BufferOffset = copy.SourceOffset,
|
||||
BufferRowLength = 0,
|
||||
BufferImageHeight = 0,
|
||||
ImageSubresource = new ImageSubresourceLayers
|
||||
{
|
||||
AspectMask = ImageAspectFlags.ColorBit,
|
||||
MipLevel = copy.MipLevel,
|
||||
BaseArrayLayer = copy.Layer,
|
||||
LayerCount = 1,
|
||||
},
|
||||
ImageOffset = new Offset3D(0, 0, 0),
|
||||
ImageExtent = new Extent3D(copy.Width, copy.Height, 1),
|
||||
};
|
||||
_vk.CmdCopyBufferToImage(
|
||||
commands,
|
||||
copy.Source,
|
||||
copy.Destination,
|
||||
ImageLayout.TransferDstOptimal,
|
||||
1,
|
||||
®ion);
|
||||
}
|
||||
|
||||
if (_imagesNeedingBarrier.Count > 0)
|
||||
TransitionImages(commands, ImageLayout.TransferDstOptimal, ImageLayout.ShaderReadOnlyOptimal, toTransfer: false);
|
||||
|
||||
// One buffer barrier for the whole batch: transfer writes become
|
||||
// readable by every consumer stage a copied buffer can feed.
|
||||
if (_bufferCopies.Count > 0)
|
||||
{
|
||||
var barrier = new MemoryBarrier2
|
||||
{
|
||||
SType = StructureType.MemoryBarrier2,
|
||||
SrcStageMask = PipelineStageFlags2.AllTransferBit,
|
||||
SrcAccessMask = AccessFlags2.TransferWriteBit,
|
||||
DstStageMask = PipelineStageFlags2.VertexInputBit
|
||||
| PipelineStageFlags2.VertexShaderBit
|
||||
| PipelineStageFlags2.FragmentShaderBit
|
||||
| PipelineStageFlags2.DrawIndirectBit,
|
||||
DstAccessMask = AccessFlags2.VertexAttributeReadBit
|
||||
| AccessFlags2.IndexReadBit
|
||||
| AccessFlags2.ShaderReadBit
|
||||
| AccessFlags2.UniformReadBit
|
||||
| AccessFlags2.IndirectCommandReadBit,
|
||||
};
|
||||
var dependency = new DependencyInfo
|
||||
{
|
||||
SType = StructureType.DependencyInfo,
|
||||
MemoryBarrierCount = 1,
|
||||
PMemoryBarriers = &barrier,
|
||||
};
|
||||
_vk.CmdPipelineBarrier2(commands, &dependency);
|
||||
}
|
||||
|
||||
_bufferCopies.Clear();
|
||||
_imageCopies.Clear();
|
||||
_imagesNeedingBarrier.Clear();
|
||||
return true;
|
||||
}
|
||||
|
||||
private void TransitionImages(
|
||||
CommandBuffer commands,
|
||||
ImageLayout oldLayout,
|
||||
ImageLayout newLayout,
|
||||
bool toTransfer)
|
||||
{
|
||||
int count = _imagesNeedingBarrier.Count;
|
||||
var barriers = new ImageMemoryBarrier2[count];
|
||||
int index = 0;
|
||||
foreach (Image image in _imagesNeedingBarrier)
|
||||
{
|
||||
barriers[index++] = new ImageMemoryBarrier2
|
||||
{
|
||||
SType = StructureType.ImageMemoryBarrier2,
|
||||
SrcStageMask = toTransfer
|
||||
? PipelineStageFlags2.AllCommandsBit
|
||||
: PipelineStageFlags2.AllTransferBit,
|
||||
SrcAccessMask = toTransfer ? AccessFlags2.None : AccessFlags2.TransferWriteBit,
|
||||
DstStageMask = toTransfer
|
||||
? PipelineStageFlags2.AllTransferBit
|
||||
: PipelineStageFlags2.FragmentShaderBit | PipelineStageFlags2.VertexShaderBit,
|
||||
DstAccessMask = toTransfer ? AccessFlags2.TransferWriteBit : AccessFlags2.ShaderReadBit,
|
||||
// Undefined discards existing contents, which is right for the
|
||||
// first upload into a fresh image and wrong for an incremental
|
||||
// one; incremental array-layer fills therefore name the layout
|
||||
// they are already in.
|
||||
OldLayout = oldLayout,
|
||||
NewLayout = newLayout,
|
||||
SrcQueueFamilyIndex = Silk.NET.Vulkan.Vk.QueueFamilyIgnored,
|
||||
DstQueueFamilyIndex = Silk.NET.Vulkan.Vk.QueueFamilyIgnored,
|
||||
Image = image,
|
||||
SubresourceRange = new ImageSubresourceRange
|
||||
{
|
||||
AspectMask = ImageAspectFlags.ColorBit,
|
||||
BaseMipLevel = 0,
|
||||
LevelCount = Silk.NET.Vulkan.Vk.RemainingMipLevels,
|
||||
BaseArrayLayer = 0,
|
||||
LayerCount = Silk.NET.Vulkan.Vk.RemainingArrayLayers,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
fixed (ImageMemoryBarrier2* first = barriers)
|
||||
{
|
||||
var dependency = new DependencyInfo
|
||||
{
|
||||
SType = StructureType.DependencyInfo,
|
||||
ImageMemoryBarrierCount = (uint)count,
|
||||
PImageMemoryBarriers = first,
|
||||
};
|
||||
_vk.CmdPipelineBarrier2(commands, &dependency);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Reclaims staging bytes and temporary buffers belonging to completed frames.</summary>
|
||||
internal void ReleaseCompleted(long completedSerial) => _ringState.Release(completedSerial);
|
||||
|
||||
private (Buffer Buffer, ulong Offset) Stage(
|
||||
ReadOnlySpan<byte> data,
|
||||
string ownerName,
|
||||
ulong alignmentBytes = 4)
|
||||
{
|
||||
long serial = _flights.OpenSerial != 0 ? _flights.OpenSerial : _flights.SubmittedSerial + 1;
|
||||
if (_ringState.TryAllocate(data.Length, alignmentBytes, serial, out ulong offset))
|
||||
{
|
||||
data.CopyTo(_stagingAllocation.AsSpan().Slice((int)offset, data.Length));
|
||||
return (_stagingBuffer, offset);
|
||||
}
|
||||
|
||||
(Buffer temporary, VulkanAllocation allocation) = CreateHostBuffer(
|
||||
(ulong)data.Length,
|
||||
BufferUsageFlags.TransferSrcBit,
|
||||
$"vk-staging-temp-{ownerName}");
|
||||
data.CopyTo(allocation.AsSpan());
|
||||
_temporaries.Add(new TemporaryStaging(temporary, allocation));
|
||||
|
||||
Buffer captured = temporary;
|
||||
VulkanAllocation capturedAllocation = allocation;
|
||||
_flights.Retire(() =>
|
||||
{
|
||||
_vk.DestroyBuffer(_device, captured, null);
|
||||
_allocator.Free(capturedAllocation);
|
||||
_temporaries.RemoveAll(entry => entry.Buffer.Handle == captured.Handle);
|
||||
});
|
||||
return (temporary, 0);
|
||||
}
|
||||
|
||||
private (Buffer Buffer, VulkanAllocation Allocation) CreateHostBuffer(
|
||||
ulong sizeBytes,
|
||||
BufferUsageFlags usage,
|
||||
string name)
|
||||
{
|
||||
var create = new BufferCreateInfo
|
||||
{
|
||||
SType = StructureType.BufferCreateInfo,
|
||||
Size = sizeBytes,
|
||||
Usage = usage,
|
||||
SharingMode = SharingMode.Exclusive,
|
||||
};
|
||||
VulkanInterop.Check(
|
||||
_vk.CreateBuffer(_device, &create, null, out Buffer buffer),
|
||||
$"vkCreateBuffer ({name})");
|
||||
|
||||
_vk.GetBufferMemoryRequirements(_device, buffer, out MemoryRequirements requirements);
|
||||
VulkanAllocation allocation = _allocator.Allocate(
|
||||
requirements,
|
||||
GpuMemoryResidency.HostWritable,
|
||||
name);
|
||||
VulkanInterop.Check(
|
||||
_vk.BindBufferMemory(_device, buffer, allocation.Memory, allocation.OffsetBytes),
|
||||
$"vkBindBufferMemory ({name})");
|
||||
_debugNames.NameBuffer(buffer, name);
|
||||
return (buffer, allocation);
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
if (_disposed)
|
||||
return;
|
||||
_disposed = true;
|
||||
|
||||
foreach (TemporaryStaging temporary in _temporaries)
|
||||
{
|
||||
_vk.DestroyBuffer(_device, temporary.Buffer, null);
|
||||
_allocator.Free(temporary.Allocation);
|
||||
}
|
||||
_temporaries.Clear();
|
||||
|
||||
_vk.DestroyBuffer(_device, _stagingBuffer, null);
|
||||
_allocator.Free(_stagingAllocation);
|
||||
_ringState.Reset();
|
||||
_bufferCopies.Clear();
|
||||
_imageCopies.Clear();
|
||||
_imagesNeedingBarrier.Clear();
|
||||
}
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue