[PATCH vkd3d 1/2] tests: Set the wrong descriptor heaps in test_descriptor_tables().
Has no effect in Windows at least on AMD. Not calling SetDescriptorHeaps() at all has no effect in Windows on any hardware so this is likely to be universal too. Signed-off-by: Conor McCarthy <cmccarthy(a)codeweavers.com> --- tests/d3d12.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tests/d3d12.c b/tests/d3d12.c index 3b0c4242..df93b71a 100644 --- a/tests/d3d12.c +++ b/tests/d3d12.c @@ -16067,7 +16067,7 @@ static void test_srv_component_mapping(void) static void test_descriptor_tables(void) { - ID3D12DescriptorHeap *heap, *sampler_heap, *heaps[2]; + ID3D12DescriptorHeap *heap, *sampler_heap, *wrong_heap, *wrong_sampler_heap, *heaps[2]; D3D12_ROOT_SIGNATURE_DESC root_signature_desc; D3D12_DESCRIPTOR_RANGE descriptor_range[4]; D3D12_CONSTANT_BUFFER_VIEW_DESC cbv_desc; @@ -16194,6 +16194,8 @@ static void test_descriptor_tables(void) heap = create_gpu_descriptor_heap(context.device, D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV, 6); sampler_heap = create_gpu_descriptor_heap(context.device, D3D12_DESCRIPTOR_HEAP_TYPE_SAMPLER, 1); + wrong_heap = create_gpu_descriptor_heap(context.device, D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV, 6); + wrong_sampler_heap = create_gpu_descriptor_heap(context.device, D3D12_DESCRIPTOR_HEAP_TYPE_SAMPLER, 1); descriptor_size = ID3D12Device_GetDescriptorHandleIncrementSize(context.device, D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV); @@ -16237,7 +16239,7 @@ static void test_descriptor_tables(void) ID3D12GraphicsCommandList_OMSetRenderTargets(command_list, 1, &context.rtv, false, NULL); ID3D12GraphicsCommandList_SetGraphicsRootSignature(command_list, context.root_signature); ID3D12GraphicsCommandList_SetPipelineState(command_list, context.pipeline_state); - heaps[0] = heap; heaps[1] = sampler_heap; + heaps[0] = wrong_heap; heaps[1] = wrong_sampler_heap; ID3D12GraphicsCommandList_SetDescriptorHeaps(command_list, ARRAY_SIZE(heaps), heaps); ID3D12GraphicsCommandList_SetGraphicsRootDescriptorTable(command_list, 0, gpu_handle); ID3D12GraphicsCommandList_SetGraphicsRootDescriptorTable(command_list, 1, @@ -16258,6 +16260,8 @@ static void test_descriptor_tables(void) ID3D12Resource_Release(textures[i]); ID3D12DescriptorHeap_Release(heap); ID3D12DescriptorHeap_Release(sampler_heap); + ID3D12DescriptorHeap_Release(wrong_heap); + ID3D12DescriptorHeap_Release(wrong_sampler_heap); destroy_test_context(&context); } -- 2.34.1
The existing implementation using virtual descriptor heaps, where Vk descriptor sets are created for the bindings in the root descriptor tables, is inefficient when multiple command lists are used with large descriptor heaps. It also cannot support updating a descriptor set after it is bound. This patch creates Vk sets for each D3D12 heap. Because D3D12 heaps can contain CBV, SRV and UAV descriptors in the same heap, multiple Vk sets are needed for each heap, however the total number of populated descriptors is never more than (heap size + UAV counter count). A new 'virtual_heaps' config option is introduced to make the old implementation available when needed. It's not always possible to determine if this is necessary when the device is created. Up to nine Vk descriptor sets may be used. It's theoretically possible to reduce this to eight by placing immutable samplers in the push descriptor set layout, but contradictions in earlier versions of the Vulkan spec made driver support inconsistent. The documentation was corrected in version 1.2.203. This patch also adds support for UAV counter descriptor arrays. It's not practical to add this in a separate patch due to complications with combining the old UAV counter implementation with the new descriptor heap implementation. Wine-Bug: https://bugs.winehq.org/show_bug.cgi?id=47713 Wine-Bug: https://bugs.winehq.org/show_bug.cgi?id=47154 Signed-off-by: Conor McCarthy <cmccarthy(a)codeweavers.com> --- v3: Create all descriptor sets when a heap is created instead of on demand. SetDescriptorHeaps() is no longer used, to match driver behaviour. --- README | 3 + libs/vkd3d/command.c | 175 ++++++++++++++++++++- libs/vkd3d/device.c | 153 +++++++++++++++++++ libs/vkd3d/resource.c | 232 ++++++++++++++++++++++++++++ libs/vkd3d/state.c | 302 +++++++++++++++++++++++++++++++------ libs/vkd3d/vkd3d_private.h | 70 +++++++++ tests/d3d12.c | 13 +- 7 files changed, 896 insertions(+), 52 deletions(-) diff --git a/README b/README index ed53f202..066a24bc 100644 --- a/README +++ b/README @@ -51,6 +51,9 @@ commas or semicolons. even when the output supports colour. * VKD3D_CONFIG - a list of options that change the behavior of libvkd3d. + * virtual_heaps - Create descriptors for each D3D12 root signature + descriptor range instead of entire descriptor heaps. Useful when push + constant or bound descriptor limits are exceeded. * vk_debug - enables Vulkan debug extensions. * VKD3D_DEBUG - controls the debug level for log messages produced by diff --git a/libs/vkd3d/command.c b/libs/vkd3d/command.c index 3e252aee..9f2d671d 100644 --- a/libs/vkd3d/command.c +++ b/libs/vkd3d/command.c @@ -1873,14 +1873,17 @@ static void d3d12_command_list_invalidate_current_render_pass(struct d3d12_comma static void d3d12_command_list_invalidate_bindings(struct d3d12_command_list *list, struct d3d12_pipeline_state *state) { - if (state && state->uav_counters.binding_count) + if (state) { enum vkd3d_pipeline_bind_point bind_point = (enum vkd3d_pipeline_bind_point)state->vk_bind_point; struct vkd3d_pipeline_bindings *bindings = &list->pipeline_bindings[bind_point]; - vkd3d_array_reserve((void **)&bindings->vk_uav_counter_views, &bindings->vk_uav_counter_views_size, - state->uav_counters.binding_count, sizeof(*bindings->vk_uav_counter_views)); - bindings->uav_counters_dirty = true; + if (state->uav_counters.binding_count) + { + vkd3d_array_reserve((void **)&bindings->vk_uav_counter_views, &bindings->vk_uav_counter_views_size, + state->uav_counters.binding_count, sizeof(*bindings->vk_uav_counter_views)); + bindings->uav_counters_dirty = true; + } } } @@ -1895,6 +1898,8 @@ static void d3d12_command_list_invalidate_root_parameters(struct d3d12_command_l bindings->descriptor_set_count = 0; bindings->descriptor_table_dirty_mask = bindings->descriptor_table_active_mask & bindings->root_signature->descriptor_table_mask; bindings->push_descriptor_dirty_mask = bindings->push_descriptor_active_mask & bindings->root_signature->push_descriptor_mask; + bindings->cbv_srv_uav_heap_id = 0; + bindings->sampler_heap_id = 0; } static bool vk_barrier_parameters_from_d3d12_resource_state(unsigned int state, unsigned int stencil_state, @@ -3002,6 +3007,151 @@ static void d3d12_command_list_update_descriptors(struct d3d12_command_list *lis d3d12_command_list_update_uav_counter_descriptors(list, bind_point); } +static unsigned int d3d12_command_list_bind_descriptor_table(struct d3d12_command_list *list, + struct vkd3d_pipeline_bindings *bindings, unsigned int index, + struct d3d12_descriptor_heap **cbv_srv_uav_heap, struct d3d12_descriptor_heap **sampler_heap) +{ + struct d3d12_descriptor_heap *heap; + const struct d3d12_desc *desc; + unsigned int offset; + + if (!(desc = bindings->descriptor_tables[index])) + return 0; + + /* AMD, Nvidia and Intel drivers on Windows work if SetDescriptorHeaps() + * is not called, so we bind heaps from the tables instead. No NULL check is + * needed here because it's checked when descriptor tables are set. */ + heap = vkd3d_gpu_descriptor_allocator_heap_from_descriptor(&list->device->gpu_descriptor_allocator, desc); + offset = desc - (const struct d3d12_desc *)heap->descriptors; + + if (heap->desc.Type == D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV) + { + if (*cbv_srv_uav_heap) + { + if (heap == *cbv_srv_uav_heap) + return offset; + /* This occurs occasionally in Rise of the Tomb Raider apparently + * due to a race condition (one of several). */ + ERR("List %p uses descriptors from more than one CBV/SRV/UAV heap.\n", list); + return ~0u; + } + *cbv_srv_uav_heap = heap; + } + else + { + if (*sampler_heap) + { + if (heap == *sampler_heap) + return offset; + ERR("List %p uses descriptors from more than one sampler heap.\n", list); + return ~0u; + } + *sampler_heap = heap; + } + + return offset; +} + +static bool d3d12_command_list_update_descriptor_tables(struct d3d12_command_list *list, + struct vkd3d_pipeline_bindings *bindings, struct d3d12_descriptor_heap **cbv_srv_uav_heap, + struct d3d12_descriptor_heap **sampler_heap) +{ + const struct vkd3d_vk_device_procs *vk_procs = &list->device->vk_procs; + const struct d3d12_root_signature *rs = bindings->root_signature; + unsigned int offsets[D3D12_MAX_ROOT_COST]; + unsigned int i, j; + + for (i = 0, j = 0; i < ARRAY_SIZE(bindings->descriptor_tables); ++i) + { + if (!(rs->descriptor_table_mask & ((uint64_t)1 << i))) + continue; + if ((offsets[j++] = d3d12_command_list_bind_descriptor_table(list, bindings, i, + cbv_srv_uav_heap, sampler_heap)) == ~0u) + return false; + } + if (j) + { + VK_CALL(vkCmdPushConstants(list->vk_command_buffer, rs->vk_pipeline_layout, VK_SHADER_STAGE_ALL, + rs->descriptor_table_offset, j * sizeof(uint32_t), offsets)); + } + return true; +} + +static void d3d12_command_list_bind_descriptor_heap(struct d3d12_command_list *list, + enum vkd3d_pipeline_bind_point bind_point, struct d3d12_descriptor_heap *heap) +{ + struct vkd3d_pipeline_bindings *bindings = &list->pipeline_bindings[bind_point]; + const struct vkd3d_vk_device_procs *vk_procs = &list->device->vk_procs; + const struct d3d12_root_signature *rs = bindings->root_signature; + enum vkd3d_vk_descriptor_set_index set; + + if (!heap) + return; + + if (heap->desc.Type == D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV) + { + if (heap->serial_id == bindings->cbv_srv_uav_heap_id) + return; + bindings->cbv_srv_uav_heap_id = heap->serial_id; + } + else + { + if (heap->serial_id == bindings->sampler_heap_id) + return; + bindings->sampler_heap_id = heap->serial_id; + } + + for (set = 0; set < ARRAY_SIZE(heap->vk_descriptor_sets); ++set) + { + VkDescriptorSet vk_descriptor_set = heap->vk_descriptor_sets[set].vk_set; + + if (!vk_descriptor_set) + continue; + + /* These sets can be shared across multiple command lists, and therefore binding must + * be synchronised. On an experimental branch in which caching of Vk descriptor writes + * greatly increased the chance of multiple threads arriving here at the same time, + * GRID 2019 crashed without the mutex lock. */ + pthread_mutex_lock(&heap->vk_descriptor_sets[set].mutex); + VK_CALL(vkCmdBindDescriptorSets(list->vk_command_buffer, bindings->vk_bind_point, rs->vk_pipeline_layout, + rs->vk_set_count + set, 1, &vk_descriptor_set, 0, NULL)); + pthread_mutex_unlock(&heap->vk_descriptor_sets[set].mutex); + } +} + +static bool d3d12_command_list_update_heap_descriptors(struct d3d12_command_list *list, + enum vkd3d_pipeline_bind_point bind_point) +{ + struct vkd3d_pipeline_bindings *bindings = &list->pipeline_bindings[bind_point]; + struct d3d12_descriptor_heap *cbv_srv_uav_heap = NULL, *sampler_heap = NULL; + const struct vkd3d_vk_device_procs *vk_procs = &list->device->vk_procs; + const struct d3d12_root_signature *rs = bindings->root_signature; + + if (!rs) + return true; + + if (bindings->descriptor_table_dirty_mask || bindings->push_descriptor_dirty_mask) + d3d12_command_list_prepare_descriptors(list, bind_point); + if (bindings->descriptor_table_dirty_mask && !d3d12_command_list_update_descriptor_tables(list, + bindings, &cbv_srv_uav_heap, &sampler_heap)) + return false; + bindings->descriptor_table_dirty_mask = 0; + + d3d12_command_list_update_push_descriptors(list, bind_point); + + if (bindings->descriptor_set_count) + { + VK_CALL(vkCmdBindDescriptorSets(list->vk_command_buffer, bindings->vk_bind_point, rs->vk_pipeline_layout, + rs->main_set, bindings->descriptor_set_count, bindings->descriptor_sets, 0, NULL)); + bindings->in_use = true; + } + + d3d12_command_list_bind_descriptor_heap(list, bind_point, cbv_srv_uav_heap); + d3d12_command_list_bind_descriptor_heap(list, bind_point, sampler_heap); + + return true; +} + static bool d3d12_command_list_update_compute_state(struct d3d12_command_list *list) { d3d12_command_list_end_current_render_pass(list); @@ -3009,7 +3159,10 @@ static bool d3d12_command_list_update_compute_state(struct d3d12_command_list *l if (!d3d12_command_list_update_compute_pipeline(list)) return false; - d3d12_command_list_update_descriptors(list, VKD3D_PIPELINE_BIND_POINT_COMPUTE); + if (list->device->use_vk_heaps) + return d3d12_command_list_update_heap_descriptors(list, VKD3D_PIPELINE_BIND_POINT_COMPUTE); + else + d3d12_command_list_update_descriptors(list, VKD3D_PIPELINE_BIND_POINT_COMPUTE); return true; } @@ -3026,7 +3179,15 @@ static bool d3d12_command_list_begin_render_pass(struct d3d12_command_list *list if (!d3d12_command_list_update_current_framebuffer(list)) return false; - d3d12_command_list_update_descriptors(list, VKD3D_PIPELINE_BIND_POINT_GRAPHICS); + if (list->device->use_vk_heaps) + { + if (!d3d12_command_list_update_heap_descriptors(list, VKD3D_PIPELINE_BIND_POINT_GRAPHICS)) + return false; + } + else + { + d3d12_command_list_update_descriptors(list, VKD3D_PIPELINE_BIND_POINT_GRAPHICS); + } if (list->current_render_pass != VK_NULL_HANDLE) return true; @@ -4133,6 +4294,8 @@ static void STDMETHODCALLTYPE d3d12_command_list_SetDescriptorHeaps(ID3D12Graphi TRACE("iface %p, heap_count %u, heaps %p.\n", iface, heap_count, heaps); /* Our current implementation does not need this method. + * In Windows it doesn't need to be called at all for correct operation, and + * at least on AMD the wrong heaps can be set here and tests still succeed. * * It could be used to validate descriptor tables but we do not have an * equivalent of the D3D12 Debug Layer. */ diff --git a/libs/vkd3d/device.c b/libs/vkd3d/device.c index 4ce3ed81..3fdc65a4 100644 --- a/libs/vkd3d/device.c +++ b/libs/vkd3d/device.c @@ -141,6 +141,112 @@ static const struct vkd3d_optional_extension_info optional_device_extensions[] = VK_EXTENSION(EXT_VERTEX_ATTRIBUTE_DIVISOR, EXT_vertex_attribute_divisor), }; +static HRESULT vkd3d_create_vk_descriptor_heap_layout(struct d3d12_device *device, unsigned int index) +{ + const struct vkd3d_vk_device_procs *vk_procs = &device->vk_procs; + VkDescriptorSetLayoutBindingFlagsCreateInfoEXT flags_info; + VkDescriptorSetLayoutCreateInfo set_desc; + VkDescriptorBindingFlagsEXT set_flags; + VkDescriptorSetLayoutBinding binding; + VkResult vr; + + binding.binding = 0; + binding.descriptorType = device->vk_descriptor_heap_layouts[index].type; + binding.descriptorCount = device->vk_descriptor_heap_layouts[index].count; + binding.stageFlags = VK_SHADER_STAGE_ALL; + binding.pImmutableSamplers = NULL; + + set_desc.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO; + set_desc.pNext = &flags_info; + set_desc.flags = VK_DESCRIPTOR_SET_LAYOUT_CREATE_UPDATE_AFTER_BIND_POOL_BIT; + set_desc.bindingCount = 1; + set_desc.pBindings = &binding; + + set_flags = VK_DESCRIPTOR_BINDING_VARIABLE_DESCRIPTOR_COUNT_BIT_EXT + | VK_DESCRIPTOR_BINDING_PARTIALLY_BOUND_BIT_EXT | VK_DESCRIPTOR_BINDING_UPDATE_AFTER_BIND_BIT_EXT + | VK_DESCRIPTOR_BINDING_UPDATE_UNUSED_WHILE_PENDING_BIT_EXT; + + flags_info.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_BINDING_FLAGS_CREATE_INFO_EXT; + flags_info.pNext = NULL; + flags_info.bindingCount = 1; + flags_info.pBindingFlags = &set_flags; + + if ((vr = VK_CALL(vkCreateDescriptorSetLayout(device->vk_device, &set_desc, NULL, + &device->vk_descriptor_heap_layouts[index].vk_set_layout))) < 0) + { + WARN("Failed to create Vulkan descriptor set layout, vr %d.\n", vr); + return hresult_from_vk_result(vr); + } + + return S_OK; +} + +static void vkd3d_vk_descriptor_heap_layouts_cleanup(struct d3d12_device *device) +{ + const struct vkd3d_vk_device_procs *vk_procs = &device->vk_procs; + enum vkd3d_vk_descriptor_set_index set; + + for (set = 0; set < ARRAY_SIZE(device->vk_descriptor_heap_layouts); ++set) + VK_CALL(vkDestroyDescriptorSetLayout(device->vk_device, device->vk_descriptor_heap_layouts[set].vk_set_layout, + NULL)); +} + +static HRESULT vkd3d_vk_descriptor_heap_layouts_init(struct d3d12_device *device) +{ + static const struct vkd3d_vk_descriptor_heap_layout vk_descriptor_heap_layouts[VKD3D_SET_INDEX_COUNT] = + { + {VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER, true, D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV}, + {VK_DESCRIPTOR_TYPE_UNIFORM_TEXEL_BUFFER, true, D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV}, + {VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE, false, D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV}, + {VK_DESCRIPTOR_TYPE_STORAGE_TEXEL_BUFFER, true, D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV}, + {VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, false, D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV}, + {VK_DESCRIPTOR_TYPE_SAMPLER, false, D3D12_DESCRIPTOR_HEAP_TYPE_SAMPLER}, + /* UAV counters */ + {VK_DESCRIPTOR_TYPE_STORAGE_TEXEL_BUFFER, true, D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV}, + }; + const struct vkd3d_device_descriptor_limits *limits = &device->vk_info.descriptor_limits; + enum vkd3d_vk_descriptor_set_index set; + HRESULT hr; + + for (set = 0; set < ARRAY_SIZE(device->vk_descriptor_heap_layouts); ++set) + device->vk_descriptor_heap_layouts[set] = vk_descriptor_heap_layouts[set]; + + if (!device->use_vk_heaps) + return S_OK; + + for (set = 0; set < ARRAY_SIZE(device->vk_descriptor_heap_layouts); ++set) + { + switch (device->vk_descriptor_heap_layouts[set].type) + { + case VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER: + device->vk_descriptor_heap_layouts[set].count = limits->uniform_buffer_max_descriptors; + break; + case VK_DESCRIPTOR_TYPE_UNIFORM_TEXEL_BUFFER: + case VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE: + device->vk_descriptor_heap_layouts[set].count = limits->sampled_image_max_descriptors; + break; + case VK_DESCRIPTOR_TYPE_STORAGE_TEXEL_BUFFER: + case VK_DESCRIPTOR_TYPE_STORAGE_IMAGE: + device->vk_descriptor_heap_layouts[set].count = limits->storage_image_max_descriptors; + break; + case VK_DESCRIPTOR_TYPE_SAMPLER: + device->vk_descriptor_heap_layouts[set].count = limits->sampler_max_descriptors; + break; + default: + ERR("Unhandled descriptor type %#x.\n", device->vk_descriptor_heap_layouts[set].type); + break; + } + + if (FAILED(hr = vkd3d_create_vk_descriptor_heap_layout(device, set))) + { + vkd3d_vk_descriptor_heap_layouts_cleanup(device); + return hr; + } + } + + return S_OK; +} + static unsigned int get_spec_version(const VkExtensionProperties *extensions, unsigned int count, const char *extension_name) { @@ -430,6 +536,7 @@ static void vkd3d_init_debug_report(struct vkd3d_instance *instance) static const struct vkd3d_debug_option vkd3d_config_options[] = { + {"virtual_heaps", VKD3D_CONFIG_FLAG_VIRTUAL_HEAPS}, /* always use virtual descriptor heaps */ {"vk_debug", VKD3D_CONFIG_FLAG_VULKAN_DEBUG}, /* enable Vulkan debug extensions */ }; @@ -1267,6 +1374,36 @@ static void vkd3d_init_feature_level(struct vkd3d_vulkan_info *vk_info, TRACE("Max feature level: %#x.\n", vk_info->max_feature_level); } +static void vkd3d_device_descriptor_limits_init(struct vkd3d_device_descriptor_limits *limits, + const VkPhysicalDeviceDescriptorIndexingPropertiesEXT *properties) +{ + const unsigned int root_provision = D3D12_MAX_ROOT_COST / 2; + unsigned int srv_divisor = 1, uav_divisor = 1; + + /* The total number of populated sampled image or storage image descriptors never exceeds the size of + * one set (or two sets if every UAV has a counter), but the total size of bound layouts will exceed + * device limits if each set size is maxDescriptorSet*, because of the D3D12 buffer + image allowance + * (and UAV counters). Breaking limits for layouts seems to work with RADV and Nvidia drivers at + * least, but let's try to stay within them if limits are high enough. */ + if (properties->maxDescriptorSetUpdateAfterBindSampledImages >= (1u << 21)) + { + srv_divisor = 2; + uav_divisor = properties->maxDescriptorSetUpdateAfterBindSampledImages >= (3u << 20) ? 3 : 2; + } + + limits->uniform_buffer_max_descriptors = min(properties->maxDescriptorSetUpdateAfterBindUniformBuffers, + properties->maxPerStageDescriptorUpdateAfterBindUniformBuffers - root_provision); + limits->sampled_image_max_descriptors = min(properties->maxDescriptorSetUpdateAfterBindSampledImages, + properties->maxPerStageDescriptorUpdateAfterBindSampledImages / srv_divisor - root_provision); + limits->storage_buffer_max_descriptors = min(properties->maxDescriptorSetUpdateAfterBindStorageBuffers, + properties->maxPerStageDescriptorUpdateAfterBindStorageBuffers - root_provision); + limits->storage_image_max_descriptors = min(properties->maxDescriptorSetUpdateAfterBindStorageImages, + properties->maxPerStageDescriptorUpdateAfterBindStorageImages / uav_divisor - root_provision); + limits->sampler_max_descriptors = min(properties->maxDescriptorSetUpdateAfterBindSamplers, + properties->maxPerStageDescriptorUpdateAfterBindSamplers - root_provision); + limits->sampler_max_descriptors = min(limits->sampler_max_descriptors, VKD3D_MAX_DESCRIPTOR_SET_SAMPLERS); +} + static HRESULT vkd3d_init_device_caps(struct d3d12_device *device, const struct vkd3d_device_create_info *create_info, struct vkd3d_physical_device_info *physical_device_info, @@ -1299,6 +1436,8 @@ static HRESULT vkd3d_init_device_caps(struct d3d12_device *device, vulkan_info->device_limits = physical_device_info->properties2.properties.limits; vulkan_info->sparse_properties = physical_device_info->properties2.properties.sparseProperties; + vkd3d_device_descriptor_limits_init(&vulkan_info->descriptor_limits, + &physical_device_info->descriptor_indexing_properties); vulkan_info->rasterization_stream = physical_device_info->xfb_properties.transformFeedbackRasterizationStreamSelect; vulkan_info->transform_feedback_queries = physical_device_info->xfb_properties.transformFeedbackQueries; vulkan_info->max_vertex_attrib_divisor = max(physical_device_info->vertex_divisor_properties.maxVertexAttribDivisor, 1); @@ -1494,6 +1633,14 @@ static HRESULT vkd3d_init_device_caps(struct d3d12_device *device, features->robustBufferAccess = VK_FALSE; } + /* Select descriptor heap implementation. Forcing virtual heaps may be useful if + * a client allocates descriptor heaps too large for the Vulkan device, or the + * root signature cost exceeds the available push constant size. Virtual heaps + * use only enough descriptors for the descriptor tables of the currently bound + * root signature, and don't require a 32-bit push constant for each table. */ + device->use_vk_heaps = vulkan_info->EXT_descriptor_indexing + && !(device->vkd3d_instance->config_flags & VKD3D_CONFIG_FLAG_VIRTUAL_HEAPS); + return S_OK; } @@ -2400,6 +2547,7 @@ static ULONG STDMETHODCALLTYPE d3d12_device_Release(ID3D12Device *iface) vkd3d_private_store_destroy(&device->private_store); vkd3d_cleanup_format_info(device); + vkd3d_vk_descriptor_heap_layouts_cleanup(device); vkd3d_uav_clear_state_cleanup(&device->uav_clear_state, device); vkd3d_destroy_null_resources(&device->null_resources, device); vkd3d_gpu_va_allocator_cleanup(&device->gpu_va_allocator); @@ -3905,6 +4053,9 @@ static HRESULT d3d12_device_init(struct d3d12_device *device, if (FAILED(hr = vkd3d_uav_clear_state_init(&device->uav_clear_state, device))) goto out_destroy_null_resources; + if (FAILED(hr = vkd3d_vk_descriptor_heap_layouts_init(device))) + goto out_cleanup_uav_clear_state; + vkd3d_render_pass_cache_init(&device->render_pass_cache); vkd3d_gpu_descriptor_allocator_init(&device->gpu_descriptor_allocator); vkd3d_gpu_va_allocator_init(&device->gpu_va_allocator); @@ -3917,6 +4068,8 @@ static HRESULT d3d12_device_init(struct d3d12_device *device, return S_OK; +out_cleanup_uav_clear_state: + vkd3d_uav_clear_state_cleanup(&device->uav_clear_state, device); out_destroy_null_resources: vkd3d_destroy_null_resources(&device->null_resources, device); out_cleanup_format_info: diff --git a/libs/vkd3d/resource.c b/libs/vkd3d/resource.c index 78b32bde..620bad28 100644 --- a/libs/vkd3d/resource.c +++ b/libs/vkd3d/resource.c @@ -2118,6 +2118,68 @@ void vkd3d_view_decref(struct vkd3d_view *view, struct d3d12_device *device) vkd3d_view_destroy(view, device); } +static void d3d12_descriptor_heap_write_buffer(struct d3d12_descriptor_heap_vk_set *descriptor_set, + const struct d3d12_desc *src) +{ + descriptor_set->vk_descriptor_write.pBufferInfo = &src->u.vk_cbv_info; +} + +static void d3d12_descriptor_heap_write_texel_buffer(struct d3d12_descriptor_heap_vk_set *descriptor_set, + const struct d3d12_desc *src) +{ + descriptor_set->vk_descriptor_write.pTexelBufferView = &src->u.view->u.vk_buffer_view; +} + +static void d3d12_descriptor_heap_write_uav_counter_texel_buffer(struct d3d12_descriptor_heap_vk_set *descriptor_set, + const struct d3d12_desc *src) +{ + descriptor_set->vk_descriptor_write.pTexelBufferView = &src->u.view->vk_counter_view; +} + +static void d3d12_descriptor_heap_write_image(struct d3d12_descriptor_heap_vk_set *descriptor_set, + const struct d3d12_desc *src) +{ + descriptor_set->vk_image_info.imageView = src->u.view->u.vk_image_view; +} + +static void d3d12_descriptor_heap_write_sampler(struct d3d12_descriptor_heap_vk_set *descriptor_set, + const struct d3d12_desc *src) +{ + descriptor_set->vk_image_info.sampler = src->u.view->u.vk_sampler; +} + +/* dst and src contain the same data unless another thread overwrites dst. The array index is + * calculated from dst, and src is thread safe. */ +static void d3d12_descriptor_heap_write_vk_descriptor(struct d3d12_descriptor_heap *descriptor_heap, + enum vkd3d_vk_descriptor_set_index set, const struct d3d12_desc *dst, const struct d3d12_desc *src) +{ + struct d3d12_descriptor_heap_vk_set *descriptor_set = &descriptor_heap->vk_descriptor_sets[set]; + const struct d3d12_device *device = descriptor_heap->device; + const struct vkd3d_vk_device_procs *vk_procs = &device->vk_procs; + + pthread_mutex_lock(&descriptor_set->mutex); + + descriptor_set->vk_descriptor_write.dstArrayElement = dst + - (const struct d3d12_desc *)descriptor_heap->descriptors; + descriptor_set->write_descriptor(descriptor_set, src); + VK_CALL(vkUpdateDescriptorSets(device->vk_device, 1, &descriptor_set->vk_descriptor_write, 0, NULL)); + + pthread_mutex_unlock(&descriptor_set->mutex); +} + +static void d3d12_desc_write_vk_heap(const struct d3d12_desc *dst, const struct d3d12_desc *src, + struct d3d12_device *device) +{ + struct d3d12_descriptor_heap *descriptor_heap; + + descriptor_heap = vkd3d_gpu_descriptor_allocator_heap_from_descriptor(&device->gpu_descriptor_allocator, dst); + + d3d12_descriptor_heap_write_vk_descriptor(descriptor_heap, vkd3d_vk_descriptor_set_index_from_vk_descriptor_type( + src->vk_descriptor_type), dst, src); + if (src->magic == VKD3D_DESCRIPTOR_MAGIC_UAV && src->u.view->vk_counter_view) + d3d12_descriptor_heap_write_vk_descriptor(descriptor_heap, VKD3D_SET_INDEX_UAV_COUNTER, dst, src); +} + void d3d12_desc_write_atomic(struct d3d12_desc *dst, const struct d3d12_desc *src, struct d3d12_device *device) { @@ -2139,6 +2201,9 @@ void d3d12_desc_write_atomic(struct d3d12_desc *dst, const struct d3d12_desc *sr /* Destroy the view after unlocking to reduce wait time. */ if (defunct_view) vkd3d_view_destroy(defunct_view, device); + + if (device->use_vk_heaps && dst->magic) + d3d12_desc_write_vk_heap(dst, src, device); } static void d3d12_desc_destroy(struct d3d12_desc *descriptor, struct d3d12_device *device) @@ -3335,6 +3400,17 @@ void d3d12_dsv_desc_create_dsv(struct d3d12_dsv_desc *dsv_desc, struct d3d12_dev } /* ID3D12DescriptorHeap */ + +static const struct ID3D12DescriptorHeapVtbl d3d12_descriptor_heap_vtbl; + +struct d3d12_descriptor_heap *unsafe_impl_from_ID3D12DescriptorHeap(ID3D12DescriptorHeap *iface) +{ + if (!iface) + return NULL; + assert(iface->lpVtbl == &d3d12_descriptor_heap_vtbl); + return CONTAINING_RECORD(iface, struct d3d12_descriptor_heap, ID3D12DescriptorHeap_iface); +} + static inline struct d3d12_descriptor_heap *impl_from_ID3D12DescriptorHeap(ID3D12DescriptorHeap *iface) { return CONTAINING_RECORD(iface, struct d3d12_descriptor_heap, ID3D12DescriptorHeap_iface); @@ -3381,9 +3457,12 @@ static ULONG STDMETHODCALLTYPE d3d12_descriptor_heap_Release(ID3D12DescriptorHea if (!refcount) { + const struct vkd3d_vk_device_procs *vk_procs; struct d3d12_device *device = heap->device; unsigned int i; + vk_procs = &device->vk_procs; + vkd3d_private_store_destroy(&heap->private_store); switch (heap->desc.Type) @@ -3430,6 +3509,12 @@ static ULONG STDMETHODCALLTYPE d3d12_descriptor_heap_Release(ID3D12DescriptorHea break; } + VK_CALL(vkDestroyDescriptorPool(device->vk_device, heap->vk_descriptor_pool, NULL)); + for (i = 0; i < ARRAY_SIZE(heap->vk_descriptor_sets); ++i) + { + pthread_mutex_destroy(&heap->vk_descriptor_sets[i].mutex); + } + vkd3d_free(heap); d3d12_device_release(device); @@ -3540,19 +3625,166 @@ static const struct ID3D12DescriptorHeapVtbl d3d12_descriptor_heap_vtbl = d3d12_descriptor_heap_GetGPUDescriptorHandleForHeapStart, }; +const enum vkd3d_vk_descriptor_set_index vk_descriptor_set_index_table[] = +{ + VKD3D_SET_INDEX_SAMPLER, + VKD3D_SET_INDEX_COUNT, + VKD3D_SET_INDEX_SAMPLED_IMAGE, + VKD3D_SET_INDEX_STORAGE_IMAGE, + VKD3D_SET_INDEX_UNIFORM_TEXEL_BUFFER, + VKD3D_SET_INDEX_STORAGE_TEXEL_BUFFER, + VKD3D_SET_INDEX_UNIFORM_BUFFER, +}; + +static HRESULT d3d12_descriptor_heap_create_descriptor_pool(struct d3d12_descriptor_heap *descriptor_heap, + struct d3d12_device *device, const D3D12_DESCRIPTOR_HEAP_DESC *desc) +{ + const struct vkd3d_vk_device_procs *vk_procs = &device->vk_procs; + VkDescriptorPoolSize pool_sizes[VKD3D_SET_INDEX_COUNT]; + struct VkDescriptorPoolCreateInfo pool_desc; + VkDevice vk_device = device->vk_device; + enum vkd3d_vk_descriptor_set_index set; + VkResult vr; + + for (set = 0, pool_desc.poolSizeCount = 0; set < ARRAY_SIZE(device->vk_descriptor_heap_layouts); ++set) + { + if (device->vk_descriptor_heap_layouts[set].applicable_heap_type == desc->Type) + { + pool_sizes[pool_desc.poolSizeCount].type = device->vk_descriptor_heap_layouts[set].type; + pool_sizes[pool_desc.poolSizeCount++].descriptorCount = desc->NumDescriptors; + } + } + + pool_desc.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO; + pool_desc.pNext = NULL; + pool_desc.flags = VK_DESCRIPTOR_POOL_CREATE_UPDATE_AFTER_BIND_BIT_EXT; + pool_desc.maxSets = pool_desc.poolSizeCount; + pool_desc.pPoolSizes = pool_sizes; + if ((vr = VK_CALL(vkCreateDescriptorPool(vk_device, &pool_desc, NULL, &descriptor_heap->vk_descriptor_pool))) < 0) + ERR("Failed to create descriptor pool, vr %d.\n", vr); + + return hresult_from_vk_result(vr); +} + +static HRESULT d3d12_descriptor_heap_create_descriptor_set(struct d3d12_descriptor_heap *descriptor_heap, + struct d3d12_device *device, unsigned int set) +{ + struct d3d12_descriptor_heap_vk_set *descriptor_set = &descriptor_heap->vk_descriptor_sets[set]; + uint32_t variable_binding_size = descriptor_heap->desc.NumDescriptors; + const struct vkd3d_vk_device_procs *vk_procs = &device->vk_procs; + VkDescriptorSetVariableDescriptorCountAllocateInfoEXT set_size; + VkDescriptorSetAllocateInfo set_desc; + VkResult vr; + + set_desc.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO; + set_desc.pNext = NULL; + set_desc.descriptorPool = descriptor_heap->vk_descriptor_pool; + set_desc.descriptorSetCount = 1; + set_desc.pSetLayouts = &device->vk_descriptor_heap_layouts[set].vk_set_layout; + set_desc.pNext = &set_size; + set_size.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_VARIABLE_DESCRIPTOR_COUNT_ALLOCATE_INFO_EXT; + set_size.pNext = NULL; + set_size.descriptorSetCount = 1; + set_size.pDescriptorCounts = &variable_binding_size; + if ((vr = VK_CALL(vkAllocateDescriptorSets(device->vk_device, &set_desc, &descriptor_set->vk_set))) >= 0) + { + descriptor_set->vk_descriptor_write.dstSet = descriptor_set->vk_set; + return S_OK; + } + + ERR("Failed to allocate descriptor set, vr %d.\n", vr); + return hresult_from_vk_result(vr); +} + +static HRESULT d3d12_descriptor_heap_vk_descriptor_sets_init(struct d3d12_descriptor_heap *descriptor_heap, + struct d3d12_device *device, const D3D12_DESCRIPTOR_HEAP_DESC *desc) +{ + enum vkd3d_vk_descriptor_set_index set; + HRESULT hr; + + descriptor_heap->vk_descriptor_pool = VK_NULL_HANDLE; + memset(descriptor_heap->vk_descriptor_sets, 0, sizeof(descriptor_heap->vk_descriptor_sets)); + + if (!device->use_vk_heaps || (desc->Type != D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV + && desc->Type != D3D12_DESCRIPTOR_HEAP_TYPE_SAMPLER)) + return S_OK; + + if (FAILED(hr = d3d12_descriptor_heap_create_descriptor_pool(descriptor_heap, device, desc))) + return hr; + + for (set = 0; set < ARRAY_SIZE(descriptor_heap->vk_descriptor_sets); ++set) + { + struct d3d12_descriptor_heap_vk_set *descriptor_set = &descriptor_heap->vk_descriptor_sets[set]; + + pthread_mutex_init(&descriptor_set->mutex, NULL); + + descriptor_set->vk_descriptor_write.sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET; + descriptor_set->vk_descriptor_write.pNext = NULL; + descriptor_set->vk_descriptor_write.dstBinding = 0; + descriptor_set->vk_descriptor_write.descriptorCount = 1; + descriptor_set->vk_descriptor_write.descriptorType = device->vk_descriptor_heap_layouts[set].type; + descriptor_set->vk_descriptor_write.pImageInfo = NULL; + descriptor_set->vk_descriptor_write.pBufferInfo = NULL; + descriptor_set->vk_descriptor_write.pTexelBufferView = NULL; + + switch(device->vk_descriptor_heap_layouts[set].type) + { + case VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER: + descriptor_set->write_descriptor = d3d12_descriptor_heap_write_buffer; + break; + case VK_DESCRIPTOR_TYPE_UNIFORM_TEXEL_BUFFER: + case VK_DESCRIPTOR_TYPE_STORAGE_TEXEL_BUFFER: + descriptor_set->write_descriptor = (set == VKD3D_SET_INDEX_UAV_COUNTER) + ? d3d12_descriptor_heap_write_uav_counter_texel_buffer + : d3d12_descriptor_heap_write_texel_buffer; + break; + case VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE: + descriptor_set->write_descriptor = d3d12_descriptor_heap_write_image; + descriptor_set->vk_descriptor_write.pImageInfo = &descriptor_set->vk_image_info; + descriptor_set->vk_image_info.sampler = VK_NULL_HANDLE; + descriptor_set->vk_image_info.imageLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL; + break; + case VK_DESCRIPTOR_TYPE_STORAGE_IMAGE: + descriptor_set->write_descriptor = d3d12_descriptor_heap_write_image; + descriptor_set->vk_descriptor_write.pImageInfo = &descriptor_set->vk_image_info; + descriptor_set->vk_image_info.sampler = VK_NULL_HANDLE; + descriptor_set->vk_image_info.imageLayout = VK_IMAGE_LAYOUT_GENERAL; + break; + case VK_DESCRIPTOR_TYPE_SAMPLER: + descriptor_set->write_descriptor = d3d12_descriptor_heap_write_sampler; + descriptor_set->vk_descriptor_write.pImageInfo = &descriptor_set->vk_image_info; + descriptor_set->vk_image_info.imageView = VK_NULL_HANDLE; + descriptor_set->vk_image_info.imageLayout = VK_IMAGE_LAYOUT_UNDEFINED; + break; + default: + FIXME("Unhandled descriptor type %#x.\n", device->vk_descriptor_heap_layouts[set].type); + return E_FAIL; + } + if (device->vk_descriptor_heap_layouts[set].applicable_heap_type == desc->Type + && FAILED(hr = d3d12_descriptor_heap_create_descriptor_set(descriptor_heap, device, set))) + return hr; + } + + return S_OK; +} + static HRESULT d3d12_descriptor_heap_init(struct d3d12_descriptor_heap *descriptor_heap, struct d3d12_device *device, const D3D12_DESCRIPTOR_HEAP_DESC *desc) { + static LONG serial_id; HRESULT hr; descriptor_heap->ID3D12DescriptorHeap_iface.lpVtbl = &d3d12_descriptor_heap_vtbl; descriptor_heap->refcount = 1; + descriptor_heap->serial_id = InterlockedIncrement(&serial_id); descriptor_heap->desc = *desc; if (FAILED(hr = vkd3d_private_store_init(&descriptor_heap->private_store))) return hr; + d3d12_descriptor_heap_vk_descriptor_sets_init(descriptor_heap, device, desc); + d3d12_device_add_ref(descriptor_heap->device = device); return S_OK; diff --git a/libs/vkd3d/state.c b/libs/vkd3d/state.c index 10503a60..57505a53 100644 --- a/libs/vkd3d/state.c +++ b/libs/vkd3d/state.c @@ -91,7 +91,9 @@ static void d3d12_root_signature_cleanup(struct d3d12_root_signature *root_signa if (root_signature->descriptor_mapping) vkd3d_free(root_signature->descriptor_mapping); + vkd3d_free(root_signature->uav_counter_mapping); vkd3d_free(root_signature->descriptor_offsets); + vkd3d_free(root_signature->uav_counter_offsets); if (root_signature->root_constants) vkd3d_free(root_signature->root_constants); @@ -327,6 +329,7 @@ static bool vk_binding_from_d3d12_descriptor_range(struct VkDescriptorSetLayoutB struct d3d12_root_signature_info { size_t binding_count; + size_t uav_range_count; size_t root_constant_count; size_t root_descriptor_count; @@ -382,6 +385,7 @@ static HRESULT d3d12_root_signature_info_count_descriptors(struct d3d12_root_sig case D3D12_DESCRIPTOR_RANGE_TYPE_UAV: /* As above. */ info->binding_count += binding_count; + ++info->uav_range_count; break; case D3D12_DESCRIPTOR_RANGE_TYPE_CBV: break; @@ -463,6 +467,7 @@ static HRESULT d3d12_root_signature_init_push_constants(struct d3d12_root_signat uint32_t *push_constant_range_count) { uint32_t push_constants_offset[D3D12_SHADER_VISIBILITY_PIXEL + 1]; + bool use_vk_heaps = root_signature->device->use_vk_heaps; unsigned int i, j, push_constant_count; uint32_t offset; @@ -475,7 +480,8 @@ static HRESULT d3d12_root_signature_init_push_constants(struct d3d12_root_signat continue; assert(p->ShaderVisibility <= D3D12_SHADER_VISIBILITY_PIXEL); - push_constants[p->ShaderVisibility].stageFlags = stage_flags_from_visibility(p->ShaderVisibility); + push_constants[p->ShaderVisibility].stageFlags = use_vk_heaps ? VK_SHADER_STAGE_ALL + : stage_flags_from_visibility(p->ShaderVisibility); push_constants[p->ShaderVisibility].size += p->u.Constants.Num32BitValues * sizeof(uint32_t); } if (push_constants[D3D12_SHADER_VISIBILITY_ALL].size) @@ -554,6 +560,8 @@ struct vkd3d_descriptor_set_context unsigned int table_index; unsigned int unbounded_offset; unsigned int descriptor_index; + unsigned int uav_counter_index; + unsigned int push_constant_index; uint32_t descriptor_binding; }; @@ -563,6 +571,7 @@ static bool vkd3d_validate_descriptor_set_count(struct d3d12_device *device, uns if (set_count > max_count) { + /* NOTE: If maxBoundDescriptorSets is < 9, try VKD3D_CONFIG=virtual_heaps */ ERR("Required descriptor set count exceeds maximum allowed count of %u.\n", max_count); return false; } @@ -740,6 +749,122 @@ static void d3d12_root_signature_map_vk_unbounded_binding(struct d3d12_root_sign offset->dynamic_offset_index = ~0u; } +static unsigned int vk_heap_binding_count_from_descriptor_range(const struct d3d12_root_descriptor_table_range *range, + unsigned int descriptor_set_size) +{ + unsigned int max_count; + + if (descriptor_set_size <= range->offset) + { + ERR("Descriptor range offset %u exceeds maximum available offset %u.\n", range->offset, descriptor_set_size - 1); + max_count = 0; + } + else + { + max_count = descriptor_set_size - range->offset; + } + + if (range->descriptor_count != UINT_MAX) + { + if (range->descriptor_count > max_count) + ERR("Range size %u exceeds available descriptor count %u.\n", range->descriptor_count, max_count); + return range->descriptor_count; + } + else + { + /* Prefer an unsupported binding count vs a zero count, because shader compilation will fail + * to match a declaration to a zero binding, resulting in failure of pipline state creation. */ + return max_count + !max_count; + } +} + +static void vkd3d_descriptor_heap_binding_from_descriptor_range(const struct d3d12_root_descriptor_table_range *range, + bool is_buffer, const struct d3d12_root_signature *root_signature, + struct vkd3d_shader_descriptor_binding *binding) +{ + const struct vkd3d_device_descriptor_limits *descriptor_limits = &root_signature->device->vk_info.descriptor_limits; + unsigned int descriptor_set_size; + + switch (range->type) + { + case VKD3D_SHADER_DESCRIPTOR_TYPE_SRV: + binding->set = is_buffer ? VKD3D_SET_INDEX_UNIFORM_TEXEL_BUFFER : VKD3D_SET_INDEX_SAMPLED_IMAGE; + descriptor_set_size = descriptor_limits->sampled_image_max_descriptors; + break; + case VKD3D_SHADER_DESCRIPTOR_TYPE_UAV: + binding->set = is_buffer ? VKD3D_SET_INDEX_STORAGE_TEXEL_BUFFER : VKD3D_SET_INDEX_STORAGE_IMAGE; + descriptor_set_size = descriptor_limits->storage_image_max_descriptors; + break; + case VKD3D_SHADER_DESCRIPTOR_TYPE_CBV: + binding->set = VKD3D_SET_INDEX_UNIFORM_BUFFER; + descriptor_set_size = descriptor_limits->uniform_buffer_max_descriptors; + break; + case VKD3D_SHADER_DESCRIPTOR_TYPE_SAMPLER: + binding->set = VKD3D_SET_INDEX_SAMPLER; + descriptor_set_size = descriptor_limits->sampler_max_descriptors; + break; + default: + FIXME("Unhandled descriptor range type type %#x.\n", range->type); + binding->set = VKD3D_SET_INDEX_SAMPLED_IMAGE; + descriptor_set_size = descriptor_limits->sampled_image_max_descriptors; + break; + } + binding->set += root_signature->vk_set_count; + binding->binding = 0; + binding->count = vk_heap_binding_count_from_descriptor_range(range, descriptor_set_size); +} + +static void d3d12_root_signature_map_vk_heap_binding(struct d3d12_root_signature *root_signature, + const struct d3d12_root_descriptor_table_range *range, bool buffer_descriptor, + enum vkd3d_shader_visibility shader_visibility, struct vkd3d_descriptor_set_context *context) +{ + struct vkd3d_shader_resource_binding *mapping = &root_signature->descriptor_mapping[context->descriptor_index]; + struct vkd3d_shader_descriptor_offset *offset = &root_signature->descriptor_offsets[context->descriptor_index++]; + + mapping->type = range->type; + mapping->register_space = range->register_space; + mapping->register_index = range->base_register_idx; + mapping->shader_visibility = shader_visibility; + mapping->flags = buffer_descriptor ? VKD3D_SHADER_BINDING_FLAG_BUFFER : VKD3D_SHADER_BINDING_FLAG_IMAGE; + vkd3d_descriptor_heap_binding_from_descriptor_range(range, buffer_descriptor, root_signature, &mapping->binding); + offset->static_offset = range->offset; + offset->dynamic_offset_index = context->push_constant_index; +} + +static void d3d12_root_signature_map_vk_heap_uav_counter(struct d3d12_root_signature *root_signature, + const struct d3d12_root_descriptor_table_range *range, enum vkd3d_shader_visibility shader_visibility, + struct vkd3d_descriptor_set_context *context) +{ + struct vkd3d_shader_uav_counter_binding *mapping = &root_signature->uav_counter_mapping[context->uav_counter_index]; + struct vkd3d_shader_descriptor_offset *offset = &root_signature->uav_counter_offsets[context->uav_counter_index++]; + + mapping->register_space = range->register_space; + mapping->register_index = range->base_register_idx; + mapping->shader_visibility = shader_visibility; + mapping->binding.set = root_signature->vk_set_count + VKD3D_SET_INDEX_UAV_COUNTER; + mapping->binding.binding = 0; + mapping->binding.count = vk_heap_binding_count_from_descriptor_range(range, + root_signature->device->vk_info.descriptor_limits.storage_image_max_descriptors); + offset->static_offset = range->offset; + offset->dynamic_offset_index = context->push_constant_index; +} + +static void d3d12_root_signature_map_descriptor_heap_binding(struct d3d12_root_signature *root_signature, + const struct d3d12_root_descriptor_table_range *range, enum vkd3d_shader_visibility shader_visibility, + struct vkd3d_descriptor_set_context *context) +{ + bool is_buffer = range->type == VKD3D_SHADER_DESCRIPTOR_TYPE_CBV; + + if (range->type == VKD3D_SHADER_DESCRIPTOR_TYPE_SRV || range->type == VKD3D_SHADER_DESCRIPTOR_TYPE_UAV) + { + d3d12_root_signature_map_vk_heap_binding(root_signature, range, true, shader_visibility, context); + if (range->type == VKD3D_SHADER_DESCRIPTOR_TYPE_UAV) + d3d12_root_signature_map_vk_heap_uav_counter(root_signature, range, shader_visibility, context); + } + + d3d12_root_signature_map_vk_heap_binding(root_signature, range, is_buffer, shader_visibility, context); +} + static void d3d12_root_signature_map_descriptor_unbounded_binding(struct d3d12_root_signature *root_signature, const struct d3d12_root_descriptor_table_range *range, unsigned int descriptor_offset, enum vkd3d_shader_visibility shader_visibility, struct vkd3d_descriptor_set_context *context) @@ -804,6 +929,7 @@ static HRESULT validate_descriptor_register_ranges(const struct d3d12_root_descr static HRESULT d3d12_root_signature_init_root_descriptor_tables(struct d3d12_root_signature *root_signature, const D3D12_ROOT_SIGNATURE_DESC *desc, struct vkd3d_descriptor_set_context *context) { + bool use_vk_heaps = root_signature->device->use_vk_heaps; struct d3d12_root_descriptor_table *table; unsigned int i, j, k, range_count; uint32_t vk_binding; @@ -871,6 +997,16 @@ static HRESULT d3d12_root_signature_init_root_descriptor_tables(struct d3d12_roo range = &table->ranges[j]; + if (use_vk_heaps) + { + /* set, binding and vk_binding_count are not used. */ + range->set = 0; + range->binding = 0; + range->vk_binding_count = 0; + d3d12_root_signature_map_descriptor_heap_binding(root_signature, range, shader_visibility, context); + continue; + } + range->set = root_signature->vk_set_count - root_signature->main_set; if (root_signature->use_descriptor_arrays) @@ -949,6 +1085,7 @@ static HRESULT d3d12_root_signature_init_root_descriptor_tables(struct d3d12_roo context->current_binding = cur_binding; } + ++context->push_constant_index; } return S_OK; @@ -1022,6 +1159,30 @@ static HRESULT d3d12_root_signature_init_static_samplers(struct d3d12_root_signa return S_OK; } +static void d3d12_root_signature_init_descriptor_table_push_constants(struct d3d12_root_signature *root_signature, + const struct vkd3d_descriptor_set_context *context) +{ + root_signature->descriptor_table_offset = 0; + if ((root_signature->descriptor_table_count = context->push_constant_index)) + { + VkPushConstantRange *range = &root_signature->push_constant_ranges[D3D12_SHADER_VISIBILITY_ALL]; + + root_signature->descriptor_table_offset = (range->size + 15) & ~15; + range->size = root_signature->descriptor_table_offset + + root_signature->descriptor_table_count * sizeof(uint32_t); + + if (range->size > root_signature->device->vk_info.device_limits.maxPushConstantsSize) + FIXME("Push constants size %u exceeds maximum allowed size %u. Try VKD3D_CONFIG=virtual_heaps.\n", + range->size, root_signature->device->vk_info.device_limits.maxPushConstantsSize); + + if (!root_signature->push_constant_range_count) + { + root_signature->push_constant_range_count = 1; + range->stageFlags = VK_SHADER_STAGE_ALL; + } + } +} + static bool vk_binding_uses_partial_binding(const VkDescriptorSetLayoutBinding *binding) { if (binding->descriptorCount == 1) @@ -1133,7 +1294,9 @@ static HRESULT d3d12_root_signature_init(struct d3d12_root_signature *root_signa const struct vkd3d_vulkan_info *vk_info = &device->vk_info; struct vkd3d_descriptor_set_context context; VkDescriptorSetLayoutBinding *binding_desc; + enum vkd3d_vk_descriptor_set_index set; struct d3d12_root_signature_info info; + bool use_vk_heaps; unsigned int i; HRESULT hr; @@ -1149,7 +1312,9 @@ static HRESULT d3d12_root_signature_init(struct d3d12_root_signature *root_signa root_signature->parameters = NULL; root_signature->flags = desc->Flags; root_signature->descriptor_mapping = NULL; + root_signature->uav_counter_mapping = NULL; root_signature->descriptor_offsets = NULL; + root_signature->uav_counter_offsets = NULL; root_signature->static_sampler_count = 0; root_signature->static_samplers = NULL; root_signature->device = device; @@ -1167,10 +1332,13 @@ static HRESULT d3d12_root_signature_init(struct d3d12_root_signature *root_signa } root_signature->binding_count = info.binding_count; + root_signature->uav_mapping_count = info.uav_range_count; root_signature->static_sampler_count = desc->NumStaticSamplers; root_signature->root_descriptor_count = info.root_descriptor_count; root_signature->use_descriptor_arrays = device->vk_info.EXT_descriptor_indexing; + use_vk_heaps = device->use_vk_heaps; + hr = E_OUTOFMEMORY; root_signature->parameter_count = desc->NumParameters; if (!(root_signature->parameters = vkd3d_calloc(root_signature->parameter_count, @@ -1179,9 +1347,15 @@ static HRESULT d3d12_root_signature_init(struct d3d12_root_signature *root_signa if (!(root_signature->descriptor_mapping = vkd3d_calloc(root_signature->binding_count, sizeof(*root_signature->descriptor_mapping)))) goto fail; + if (use_vk_heaps && !(root_signature->uav_counter_mapping = vkd3d_calloc(root_signature->uav_mapping_count, + sizeof(*root_signature->uav_counter_mapping)))) + goto fail; if (root_signature->use_descriptor_arrays && !(root_signature->descriptor_offsets = vkd3d_calloc( root_signature->binding_count, sizeof(*root_signature->descriptor_offsets)))) goto fail; + if (use_vk_heaps && !(root_signature->uav_counter_offsets = vkd3d_calloc(root_signature->uav_mapping_count, + sizeof(*root_signature->uav_counter_offsets)))) + goto fail; root_signature->root_constant_count = info.root_constant_count; if (!(root_signature->root_constants = vkd3d_calloc(root_signature->root_constant_count, sizeof(*root_signature->root_constants)))) @@ -1213,8 +1387,13 @@ static HRESULT d3d12_root_signature_init(struct d3d12_root_signature *root_signa goto fail; if (FAILED(hr = d3d12_root_signature_init_static_samplers(root_signature, device, desc, &context))) goto fail; + if (use_vk_heaps && FAILED(hr = d3d12_root_signature_append_descriptor_set_layout(root_signature, &context, 0))) + goto fail; + context.push_constant_index = 0; if (FAILED(hr = d3d12_root_signature_init_root_descriptor_tables(root_signature, desc, &context))) goto fail; + if (use_vk_heaps) + d3d12_root_signature_init_descriptor_table_push_constants(root_signature, &context); if (FAILED(hr = d3d12_root_signature_append_descriptor_set_layout(root_signature, &context, 0))) goto fail; @@ -1227,7 +1406,13 @@ static HRESULT d3d12_root_signature_init(struct d3d12_root_signature *root_signa vk_layouts[i] = root_signature->descriptor_set_layouts[i].vk_layout; } - if (FAILED(hr = vkd3d_create_pipeline_layout(device, root_signature->vk_set_count, + if (use_vk_heaps) + { + for (set = 0; set < ARRAY_SIZE(device->vk_descriptor_heap_layouts); ++set) + vk_layouts[i++] = device->vk_descriptor_heap_layouts[set].vk_set_layout; + } + + if (FAILED(hr = vkd3d_create_pipeline_layout(device, i, vk_layouts, root_signature->push_constant_range_count, root_signature->push_constant_ranges, &root_signature->vk_pipeline_layout))) goto fail; @@ -1835,6 +2020,11 @@ static HRESULT d3d12_pipeline_state_init_uav_counters(struct d3d12_pipeline_stat descriptor_binding = 0; for (set_index = 0; set_index < root_signature->vk_set_count; ++set_index) set_layouts[set_index] = root_signature->descriptor_set_layouts[set_index].vk_layout; + if (device->use_vk_heaps) + { + for (i = 0; i < ARRAY_SIZE(device->vk_descriptor_heap_layouts); ++i) + set_layouts[set_index++] = device->vk_descriptor_heap_layouts[i].vk_set_layout; + } for (i = 0, j = 0; i < shader_info->descriptor_count; ++i) { @@ -1913,21 +2103,24 @@ static HRESULT d3d12_pipeline_state_init_compute(struct d3d12_pipeline_state *st return E_INVALIDARG; } - shader_info.type = VKD3D_SHADER_STRUCTURE_TYPE_SCAN_DESCRIPTOR_INFO; - shader_info.next = NULL; - if ((ret = vkd3d_scan_dxbc(&desc->CS, &shader_info)) < 0) + if (!device->use_vk_heaps) { - WARN("Failed to scan shader bytecode, vkd3d result %d.\n", ret); - return hresult_from_vkd3d_result(ret); - } + shader_info.type = VKD3D_SHADER_STRUCTURE_TYPE_SCAN_DESCRIPTOR_INFO; + shader_info.next = NULL; + if ((ret = vkd3d_scan_dxbc(&desc->CS, &shader_info)) < 0) + { + WARN("Failed to scan shader bytecode, vkd3d result %d.\n", ret); + return hresult_from_vkd3d_result(ret); + } - if (FAILED(hr = d3d12_pipeline_state_init_uav_counters(state, - device, root_signature, &shader_info, VK_SHADER_STAGE_COMPUTE_BIT))) - { - WARN("Failed to create descriptor set layout for UAV counters, hr %#x.\n", hr); - return hr; + if (FAILED(hr = d3d12_pipeline_state_init_uav_counters(state, + device, root_signature, &shader_info, VK_SHADER_STAGE_COMPUTE_BIT))) + { + WARN("Failed to create descriptor set layout for UAV counters, hr %#x.\n", hr); + return hr; + } + vkd3d_shader_free_scan_descriptor_info(&shader_info); } - vkd3d_shader_free_scan_descriptor_info(&shader_info); memset(&target_info, 0, sizeof(target_info)); target_info.type = VKD3D_SHADER_STRUCTURE_TYPE_SPIRV_TARGET_INFO; @@ -1939,10 +2132,10 @@ static HRESULT d3d12_pipeline_state_init_compute(struct d3d12_pipeline_state *st { offset_info.type = VKD3D_SHADER_STRUCTURE_TYPE_DESCRIPTOR_OFFSET_INFO; offset_info.next = NULL; - offset_info.descriptor_table_offset = 0; - offset_info.descriptor_table_count = 0; + offset_info.descriptor_table_offset = root_signature->descriptor_table_offset; + offset_info.descriptor_table_count = device->use_vk_heaps ? root_signature->descriptor_table_count : 0; offset_info.binding_offsets = root_signature->descriptor_offsets; - offset_info.uav_counter_offsets = NULL; + offset_info.uav_counter_offsets = root_signature->uav_counter_offsets; vkd3d_prepend_struct(&target_info, &offset_info); } @@ -1954,8 +2147,16 @@ static HRESULT d3d12_pipeline_state_init_compute(struct d3d12_pipeline_state *st shader_interface.push_constant_buffer_count = root_signature->root_constant_count; shader_interface.combined_samplers = NULL; shader_interface.combined_sampler_count = 0; - shader_interface.uav_counters = state->uav_counters.bindings; - shader_interface.uav_counter_count = state->uav_counters.binding_count; + if (root_signature->uav_counter_mapping) + { + shader_interface.uav_counters = root_signature->uav_counter_mapping; + shader_interface.uav_counter_count = root_signature->uav_mapping_count; + } + else + { + shader_interface.uav_counters = state->uav_counters.bindings; + shader_interface.uav_counter_count = state->uav_counters.binding_count; + } vk_pipeline_layout = state->uav_counters.vk_pipeline_layout ? state->uav_counters.vk_pipeline_layout : root_signature->vk_pipeline_layout; @@ -2448,13 +2649,14 @@ static HRESULT d3d12_pipeline_state_init_graphics(struct d3d12_pipeline_state *s VkVertexInputBindingDivisorDescriptionEXT *binding_divisor; const struct vkd3d_vulkan_info *vk_info = &device->vk_info; uint32_t instance_divisors[D3D12_VS_INPUT_REGISTER_COUNT]; + struct vkd3d_shader_spirv_target_info *stage_target_info; uint32_t aligned_offsets[D3D12_VS_INPUT_REGISTER_COUNT]; + struct vkd3d_shader_descriptor_offset_info offset_info; struct vkd3d_shader_parameter ps_shader_parameters[1]; struct vkd3d_shader_transform_feedback_info xfb_info; struct vkd3d_shader_spirv_target_info ps_target_info; struct vkd3d_shader_interface_info shader_interface; - struct vkd3d_shader_descriptor_offset_info offset_info; - struct vkd3d_shader_spirv_target_info *target_info; + struct vkd3d_shader_spirv_target_info target_info; const struct d3d12_root_signature *root_signature; struct vkd3d_shader_signature input_signature; bool have_attachment, is_dsv_format_unknown; @@ -2659,6 +2861,12 @@ static HRESULT d3d12_pipeline_state_init_graphics(struct d3d12_pipeline_state *s } } + memset(&target_info, 0, sizeof(target_info)); + target_info.type = VKD3D_SHADER_STRUCTURE_TYPE_SPIRV_TARGET_INFO; + target_info.environment = VKD3D_SHADER_SPIRV_ENVIRONMENT_VULKAN_1_0; + target_info.extensions = vk_info->shader_extensions; + target_info.extension_count = vk_info->shader_extension_count; + graphics->xfb_enabled = false; if (so_desc->NumEntries) { @@ -2707,10 +2915,10 @@ static HRESULT d3d12_pipeline_state_init_graphics(struct d3d12_pipeline_state *s { offset_info.type = VKD3D_SHADER_STRUCTURE_TYPE_DESCRIPTOR_OFFSET_INFO; offset_info.next = NULL; - offset_info.descriptor_table_offset = 0; - offset_info.descriptor_table_count = 0; + offset_info.descriptor_table_offset = root_signature->descriptor_table_offset; + offset_info.descriptor_table_count = root_signature->descriptor_table_count; offset_info.binding_offsets = root_signature->descriptor_offsets; - offset_info.uav_counter_offsets = NULL; + offset_info.uav_counter_offsets = root_signature->uav_counter_offsets; vkd3d_prepend_struct(&shader_interface, &offset_info); } @@ -2726,24 +2934,27 @@ static HRESULT d3d12_pipeline_state_init_graphics(struct d3d12_pipeline_state *s if (!b->pShaderBytecode) continue; - if ((ret = vkd3d_scan_dxbc(b, &shader_info)) < 0) + if (!device->use_vk_heaps) { - WARN("Failed to scan shader bytecode, stage %#x, vkd3d result %d.\n", - shader_stages[i].stage, ret); - hr = hresult_from_vkd3d_result(ret); - goto fail; - } - if (FAILED(hr = d3d12_pipeline_state_init_uav_counters(state, - device, root_signature, &shader_info, shader_stages[i].stage))) - { - WARN("Failed to create descriptor set layout for UAV counters, hr %#x.\n", hr); - goto fail; + if ((ret = vkd3d_scan_dxbc(b, &shader_info)) < 0) + { + WARN("Failed to scan shader bytecode, stage %#x, vkd3d result %d.\n", + shader_stages[i].stage, ret); + hr = hresult_from_vkd3d_result(ret); + goto fail; + } + if (FAILED(hr = d3d12_pipeline_state_init_uav_counters(state, + device, root_signature, &shader_info, shader_stages[i].stage))) + { + WARN("Failed to create descriptor set layout for UAV counters, hr %#x.\n", hr); + goto fail; + } + vkd3d_shader_free_scan_descriptor_info(&shader_info); } - vkd3d_shader_free_scan_descriptor_info(&shader_info); shader_interface.uav_counters = NULL; shader_interface.uav_counter_count = 0; - target_info = NULL; + stage_target_info = &target_info; switch (shader_stages[i].stage) { case VK_SHADER_STAGE_VERTEX_BIT: @@ -2768,9 +2979,11 @@ static HRESULT d3d12_pipeline_state_init_graphics(struct d3d12_pipeline_state *s break; case VK_SHADER_STAGE_FRAGMENT_BIT: - shader_interface.uav_counters = state->uav_counters.bindings; - shader_interface.uav_counter_count = state->uav_counters.binding_count; - target_info = &ps_target_info; + shader_interface.uav_counters = root_signature->uav_counter_mapping + ? root_signature->uav_counter_mapping : state->uav_counters.bindings; + shader_interface.uav_counter_count = root_signature->uav_counter_mapping + ? root_signature->uav_mapping_count : state->uav_counters.binding_count; + stage_target_info = &ps_target_info; break; default: @@ -2779,10 +2992,15 @@ static HRESULT d3d12_pipeline_state_init_graphics(struct d3d12_pipeline_state *s } shader_interface.next = NULL; + xfb_info.next = NULL; + ps_target_info.next = NULL; + target_info.next = NULL; + offset_info.next = NULL; if (shader_stages[i].stage == xfb_stage) vkd3d_prepend_struct(&shader_interface, &xfb_info); - if (target_info) - vkd3d_prepend_struct(&shader_interface, target_info); + vkd3d_prepend_struct(&shader_interface, stage_target_info); + if (root_signature->descriptor_offsets) + vkd3d_prepend_struct(&shader_interface, &offset_info); if (FAILED(hr = create_shader_stage(device, &graphics->stages[graphics->stage_count], shader_stages[i].stage, b, &shader_interface))) diff --git a/libs/vkd3d/vkd3d_private.h b/libs/vkd3d/vkd3d_private.h index 047f4a29..575f538b 100644 --- a/libs/vkd3d/vkd3d_private.h +++ b/libs/vkd3d/vkd3d_private.h @@ -59,6 +59,9 @@ #define VKD3D_MAX_SHADER_STAGES 5u #define VKD3D_MAX_VK_SYNC_OBJECTS 4u #define VKD3D_MAX_DESCRIPTOR_SETS 64u +#define VKD3D_MAX_BOUND_DESCRIPTOR_HEAPS 2u +/* D3D12 binding tier 3 has a limit of 2048 samplers. */ +#define VKD3D_MAX_DESCRIPTOR_SET_SAMPLERS 2048u struct d3d12_command_list; struct d3d12_device; @@ -92,6 +95,15 @@ HRESULT hresult_from_errno(int rc); HRESULT hresult_from_vk_result(VkResult vr); HRESULT hresult_from_vkd3d_result(int vkd3d_result); +struct vkd3d_device_descriptor_limits +{ + unsigned int uniform_buffer_max_descriptors; + unsigned int sampled_image_max_descriptors; + unsigned int storage_buffer_max_descriptors; + unsigned int storage_image_max_descriptors; + unsigned int sampler_max_descriptors; +}; + struct vkd3d_vulkan_info { /* KHR instance extensions */ @@ -126,6 +138,7 @@ struct vkd3d_vulkan_info VkPhysicalDeviceLimits device_limits; VkPhysicalDeviceSparseProperties sparse_properties; + struct vkd3d_device_descriptor_limits descriptor_limits; VkPhysicalDeviceTexelBufferAlignmentPropertiesEXT texel_buffer_alignment_properties; @@ -138,6 +151,7 @@ struct vkd3d_vulkan_info enum vkd3d_config_flags { VKD3D_CONFIG_FLAG_VULKAN_DEBUG = 0x00000001, + VKD3D_CONFIG_FLAG_VIRTUAL_HEAPS = 0x00000002, }; struct vkd3d_instance @@ -617,11 +631,53 @@ static inline struct d3d12_dsv_desc *d3d12_dsv_desc_from_cpu_handle(D3D12_CPU_DE void d3d12_dsv_desc_create_dsv(struct d3d12_dsv_desc *dsv_desc, struct d3d12_device *device, struct d3d12_resource *resource, const D3D12_DEPTH_STENCIL_VIEW_DESC *desc); +enum vkd3d_vk_descriptor_set_index +{ + VKD3D_SET_INDEX_UNIFORM_BUFFER = 0, + VKD3D_SET_INDEX_UNIFORM_TEXEL_BUFFER = 1, + VKD3D_SET_INDEX_SAMPLED_IMAGE = 2, + VKD3D_SET_INDEX_STORAGE_TEXEL_BUFFER = 3, + VKD3D_SET_INDEX_STORAGE_IMAGE = 4, + VKD3D_SET_INDEX_SAMPLER = 5, + VKD3D_SET_INDEX_UAV_COUNTER = 6, + VKD3D_SET_INDEX_COUNT = 7 +}; + +extern const enum vkd3d_vk_descriptor_set_index vk_descriptor_set_index_table[]; + +static inline enum vkd3d_vk_descriptor_set_index vkd3d_vk_descriptor_set_index_from_vk_descriptor_type( + VkDescriptorType type) +{ + assert(type <= VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER); + assert(vk_descriptor_set_index_table[type] < VKD3D_SET_INDEX_COUNT); + + return vk_descriptor_set_index_table[type]; +} + +struct vkd3d_vk_descriptor_heap_layout +{ + VkDescriptorType type; + bool buffer_dimension; + D3D12_DESCRIPTOR_HEAP_TYPE applicable_heap_type; + unsigned int count; + VkDescriptorSetLayout vk_set_layout; +}; + +struct d3d12_descriptor_heap_vk_set +{ + VkDescriptorSet vk_set; + pthread_mutex_t mutex; + VkWriteDescriptorSet vk_descriptor_write; + VkDescriptorImageInfo vk_image_info; + void (*write_descriptor)(struct d3d12_descriptor_heap_vk_set *descriptor_set, const struct d3d12_desc *src); +}; + /* ID3D12DescriptorHeap */ struct d3d12_descriptor_heap { ID3D12DescriptorHeap ID3D12DescriptorHeap_iface; LONG refcount; + unsigned int serial_id; D3D12_DESCRIPTOR_HEAP_DESC desc; @@ -629,11 +685,15 @@ struct d3d12_descriptor_heap struct vkd3d_private_store private_store; + VkDescriptorPool vk_descriptor_pool; + struct d3d12_descriptor_heap_vk_set vk_descriptor_sets[VKD3D_SET_INDEX_COUNT]; + BYTE descriptors[]; }; HRESULT d3d12_descriptor_heap_create(struct d3d12_device *device, const D3D12_DESCRIPTOR_HEAP_DESC *desc, struct d3d12_descriptor_heap **descriptor_heap); +struct d3d12_descriptor_heap *unsafe_impl_from_ID3D12DescriptorHeap(ID3D12DescriptorHeap *iface); /* ID3D12QueryHeap */ struct d3d12_query_heap @@ -743,8 +803,13 @@ struct d3d12_root_signature D3D12_ROOT_SIGNATURE_FLAGS flags; unsigned int binding_count; + unsigned int uav_mapping_count; struct vkd3d_shader_resource_binding *descriptor_mapping; + struct vkd3d_shader_uav_counter_binding *uav_counter_mapping; struct vkd3d_shader_descriptor_offset *descriptor_offsets; + struct vkd3d_shader_descriptor_offset *uav_counter_offsets; + unsigned int descriptor_table_offset; + unsigned int descriptor_table_count; unsigned int root_constant_count; struct vkd3d_shader_push_constant_buffer *root_constants; @@ -963,6 +1028,8 @@ struct vkd3d_pipeline_bindings struct d3d12_desc *descriptor_tables[D3D12_MAX_ROOT_COST]; uint64_t descriptor_table_dirty_mask; uint64_t descriptor_table_active_mask; + unsigned int cbv_srv_uav_heap_id; + unsigned int sampler_heap_id; VkBufferView *vk_uav_counter_views; size_t vk_uav_counter_views_size; @@ -1210,6 +1277,9 @@ struct d3d12_device const struct vkd3d_format_compatibility_list *format_compatibility_lists; struct vkd3d_null_resources null_resources; struct vkd3d_uav_clear_state uav_clear_state; + + struct vkd3d_vk_descriptor_heap_layout vk_descriptor_heap_layouts[VKD3D_SET_INDEX_COUNT]; + bool use_vk_heaps; }; HRESULT d3d12_device_create(struct vkd3d_instance *instance, diff --git a/tests/d3d12.c b/tests/d3d12.c index df93b71a..57b39913 100644 --- a/tests/d3d12.c +++ b/tests/d3d12.c @@ -16747,13 +16747,15 @@ static void test_update_descriptor_tables(void) destroy_test_context(&context); } -/* This cannot be implemented reasonably in Vulkan. Vulkan doesn't allow - * updating descriptor sets after the vkCmdBindDescriptorSets() command - * is recorded. +/* This requires the Vulkan descriptor indexing extension and Vulkan-backed + * descriptor heaps. Vulkan doesn't allow updating descriptor sets after the + * vkCmdBindDescriptorSets() command is recorded unless the update-after-bind + * feature of descriptor indexing is used. */ static void test_update_descriptor_heap_after_closing_command_list(void) { ID3D12Resource *red_texture, *green_texture; + D3D12_RESOURCE_BINDING_TIER binding_tier; ID3D12GraphicsCommandList *command_list; D3D12_CPU_DESCRIPTOR_HANDLE cpu_handle; ID3D12DescriptorHeap *cpu_heap, *heap; @@ -16803,6 +16805,8 @@ static void test_update_descriptor_heap_after_closing_command_list(void) command_list = context.list; queue = context.queue; + binding_tier = get_resource_binding_tier(context.device); + context.root_signature = create_texture_root_signature(context.device, D3D12_SHADER_VISIBILITY_PIXEL, 0, 0); context.pipeline_state = create_pipeline_state(context.device, @@ -16864,7 +16868,8 @@ static void test_update_descriptor_heap_after_closing_command_list(void) D3D12_RESOURCE_STATE_RENDER_TARGET, D3D12_RESOURCE_STATE_COPY_SOURCE); get_texture_readback_with_command_list(context.render_target, 0, &rb, queue, command_list); value = get_readback_uint(&rb, 0, 0, 0); - todo ok(value == 0xff00ff00, "Got unexpected value %#x.\n", value); + todo_if(binding_tier < D3D12_RESOURCE_BINDING_TIER_3) + ok(value == 0xff00ff00, "Got unexpected value %#x.\n", value); release_resource_readback(&rb); ID3D12DescriptorHeap_Release(cpu_heap); -- 2.34.1
On Thu, 13 Jan 2022 at 06:31, Conor McCarthy <cmccarthy(a)codeweavers.com> wrote:
* VKD3D_CONFIG - a list of options that change the behavior of libvkd3d. + * virtual_heaps - Create descriptors for each D3D12 root signature + descriptor range instead of entire descriptor heaps. Useful when push + constant or bound descriptor limits are exceeded. * vk_debug - enables Vulkan debug extensions.
Being able to explicitly select the older implementation is useful, but it seems unfortunate that it would be required for some applications. What would be required in order to fall back to the current implementation at run-time for applications that need it?
@@ -1873,14 +1873,17 @@ static void d3d12_command_list_invalidate_current_render_pass(struct d3d12_comma static void d3d12_command_list_invalidate_bindings(struct d3d12_command_list *list, struct d3d12_pipeline_state *state) { - if (state && state->uav_counters.binding_count) + if (state) { enum vkd3d_pipeline_bind_point bind_point = (enum vkd3d_pipeline_bind_point)state->vk_bind_point; struct vkd3d_pipeline_bindings *bindings = &list->pipeline_bindings[bind_point];
- vkd3d_array_reserve((void **)&bindings->vk_uav_counter_views, &bindings->vk_uav_counter_views_size, - state->uav_counters.binding_count, sizeof(*bindings->vk_uav_counter_views)); - bindings->uav_counters_dirty = true; + if (state->uav_counters.binding_count) + { + vkd3d_array_reserve((void **)&bindings->vk_uav_counter_views, &bindings->vk_uav_counter_views_size, + state->uav_counters.binding_count, sizeof(*bindings->vk_uav_counter_views)); + bindings->uav_counters_dirty = true; + } } }
This change looks superfluous.
+static unsigned int d3d12_command_list_bind_descriptor_table(struct d3d12_command_list *list, + struct vkd3d_pipeline_bindings *bindings, unsigned int index, + struct d3d12_descriptor_heap **cbv_srv_uav_heap, struct d3d12_descriptor_heap **sampler_heap) +{ + struct d3d12_descriptor_heap *heap; + const struct d3d12_desc *desc; + unsigned int offset; + + if (!(desc = bindings->descriptor_tables[index])) + return 0; + + /* AMD, Nvidia and Intel drivers on Windows work if SetDescriptorHeaps() + * is not called, so we bind heaps from the tables instead. No NULL check is + * needed here because it's checked when descriptor tables are set. */ + heap = vkd3d_gpu_descriptor_allocator_heap_from_descriptor(&list->device->gpu_descriptor_allocator, desc); + offset = desc - (const struct d3d12_desc *)heap->descriptors; + + if (heap->desc.Type == D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV) + { + if (*cbv_srv_uav_heap) + { + if (heap == *cbv_srv_uav_heap) + return offset; + /* This occurs occasionally in Rise of the Tomb Raider apparently + * due to a race condition (one of several). */ + ERR("List %p uses descriptors from more than one CBV/SRV/UAV heap.\n", list); + return ~0u; + }
Things like memory corruption aside, applications should generally not be able to trigger ERRs. Should this be a WARN instead, or are we missing some synchronisation somewhere?
+static void d3d12_command_list_bind_descriptor_heap(struct d3d12_command_list *list, + enum vkd3d_pipeline_bind_point bind_point, struct d3d12_descriptor_heap *heap) +{ + struct vkd3d_pipeline_bindings *bindings = &list->pipeline_bindings[bind_point]; + const struct vkd3d_vk_device_procs *vk_procs = &list->device->vk_procs; + const struct d3d12_root_signature *rs = bindings->root_signature; + enum vkd3d_vk_descriptor_set_index set; + + if (!heap) + return; + + if (heap->desc.Type == D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV) + { + if (heap->serial_id == bindings->cbv_srv_uav_heap_id) + return; + bindings->cbv_srv_uav_heap_id = heap->serial_id; + } + else + { + if (heap->serial_id == bindings->sampler_heap_id) + return; + bindings->sampler_heap_id = heap->serial_id; + } + + for (set = 0; set < ARRAY_SIZE(heap->vk_descriptor_sets); ++set) + { + VkDescriptorSet vk_descriptor_set = heap->vk_descriptor_sets[set].vk_set; + + if (!vk_descriptor_set) + continue; + + /* These sets can be shared across multiple command lists, and therefore binding must + * be synchronised. On an experimental branch in which caching of Vk descriptor writes + * greatly increased the chance of multiple threads arriving here at the same time, + * GRID 2019 crashed without the mutex lock. */ + pthread_mutex_lock(&heap->vk_descriptor_sets[set].mutex); + VK_CALL(vkCmdBindDescriptorSets(list->vk_command_buffer, bindings->vk_bind_point, rs->vk_pipeline_layout, + rs->vk_set_count + set, 1, &vk_descriptor_set, 0, NULL)); + pthread_mutex_unlock(&heap->vk_descriptor_sets[set].mutex); + } +} + Does this need a mutex per descriptor set? It seems like we always lock/bind all the sets that exist for a particular heap; would it make sense to have a single mutex per heap?
+static HRESULT vkd3d_create_vk_descriptor_heap_layout(struct d3d12_device *device, unsigned int index) +{ + const struct vkd3d_vk_device_procs *vk_procs = &device->vk_procs; + VkDescriptorSetLayoutBindingFlagsCreateInfoEXT flags_info; + VkDescriptorSetLayoutCreateInfo set_desc; + VkDescriptorBindingFlagsEXT set_flags; + VkDescriptorSetLayoutBinding binding; + VkResult vr; + + binding.binding = 0; + binding.descriptorType = device->vk_descriptor_heap_layouts[index].type; + binding.descriptorCount = device->vk_descriptor_heap_layouts[index].count; + binding.stageFlags = VK_SHADER_STAGE_ALL; + binding.pImmutableSamplers = NULL; + + set_desc.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO; + set_desc.pNext = &flags_info; + set_desc.flags = VK_DESCRIPTOR_SET_LAYOUT_CREATE_UPDATE_AFTER_BIND_POOL_BIT; + set_desc.bindingCount = 1; + set_desc.pBindings = &binding; + That should use VK_DESCRIPTOR_SET_LAYOUT_CREATE_UPDATE_AFTER_BIND_POOL_BIT_EXT. VK_DESCRIPTOR_SET_LAYOUT_CREATE_UPDATE_AFTER_BIND_POOL_BIT doesn't exist in Vulkan-Headers 1.1.113 (which is the current minimum required version), and technically it's part of Vulkan 1.2 instead of VK_EXT_descriptor_indexing.
+static void vkd3d_device_descriptor_limits_init(struct vkd3d_device_descriptor_limits *limits, + const VkPhysicalDeviceDescriptorIndexingPropertiesEXT *properties) +{ + const unsigned int root_provision = D3D12_MAX_ROOT_COST / 2; + unsigned int srv_divisor = 1, uav_divisor = 1; + + /* The total number of populated sampled image or storage image descriptors never exceeds the size of + * one set (or two sets if every UAV has a counter), but the total size of bound layouts will exceed + * device limits if each set size is maxDescriptorSet*, because of the D3D12 buffer + image allowance + * (and UAV counters). Breaking limits for layouts seems to work with RADV and Nvidia drivers at + * least, but let's try to stay within them if limits are high enough. */ + if (properties->maxDescriptorSetUpdateAfterBindSampledImages >= (1u << 21)) + { + srv_divisor = 2; + uav_divisor = properties->maxDescriptorSetUpdateAfterBindSampledImages >= (3u << 20) ? 3 : 2; + } + + limits->uniform_buffer_max_descriptors = min(properties->maxDescriptorSetUpdateAfterBindUniformBuffers, + properties->maxPerStageDescriptorUpdateAfterBindUniformBuffers - root_provision); + limits->sampled_image_max_descriptors = min(properties->maxDescriptorSetUpdateAfterBindSampledImages, + properties->maxPerStageDescriptorUpdateAfterBindSampledImages / srv_divisor - root_provision); + limits->storage_buffer_max_descriptors = min(properties->maxDescriptorSetUpdateAfterBindStorageBuffers, + properties->maxPerStageDescriptorUpdateAfterBindStorageBuffers - root_provision); + limits->storage_image_max_descriptors = min(properties->maxDescriptorSetUpdateAfterBindStorageImages, + properties->maxPerStageDescriptorUpdateAfterBindStorageImages / uav_divisor - root_provision); + limits->sampler_max_descriptors = min(properties->maxDescriptorSetUpdateAfterBindSamplers, + properties->maxPerStageDescriptorUpdateAfterBindSamplers - root_provision); + limits->sampler_max_descriptors = min(limits->sampler_max_descriptors, VKD3D_MAX_DESCRIPTOR_SET_SAMPLERS); +} + The descriptor limit changes would probably make sense as a separate patch. Note that we need these limits for d3d12_command_allocator_allocate_descriptor_pool() and vk_binding_count_from_descriptor_range() as well, as pointed out by Giovanni back in October.
+const enum vkd3d_vk_descriptor_set_index vk_descriptor_set_index_table[] = +{ + VKD3D_SET_INDEX_SAMPLER, + VKD3D_SET_INDEX_COUNT, + VKD3D_SET_INDEX_SAMPLED_IMAGE, + VKD3D_SET_INDEX_STORAGE_IMAGE, + VKD3D_SET_INDEX_UNIFORM_TEXEL_BUFFER, + VKD3D_SET_INDEX_STORAGE_TEXEL_BUFFER, + VKD3D_SET_INDEX_UNIFORM_BUFFER, +}; + This table is only used by vkd3d_vk_descriptor_set_index_from_vk_descriptor_type(); we may as well make it local to that function. Somewhat similarly, it seems like vkd3d_vk_descriptor_set_index_from_vk_descriptor_type() is only used in resource.c, so we may as well make it local to that file.
This table is indexed by VkDescriptorType. We could make that more obvious by either using designated initialisers, or by adding equivalent comments.
+static HRESULT d3d12_descriptor_heap_create_descriptor_set(struct d3d12_descriptor_heap *descriptor_heap, + struct d3d12_device *device, unsigned int set) +{ + struct d3d12_descriptor_heap_vk_set *descriptor_set = &descriptor_heap->vk_descriptor_sets[set]; + uint32_t variable_binding_size = descriptor_heap->desc.NumDescriptors; + const struct vkd3d_vk_device_procs *vk_procs = &device->vk_procs; + VkDescriptorSetVariableDescriptorCountAllocateInfoEXT set_size; + VkDescriptorSetAllocateInfo set_desc; + VkResult vr; + + set_desc.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO; + set_desc.pNext = NULL; + set_desc.descriptorPool = descriptor_heap->vk_descriptor_pool; + set_desc.descriptorSetCount = 1; + set_desc.pSetLayouts = &device->vk_descriptor_heap_layouts[set].vk_set_layout; + set_desc.pNext = &set_size; + set_size.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_VARIABLE_DESCRIPTOR_COUNT_ALLOCATE_INFO_EXT; + set_size.pNext = NULL; + set_size.descriptorSetCount = 1; + set_size.pDescriptorCounts = &variable_binding_size; + if ((vr = VK_CALL(vkAllocateDescriptorSets(device->vk_device, &set_desc, &descriptor_set->vk_set))) >= 0) + { + descriptor_set->vk_descriptor_write.dstSet = descriptor_set->vk_set; + return S_OK; + } + + ERR("Failed to allocate descriptor set, vr %d.\n", vr); + return hresult_from_vk_result(vr); +} + "set_desc.pNext" is written twice above.
+static HRESULT d3d12_descriptor_heap_vk_descriptor_sets_init(struct d3d12_descriptor_heap *descriptor_heap, + struct d3d12_device *device, const D3D12_DESCRIPTOR_HEAP_DESC *desc) +{ + enum vkd3d_vk_descriptor_set_index set; + HRESULT hr; + + descriptor_heap->vk_descriptor_pool = VK_NULL_HANDLE; + memset(descriptor_heap->vk_descriptor_sets, 0, sizeof(descriptor_heap->vk_descriptor_sets)); + + if (!device->use_vk_heaps || (desc->Type != D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV + && desc->Type != D3D12_DESCRIPTOR_HEAP_TYPE_SAMPLER)) + return S_OK; + + if (FAILED(hr = d3d12_descriptor_heap_create_descriptor_pool(descriptor_heap, device, desc))) + return hr; + + for (set = 0; set < ARRAY_SIZE(descriptor_heap->vk_descriptor_sets); ++set) + { + struct d3d12_descriptor_heap_vk_set *descriptor_set = &descriptor_heap->vk_descriptor_sets[set]; + + pthread_mutex_init(&descriptor_set->mutex, NULL); + + descriptor_set->vk_descriptor_write.sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET; + descriptor_set->vk_descriptor_write.pNext = NULL; + descriptor_set->vk_descriptor_write.dstBinding = 0; + descriptor_set->vk_descriptor_write.descriptorCount = 1; + descriptor_set->vk_descriptor_write.descriptorType = device->vk_descriptor_heap_layouts[set].type; + descriptor_set->vk_descriptor_write.pImageInfo = NULL; + descriptor_set->vk_descriptor_write.pBufferInfo = NULL; + descriptor_set->vk_descriptor_write.pTexelBufferView = NULL; + + switch(device->vk_descriptor_heap_layouts[set].type) + { + case VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER: + descriptor_set->write_descriptor = d3d12_descriptor_heap_write_buffer; + break; + case VK_DESCRIPTOR_TYPE_UNIFORM_TEXEL_BUFFER: + case VK_DESCRIPTOR_TYPE_STORAGE_TEXEL_BUFFER: + descriptor_set->write_descriptor = (set == VKD3D_SET_INDEX_UAV_COUNTER) + ? d3d12_descriptor_heap_write_uav_counter_texel_buffer + : d3d12_descriptor_heap_write_texel_buffer; + break; + case VK_DESCRIPTOR_TYPE_SAMPLED_IMAGE: + descriptor_set->write_descriptor = d3d12_descriptor_heap_write_image; + descriptor_set->vk_descriptor_write.pImageInfo = &descriptor_set->vk_image_info; + descriptor_set->vk_image_info.sampler = VK_NULL_HANDLE; + descriptor_set->vk_image_info.imageLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL; + break; + case VK_DESCRIPTOR_TYPE_STORAGE_IMAGE: + descriptor_set->write_descriptor = d3d12_descriptor_heap_write_image; + descriptor_set->vk_descriptor_write.pImageInfo = &descriptor_set->vk_image_info; + descriptor_set->vk_image_info.sampler = VK_NULL_HANDLE; + descriptor_set->vk_image_info.imageLayout = VK_IMAGE_LAYOUT_GENERAL; + break; + case VK_DESCRIPTOR_TYPE_SAMPLER: + descriptor_set->write_descriptor = d3d12_descriptor_heap_write_sampler; + descriptor_set->vk_descriptor_write.pImageInfo = &descriptor_set->vk_image_info; + descriptor_set->vk_image_info.imageView = VK_NULL_HANDLE; + descriptor_set->vk_image_info.imageLayout = VK_IMAGE_LAYOUT_UNDEFINED; + break; + default: + FIXME("Unhandled descriptor type %#x.\n", device->vk_descriptor_heap_layouts[set].type); + return E_FAIL; + } + if (device->vk_descriptor_heap_layouts[set].applicable_heap_type == desc->Type + && FAILED(hr = d3d12_descriptor_heap_create_descriptor_set(descriptor_heap, device, set))) + return hr; + } + + return S_OK; +} + For small/trivial functions like e.g. d3d12_descriptor_heap_write_image(), calling them through a function pointer tends to not be much faster than just using a switch. I don't think it helps much in terms of clarity either. In terms of efficiency, it would probably help to write an entire range of descriptors in a single call, instead of writing single descriptors at a time.
static HRESULT d3d12_descriptor_heap_init(struct d3d12_descriptor_heap *descriptor_heap, struct d3d12_device *device, const D3D12_DESCRIPTOR_HEAP_DESC *desc) { + static LONG serial_id; HRESULT hr;
descriptor_heap->ID3D12DescriptorHeap_iface.lpVtbl = &d3d12_descriptor_heap_vtbl; descriptor_heap->refcount = 1; + descriptor_heap->serial_id = InterlockedIncrement(&serial_id);
descriptor_heap->desc = *desc;
if (FAILED(hr = vkd3d_private_store_init(&descriptor_heap->private_store))) return hr;
+ d3d12_descriptor_heap_vk_descriptor_sets_init(descriptor_heap, device, desc); + d3d12_device_add_ref(descriptor_heap->device = device);
return S_OK;
"serial_id" above can wrap, and for a 32-bit value that may not even take all that long.
+static void d3d12_root_signature_init_descriptor_table_push_constants(struct d3d12_root_signature *root_signature, + const struct vkd3d_descriptor_set_context *context) +{ + root_signature->descriptor_table_offset = 0; + if ((root_signature->descriptor_table_count = context->push_constant_index)) + { + VkPushConstantRange *range = &root_signature->push_constant_ranges[D3D12_SHADER_VISIBILITY_ALL]; + + root_signature->descriptor_table_offset = (range->size + 15) & ~15; + range->size = root_signature->descriptor_table_offset + + root_signature->descriptor_table_count * sizeof(uint32_t); + Note that we have an align() helper.
@@ -2448,13 +2649,14 @@ static HRESULT d3d12_pipeline_state_init_graphics(struct d3d12_pipeline_state *s VkVertexInputBindingDivisorDescriptionEXT *binding_divisor; const struct vkd3d_vulkan_info *vk_info = &device->vk_info; uint32_t instance_divisors[D3D12_VS_INPUT_REGISTER_COUNT]; + struct vkd3d_shader_spirv_target_info *stage_target_info; uint32_t aligned_offsets[D3D12_VS_INPUT_REGISTER_COUNT]; + struct vkd3d_shader_descriptor_offset_info offset_info; struct vkd3d_shader_parameter ps_shader_parameters[1]; struct vkd3d_shader_transform_feedback_info xfb_info; struct vkd3d_shader_spirv_target_info ps_target_info; struct vkd3d_shader_interface_info shader_interface; - struct vkd3d_shader_descriptor_offset_info offset_info; - struct vkd3d_shader_spirv_target_info *target_info; + struct vkd3d_shader_spirv_target_info target_info; const struct d3d12_root_signature *root_signature; struct vkd3d_shader_signature input_signature; bool have_attachment, is_dsv_format_unknown; @@ -2659,6 +2861,12 @@ static HRESULT d3d12_pipeline_state_init_graphics(struct d3d12_pipeline_state *s } }
+ memset(&target_info, 0, sizeof(target_info)); + target_info.type = VKD3D_SHADER_STRUCTURE_TYPE_SPIRV_TARGET_INFO; + target_info.environment = VKD3D_SHADER_SPIRV_ENVIRONMENT_VULKAN_1_0; + target_info.extensions = vk_info->shader_extensions; + target_info.extension_count = vk_info->shader_extension_count; + The "target_info" related changes here probably make sense as a separate patch.
diff --git a/libs/vkd3d/vkd3d_private.h b/libs/vkd3d/vkd3d_private.h index 047f4a29..575f538b 100644 --- a/libs/vkd3d/vkd3d_private.h +++ b/libs/vkd3d/vkd3d_private.h @@ -59,6 +59,9 @@ #define VKD3D_MAX_SHADER_STAGES 5u #define VKD3D_MAX_VK_SYNC_OBJECTS 4u #define VKD3D_MAX_DESCRIPTOR_SETS 64u +#define VKD3D_MAX_BOUND_DESCRIPTOR_HEAPS 2u +/* D3D12 binding tier 3 has a limit of 2048 samplers. */ +#define VKD3D_MAX_DESCRIPTOR_SET_SAMPLERS 2048u
VKD3D_MAX_BOUND_DESCRIPTOR_HEAPS above is unused.
@@ -1210,6 +1277,9 @@ struct d3d12_device const struct vkd3d_format_compatibility_list *format_compatibility_lists; struct vkd3d_null_resources null_resources; struct vkd3d_uav_clear_state uav_clear_state; + + struct vkd3d_vk_descriptor_heap_layout vk_descriptor_heap_layouts[VKD3D_SET_INDEX_COUNT]; + bool use_vk_heaps; };
I notice there are a fair amount of "use_vk_heaps" checks sprinkled through the code. While not the worst thing in the world, it's not quite ideal either. Anything we could do about that?
February 1, 2022 3:08 AM, "Henri Verbeet" <hverbeet(a)gmail.com> wrote:
* VKD3D_CONFIG - a list of options that change the behavior of libvkd3d. + * virtual_heaps - Create descriptors for each D3D12 root signature + descriptor range instead of entire descriptor heaps. Useful when push + constant or bound descriptor limits are exceeded. * vk_debug - enables Vulkan debug extensions.
Being able to explicitly select the older implementation is useful, but it seems unfortunate that it would be required for some applications. What would be required in order to fall back to the current implementation at run-time for applications that need it?
It's complicated. By the time the problem appears, some root signatures and pipeline state objects could have already been created. To fall back to the current implementation they would need to be reconfigured for it. It's worse if the problem only arises when Vulkan descriptor sets are bound. I'll see if the maximum number of descriptor sets used can be reduced to 8, which would help a bit.
+ /* This occurs occasionally in Rise of the Tomb Raider apparently + * due to a race condition (one of several). */ + ERR("List %p uses descriptors from more than one CBV/SRV/UAV heap.\n", list); + return ~0u; + }
Things like memory corruption aside, applications should generally not be able to trigger ERRs. Should this be a WARN instead, or are we missing some synchronisation somewhere?
Back when I tried using a mutex to prevent descriptor tables being updated while descriptor sets were bound, it had no effect. The problem is intermittent and isn't occurring at all at the moment, so I've been unable to check it again.
Does this need a mutex per descriptor set? It seems like we always lock/bind all the sets that exist for a particular heap; would it make sense to have a single mutex per heap?
The only potential problem is slowing down multithreaded copying of descriptors where different threads copy different types, but I think that situation is unlikely to arise.
The descriptor limit changes would probably make sense as a separate patch. Note that we need these limits for d3d12_command_allocator_allocate_descriptor_pool() and vk_binding_count_from_descriptor_range() as well, as pointed out by Giovanni back in October.
So use an arbitrary fixed size for the descriptor set layout in the initial patch?
For small/trivial functions like e.g. d3d12_descriptor_heap_write_image(), calling them through a function pointer tends to not be much faster than just using a switch. I don't think it helps much in terms of clarity either. In terms of efficiency, it would probably help to write an entire range of descriptors in a single call, instead of writing single descriptors at a time.
Optimisation of descriptor copying is in a separate patch. It still uses function pointers but I'll try it without those.
static HRESULT d3d12_descriptor_heap_init(struct d3d12_descriptor_heap *descriptor_heap, struct d3d12_device *device, const D3D12_DESCRIPTOR_HEAP_DESC *desc) { + static LONG serial_id; HRESULT hr;
descriptor_heap->ID3D12DescriptorHeap_iface.lpVtbl = &d3d12_descriptor_heap_vtbl; descriptor_heap->refcount = 1; + descriptor_heap->serial_id = InterlockedIncrement(&serial_id);
descriptor_heap->desc = *desc;
if (FAILED(hr = vkd3d_private_store_init(&descriptor_heap->private_store))) return hr;
+ d3d12_descriptor_heap_vk_descriptor_sets_init(descriptor_heap, device, desc); + d3d12_device_add_ref(descriptor_heap->device = device);
return S_OK;
"serial_id" above can wrap, and for a 32-bit value that may not even take all that long.
Even if 10 descriptor heaps are created/destroyed per frame it would take 1000 hours of rendering at 120 fps to wrap. Since only two heaps can be bound per command list I figured it's unlikely to ever cause a problem.
I notice there are a fair amount of "use_vk_heaps" checks sprinkled through the code. While not the worst thing in the world, it's not quite ideal either. Anything we could do about that?
A few function pointers would be neater.
On Tue, 1 Feb 2022 at 15:20, Conor McCarthy <cmccarthy(a)codeweavers.com> wrote:
February 1, 2022 3:08 AM, "Henri Verbeet" <hverbeet(a)gmail.com> wrote:
The descriptor limit changes would probably make sense as a separate patch. Note that we need these limits for d3d12_command_allocator_allocate_descriptor_pool() and vk_binding_count_from_descriptor_range() as well, as pointed out by Giovanni back in October.
So use an arbitrary fixed size for the descriptor set layout in the initial patch?
Well, no. But introducing the descriptor limits first and using them in the places mentioned above would (slightly) reduce the size of this patch. I.e., the general idea is to reduce the size of more complicated patches like this one by taking care of more peripheral issues beforehand, so that review can focus on the core changes as much as possible.
"serial_id" above can wrap, and for a 32-bit value that may not even take all that long.
Even if 10 descriptor heaps are created/destroyed per frame it would take 1000 hours of rendering at 120 fps to wrap. Since only two heaps can be bound per command list I figured it's unlikely to ever cause a problem.
Right, it's certainly not likely to wrap for typical "Desktop gaming" use-cases, but I could also think up use-cases where you'd want a process to run for longer than 1000 hours. (And hard to reproduce issues are simply just that much more frustrating to debug...)
participants (3)
-
Conor McCarthy -
Henri Verbeet -
Henri Verbeet