[PATCH v10 0/5] MR44: vkd3d-shader/dxil: Read and validate DXIL bitcode blocks.
Block records are not processed; only the bitcode is validated. -- v10: vkd3d-shader/dxil: Read and validate global abbreviated operands. vkd3d-shader/dxil: Read and validate local abbreviated operands. vkd3d-compiler: Introduce dxbc-dxil and dxbc-auto source types. vkd3d-shader/dxbc: Introduce an option to autodetect the DXBC source type. vkd3d-shader/dxil: Read and validate DXIL bitcode unabbreviated blocks. https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/44
From: Conor McCarthy <cmccarthy(a)codeweavers.com> --- Makefile.am | 1 + include/vkd3d_shader.h | 5 + libs/vkd3d-shader/d3d_asm.c | 6 + libs/vkd3d-shader/dxbc.c | 10 +- libs/vkd3d-shader/dxil.c | 579 +++++++++++++++++++++++ libs/vkd3d-shader/tpf.c | 1 + libs/vkd3d-shader/vkd3d_shader_main.c | 22 +- libs/vkd3d-shader/vkd3d_shader_private.h | 13 + 8 files changed, 631 insertions(+), 6 deletions(-) create mode 100644 libs/vkd3d-shader/dxil.c diff --git a/Makefile.am b/Makefile.am index e06d0eeb..88975070 100644 --- a/Makefile.am +++ b/Makefile.am @@ -265,6 +265,7 @@ libvkd3d_shader_la_SOURCES = \ libs/vkd3d-shader/d3d_asm.c \ libs/vkd3d-shader/d3dbc.c \ libs/vkd3d-shader/dxbc.c \ + libs/vkd3d-shader/dxil.c \ libs/vkd3d-shader/glsl.c \ libs/vkd3d-shader/hlsl.c \ libs/vkd3d-shader/hlsl.h \ diff --git a/include/vkd3d_shader.h b/include/vkd3d_shader.h index 859b8c79..64ad59dc 100644 --- a/include/vkd3d_shader.h +++ b/include/vkd3d_shader.h @@ -610,6 +610,11 @@ enum vkd3d_shader_source_type * model 1, 2, and 3 shaders. \since 1.3 */ VKD3D_SHADER_SOURCE_D3D_BYTECODE, + /** + * A 'DirectX Intermediate Language' shader embedded in a DXBC container. This is + * the format used for Direct3D shader model 6 shaders. \since 1.8 + */ + VKD3D_SHADER_SOURCE_DXBC_DXIL, VKD3D_FORCE_32_BIT_ENUM(VKD3D_SHADER_SOURCE_TYPE), }; diff --git a/libs/vkd3d-shader/d3d_asm.c b/libs/vkd3d-shader/d3d_asm.c index 9438bfac..0a8c2282 100644 --- a/libs/vkd3d-shader/d3d_asm.c +++ b/libs/vkd3d-shader/d3d_asm.c @@ -1926,6 +1926,12 @@ enum vkd3d_result vkd3d_dxbc_binary_to_text(const struct vkd3d_shader_instructio shader_get_type_prefix(shader_version->type), shader_version->major, shader_version->minor, compiler.colours.reset); + if (shader_version->major >= 6) + { + FIXME("Shader model 6 tracing is not implemented yet.\n"); + return VKD3D_ERROR_NOT_IMPLEMENTED; + } + indent = 0; for (i = 0; i < instructions->count; ++i) { diff --git a/libs/vkd3d-shader/dxbc.c b/libs/vkd3d-shader/dxbc.c index 743ba369..738e1388 100644 --- a/libs/vkd3d-shader/dxbc.c +++ b/libs/vkd3d-shader/dxbc.c @@ -491,8 +491,14 @@ static int shdr_handler(const struct vkd3d_shader_dxbc_section_desc *section, return ret; break; + case TAG_DXIL: case TAG_SHDR: case TAG_SHEX: + if ((section->tag == TAG_DXIL) != desc->is_dxil) + { + TRACE("Skipping chunk %#x.\n", section->tag); + break; + } if (desc->byte_code) FIXME("Multiple shader code chunks.\n"); desc->byte_code = section->data.code; @@ -503,10 +509,6 @@ static int shdr_handler(const struct vkd3d_shader_dxbc_section_desc *section, TRACE("Skipping AON9 shader code chunk.\n"); break; - case TAG_DXIL: - FIXME("Skipping DXIL shader model 6+ code chunk.\n"); - break; - default: TRACE("Skipping chunk %#x.\n", section->tag); break; diff --git a/libs/vkd3d-shader/dxil.c b/libs/vkd3d-shader/dxil.c new file mode 100644 index 00000000..eed95e20 --- /dev/null +++ b/libs/vkd3d-shader/dxil.c @@ -0,0 +1,579 @@ +/* + * Copyright 2023 Conor McCarthy for CodeWeavers + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA + */ + +#include "vkd3d_shader_private.h" + +#define VKD3D_SM6_VERSION_MAJOR(version) (((version) >> 4) & 0xf) +#define VKD3D_SM6_VERSION_MINOR(version) (((version) >> 0) & 0xf) + +#define BITCODE_MAGIC VKD3D_MAKE_TAG('B', 'C', 0xc0, 0xde) + +enum bitcode_block_id +{ + BLOCKINFO_BLOCK = 0, + MODULE_BLOCK = 8, + PARAMATTR_BLOCK = 9, + PARAMATTR_GROUP_BLOCK = 10, + CONSTANTS_BLOCK = 11, + FUNCTION_BLOCK = 12, + VALUE_SYMTAB_BLOCK = 14, + METADATA_BLOCK = 15, + METADATA_ATTACHMENT_BLOCK = 16, + TYPE_BLOCK = 17, + USELIST_BLOCK = 18, +}; + +enum bitcode_blockinfo_code +{ + SETBID = 1, + BLOCKNAME = 2, + SETRECORDNAME = 3, +}; + +enum bitcode_block_abbreviation +{ + END_BLOCK = 0, + ENTER_SUBBLOCK = 1, + DEFINE_ABBREV = 2, + UNABBREV_RECORD = 3, +}; + +struct dxil_record +{ + unsigned int code; + unsigned int operand_count; + uint64_t operands[]; +}; + +struct dxil_block +{ + const struct dxil_block *parent; + enum bitcode_block_id id; + unsigned int abbrev_len; + unsigned int start; + unsigned int length; + unsigned int level; + + unsigned int blockinfo_bid; + + struct dxil_block **child_blocks; + size_t child_block_capacity; + size_t child_block_count; + + struct dxil_record **records; + size_t record_capacity; + size_t record_count; +}; + +struct sm6_parser +{ + const uint32_t *ptr, *start, *end; + unsigned int bitpos; + + struct dxil_block root_block; + struct dxil_block *current_block; + + struct vkd3d_shader_parser p; +}; + +static struct sm6_parser *sm6_parser(struct vkd3d_shader_parser *parser) +{ + return CONTAINING_RECORD(parser, struct sm6_parser, p); +} + +static bool sm6_parser_is_end(struct sm6_parser *sm6) +{ + return sm6->ptr == sm6->end; +} + +static uint32_t sm6_parser_read_uint32(struct sm6_parser *sm6) +{ + if (sm6_parser_is_end(sm6)) + { + sm6->p.failed = true; + return 0; + } + return *sm6->ptr++; +} + +static uint32_t sm6_parser_read_bits(struct sm6_parser *sm6, unsigned int length) +{ + unsigned int l, prev_len = 0; + uint32_t bits; + + if (!length) + return 0; + + assert(length < 32); + + if (sm6_parser_is_end(sm6)) + { + sm6->p.failed = true; + return 0; + } + + assert(sm6->bitpos < 32); + bits = *sm6->ptr >> sm6->bitpos; + l = 32 - sm6->bitpos; + if (l <= length) + { + ++sm6->ptr; + if (sm6_parser_is_end(sm6) && l < length) + { + sm6->p.failed = true; + return bits; + } + sm6->bitpos = 0; + bits |= *sm6->ptr << l; + prev_len = l; + } + sm6->bitpos += length - prev_len; + + return bits & ((1 << length) - 1); +} + +static uint64_t sm6_parser_read_vbr(struct sm6_parser *sm6, unsigned int length) +{ + unsigned int bits, flag, mask, shift = 0; + uint64_t result = 0; + + if (!length) + return 0; + + if (sm6_parser_is_end(sm6)) + { + sm6->p.failed = true; + return 0; + } + + flag = 1 << (length - 1); + mask = flag - 1; + do + { + bits = sm6_parser_read_bits(sm6, length); + result |= (uint64_t)(bits & mask) << shift; + shift += length - 1; + } while ((bits & flag) && !sm6->p.failed && shift < 64); + + sm6->p.failed |= !!(bits & flag); + + return result; +} + +static void sm6_parser_align_32(struct sm6_parser *sm6) +{ + if (!sm6->bitpos) + return; + + if (sm6_parser_is_end(sm6)) + { + sm6->p.failed = true; + return; + } + + ++sm6->ptr; + sm6->bitpos = 0; +} + +static bool dxil_block_handle_blockinfo_record(struct dxil_block *block, struct dxil_record *record) +{ + /* BLOCKINFO blocks must only occur immediately below the module root block. */ + if (block->level > 1) + { + WARN("Invalid blockinfo block level %u.\n", block->level); + return false; + } + + switch (record->code) + { + case SETBID: + if (!record->operand_count) + { + WARN("Missing id operand.\n"); + return false; + } + if (record->operands[0] > UINT_MAX) + WARN("Truncating block id %"PRIu64".\n", record->operands[0]); + block->blockinfo_bid = record->operands[0]; + break; + case BLOCKNAME: + case SETRECORDNAME: + break; + default: + FIXME("Unhandled BLOCKINFO record type %u.\n", record->code); + break; + } + + return true; +} + +static enum vkd3d_result dxil_block_add_record(struct dxil_block *block, struct dxil_record *record) +{ + unsigned int reserve; + + switch (block->id) + { + /* Rough initial reserve sizes for small shaders. */ + case CONSTANTS_BLOCK: reserve = 32; break; + case FUNCTION_BLOCK: reserve = 128; break; + case METADATA_BLOCK: reserve = 32; break; + case TYPE_BLOCK: reserve = 32; break; + default: reserve = 8; break; + } + reserve = max(reserve, block->record_count + 1); + if (!vkd3d_array_reserve((void **)&block->records, &block->record_capacity, reserve, sizeof(*block->records))) + { + ERR("Failed to allocate %u records.\n", reserve); + return VKD3D_ERROR_OUT_OF_MEMORY; + } + + if (block->id == BLOCKINFO_BLOCK && !dxil_block_handle_blockinfo_record(block, record)) + return VKD3D_ERROR_INVALID_SHADER; + + block->records[block->record_count++] = record; + + return VKD3D_OK; +} + +static enum vkd3d_result sm6_parser_read_unabbrev_record(struct sm6_parser *sm6) +{ + struct dxil_block *block = sm6->current_block; + enum vkd3d_result ret = VKD3D_OK; + unsigned int code, count, i; + struct dxil_record *record; + + code = sm6_parser_read_vbr(sm6, 6); + + count = sm6_parser_read_vbr(sm6, 6); + if (!(record = vkd3d_malloc(sizeof(*record) + count * sizeof(record->operands[0])))) + { + ERR("Failed to allocate record with %u operands.\n", count); + return VKD3D_ERROR_OUT_OF_MEMORY; + } + + record->code = code; + record->operand_count = count; + + for (i = 0; i < count; ++i) + record->operands[i] = sm6_parser_read_vbr(sm6, 6); + if (sm6->p.failed) + ret = VKD3D_ERROR_INVALID_SHADER; + + if (ret < 0 || (ret = dxil_block_add_record(block, record)) < 0) + vkd3d_free(record); + + return ret; +} + +static enum vkd3d_result dxil_block_init(struct dxil_block *block, const struct dxil_block *parent, + struct sm6_parser *sm6); + +static enum vkd3d_result dxil_block_read(struct dxil_block *parent, struct sm6_parser *sm6) +{ + unsigned int reserve = (parent->id == MODULE_BLOCK) ? 12 : 2; + struct dxil_block *block; + enum vkd3d_result ret; + + sm6->current_block = parent; + + do + { + unsigned int abbrev_id = sm6_parser_read_bits(sm6, parent->abbrev_len); + + switch (abbrev_id) + { + case END_BLOCK: + sm6_parser_align_32(sm6); + return VKD3D_OK; + + case ENTER_SUBBLOCK: + if (parent->id != MODULE_BLOCK && parent->id != FUNCTION_BLOCK) + { + WARN("Invalid subblock parent id %u.\n", parent->id); + return VKD3D_ERROR_INVALID_SHADER; + } + + if (!vkd3d_array_reserve((void **)&parent->child_blocks, &parent->child_block_capacity, + max(reserve, parent->child_block_count + 1), sizeof(*parent->child_blocks)) + || !(block = vkd3d_calloc(1, sizeof(*block)))) + { + ERR("Failed to allocate block.\n"); + return VKD3D_ERROR_OUT_OF_MEMORY; + } + + if ((ret = dxil_block_init(block, parent, sm6)) < 0) + { + vkd3d_free(block); + return ret; + } + + parent->child_blocks[parent->child_block_count++] = block; + sm6->current_block = parent; + break; + + case DEFINE_ABBREV: + FIXME("Unhandled abbreviation definition.\n"); + return VKD3D_ERROR_INVALID_SHADER; + + case UNABBREV_RECORD: + if ((ret = sm6_parser_read_unabbrev_record(sm6)) < 0) + { + WARN("Failed to read unabbreviated record.\n"); + return ret; + } + break; + + default: + FIXME("Unhandled abbreviated record %u.\n", abbrev_id); + return VKD3D_ERROR_INVALID_SHADER; + } + } while (!sm6->p.failed); + + return VKD3D_ERROR_INVALID_SHADER; +} + +static void dxil_block_destroy(struct dxil_block *block) +{ + unsigned int i; + + for (i = 0; i < block->record_count; ++i) + vkd3d_free(block->records[i]); + vkd3d_free(block->records); + + for (i = 0; i < block->child_block_count; ++i) + { + dxil_block_destroy(block->child_blocks[i]); + vkd3d_free(block->child_blocks[i]); + } + vkd3d_free(block->child_blocks); + + block->records = NULL; + block->record_count = 0; + block->child_blocks = NULL; + block->child_block_count = 0; +} + +static enum vkd3d_result dxil_block_init(struct dxil_block *block, const struct dxil_block *parent, + struct sm6_parser *sm6) +{ + enum vkd3d_result ret; + + block->parent = parent; + block->level = parent ? parent->level + 1 : 0; + block->id = sm6_parser_read_vbr(sm6, 8); + block->abbrev_len = sm6_parser_read_vbr(sm6, 4); + sm6_parser_align_32(sm6); + block->length = sm6_parser_read_uint32(sm6); + block->start = sm6->ptr - sm6->start; + + if (sm6->p.failed) + return VKD3D_ERROR_INVALID_SHADER; + + if ((ret = dxil_block_read(block, sm6)) < 0) + dxil_block_destroy(block); + + return ret; +} + +static void sm6_parser_destroy(struct vkd3d_shader_parser *parser) +{ + struct sm6_parser *sm6 = sm6_parser(parser); + + dxil_block_destroy(&sm6->root_block); + shader_instruction_array_destroy(&parser->instructions); + free_shader_desc(&parser->shader_desc); + vkd3d_free(sm6); +} + +static const struct vkd3d_shader_parser_ops sm6_parser_ops = +{ + .parser_destroy = sm6_parser_destroy, +}; + +static enum vkd3d_result sm6_parser_init(struct sm6_parser *sm6, const uint32_t *byte_code, size_t byte_code_size, + const char *source_name, struct vkd3d_shader_message_context *message_context) +{ + const struct vkd3d_shader_location location = {.source_name = source_name}; + uint32_t version_token, dxil_version, token_count; + enum bitcode_block_abbreviation abbr; + struct vkd3d_shader_version version; + unsigned int count, length; + struct dxil_block *block; + enum vkd3d_result ret; + + count = byte_code_size / sizeof(*byte_code); + if (count < 6) + { + WARN("Invalid data size %zu.\n", byte_code_size); + vkd3d_shader_error(message_context, &location, VKD3D_SHADER_ERROR_DXIL_INVALID_SIZE, + "DXIL chunk size %zu is smaller than the DXIL header size.", byte_code_size); + return VKD3D_ERROR_INVALID_SHADER; + } + + version_token = byte_code[0]; + TRACE("Compiler version: 0x%08x.\n", version_token); + token_count = byte_code[1]; + TRACE("Token count: %u.\n", token_count); + + if (token_count < 6 || count < token_count) + { + WARN("Invalid token count %u (word count %u).\n", token_count, count); + vkd3d_shader_error(message_context, &location, VKD3D_SHADER_ERROR_DXIL_INVALID_CHUNK_SIZE, + "DXIL chunk token count %#x is invalid (word count %u).", token_count, count); + return VKD3D_ERROR_INVALID_SHADER; + } + + if (byte_code[2] != TAG_DXIL) + WARN("Unrecognised magic number 0x%08x.\n", byte_code[2]); + + dxil_version = byte_code[3]; + if (dxil_version != 0x100) + WARN("Unknown DXIL version: 0x%08x.\n", dxil_version); + else + TRACE("DXIL version: 0x%08x.\n", dxil_version); + + if (byte_code[4] < 16 || byte_code[4] >= byte_code_size) + { + WARN("Invalid bitcode chunk offset %#x (data size %zu).\n", byte_code[4], byte_code_size); + vkd3d_shader_error(message_context, &location, VKD3D_SHADER_ERROR_DXIL_INVALID_CHUNK_OFFSET, + "DXIL bitcode chunk has invalid offset %#x (data size %#zx).", byte_code[4], byte_code_size); + return VKD3D_ERROR_INVALID_SHADER; + } + if (byte_code[4] + byte_code[5] > byte_code_size) + { + WARN("Invalid bitcode chunk size %#x (data size %zu, chunk offset %#x).\n", + byte_code[5], byte_code_size, byte_code[4]); + vkd3d_shader_error(message_context, &location, VKD3D_SHADER_ERROR_DXIL_INVALID_CHUNK_SIZE, + "DXIL bitcode chunk has invalid size %#x (data size %#zx, chunk offset %#x).", + byte_code[5], byte_code_size, byte_code[4]); + return VKD3D_ERROR_INVALID_SHADER; + } + + sm6->start = (const uint32_t *)((const char*)&byte_code[2] + byte_code[4]); + if (sm6->start[0] != BITCODE_MAGIC) + WARN("Unrecognised magic number 0x%08x.\n", sm6->start[0]); + + sm6->end = &sm6->start[(byte_code[5] + sizeof(*sm6->start) - 1) / sizeof(*sm6->start)]; + + if ((version.type = version_token >> 16) >= VKD3D_SHADER_TYPE_COUNT) + FIXME("Unrecognised shader type %#x.\n", version.type); + + version.major = VKD3D_SM6_VERSION_MAJOR(version_token); + version.minor = VKD3D_SM6_VERSION_MINOR(version_token); + + if ((abbr = sm6->start[1] & 3) != ENTER_SUBBLOCK) + { + WARN("Initial block abbreviation %u is not ENTER_SUBBLOCK.\n", abbr); + vkd3d_shader_error(message_context, &location, VKD3D_SHADER_ERROR_DXIL_INVALID_BITCODE, + "DXIL bitcode chunk has invalid initial block abbreviation %u.", abbr); + return VKD3D_ERROR_INVALID_SHADER; + } + + count = max(token_count, 400) - 400; + vkd3d_shader_parser_init(&sm6->p, message_context, source_name, &version, &sm6_parser_ops, + (count + (count >> 2)) / 2u + 10); + sm6->ptr = &sm6->start[1]; + sm6->bitpos = 2; + + block = &sm6->root_block; + if ((ret = dxil_block_init(block, NULL, sm6)) < 0) + { + if (ret == VKD3D_ERROR_OUT_OF_MEMORY) + vkd3d_shader_error(message_context, &location, VKD3D_SHADER_ERROR_DXIL_OUT_OF_MEMORY, + "Out of memory parsing DXIL bitcode chunk."); + else if (ret == VKD3D_ERROR_INVALID_SHADER) + vkd3d_shader_error(message_context, &location, VKD3D_SHADER_ERROR_DXIL_INVALID_BITCODE, + "DXIL bitcode chunk has invalid bitcode."); + else + vkd3d_unreachable(); + return ret; + } + + length = sm6->ptr - sm6->start - block->start; + if (length != block->length) + { + WARN("Invalid block length %u; expected %u.\n", length, block->length); + vkd3d_shader_parser_warning(&sm6->p, VKD3D_SHADER_WARNING_DXIL_INVALID_BLOCK_LENGTH, + "Root block ends with length %u but indicated length is %u.", length, block->length); + } + if (sm6->ptr != sm6->end) + { + unsigned int expected_length = sm6->end - sm6->start; + length = sm6->ptr - sm6->start; + WARN("Invalid module length %u; expected %u.\n", length, expected_length); + vkd3d_shader_parser_warning(&sm6->p, VKD3D_SHADER_WARNING_DXIL_INVALID_MODULE_LENGTH, + "Module ends with length %u but indicated length is %u.", length, expected_length); + } + + dxil_block_destroy(&sm6->root_block); + + return VKD3D_OK; +} + +int vkd3d_shader_sm6_parser_create(const struct vkd3d_shader_compile_info *compile_info, + struct vkd3d_shader_message_context *message_context, struct vkd3d_shader_parser **parser) +{ + struct vkd3d_shader_desc *shader_desc; + uint32_t *byte_code = NULL; + struct sm6_parser *sm6; + int ret; + + if (!(sm6 = vkd3d_calloc(1, sizeof(*sm6)))) + { + ERR("Failed to allocate parser.\n"); + return VKD3D_ERROR_OUT_OF_MEMORY; + } + + shader_desc = &sm6->p.shader_desc; + shader_desc->is_dxil = true; + if ((ret = shader_extract_from_dxbc(&compile_info->source, message_context, compile_info->source_name, + shader_desc)) < 0) + { + WARN("Failed to extract shader, vkd3d result %d.\n", ret); + vkd3d_free(sm6); + return ret; + } + + sm6->p.shader_desc = *shader_desc; + shader_desc = &sm6->p.shader_desc; + + if (((uintptr_t)shader_desc->byte_code & (VKD3D_DXBC_CHUNK_ALIGNMENT - 1))) + { + /* LLVM bitcode should be 32-bit aligned, but before dxc v1.7.2207 this was not always the case in the DXBC + * container due to missing padding after signature names. Get an aligned copy to prevent unaligned access. */ + if (!(byte_code = vkd3d_malloc(align(shader_desc->byte_code_size, VKD3D_DXBC_CHUNK_ALIGNMENT)))) + ERR("Failed to allocate aligned chunk. Unaligned access will occur.\n"); + else + memcpy(byte_code, shader_desc->byte_code, shader_desc->byte_code_size); + } + + ret = sm6_parser_init(sm6, byte_code ? byte_code : shader_desc->byte_code, shader_desc->byte_code_size, + compile_info->source_name, message_context); + vkd3d_free(byte_code); + + if (ret < 0) + { + WARN("Failed to initialise shader parser.\n"); + sm6_parser_destroy(&sm6->p); + return ret; + } + + *parser = &sm6->p; + + return ret; +} diff --git a/libs/vkd3d-shader/tpf.c b/libs/vkd3d-shader/tpf.c index eadad3b9..521e0daa 100644 --- a/libs/vkd3d-shader/tpf.c +++ b/libs/vkd3d-shader/tpf.c @@ -2442,6 +2442,7 @@ int vkd3d_shader_sm4_parser_create(const struct vkd3d_shader_compile_info *compi } shader_desc = &sm4->p.shader_desc; + shader_desc->is_dxil = false; if ((ret = shader_extract_from_dxbc(&compile_info->source, message_context, compile_info->source_name, shader_desc)) < 0) { diff --git a/libs/vkd3d-shader/vkd3d_shader_main.c b/libs/vkd3d-shader/vkd3d_shader_main.c index 409d759c..d767a1a6 100644 --- a/libs/vkd3d-shader/vkd3d_shader_main.c +++ b/libs/vkd3d-shader/vkd3d_shader_main.c @@ -415,6 +415,8 @@ static const char *shader_get_source_type_suffix(enum vkd3d_shader_source_type t return "hlsl"; case VKD3D_SHADER_SOURCE_D3D_BYTECODE: return "d3dbc"; + case VKD3D_SHADER_SOURCE_DXBC_DXIL: + return "dxil"; default: FIXME("Unhandled source type %#x.\n", type); return "bin"; @@ -1088,6 +1090,12 @@ static int scan_with_parser(const struct vkd3d_shader_compile_info *compile_info vkd3d_shader_trace(&parser->instructions, &parser->shader_version); } + if (parser->shader_version.major >= 6) + { + FIXME("DXIL scanning is not implemented yet.\n"); + return VKD3D_ERROR_NOT_IMPLEMENTED; + } + for (i = 0; i < parser->instructions.count; ++i) { instruction = &parser->instructions.elements[i]; @@ -1109,7 +1117,10 @@ static int scan_dxbc(const struct vkd3d_shader_compile_info *compile_info, struct vkd3d_shader_parser *parser; int ret; - if ((ret = vkd3d_shader_sm4_parser_create(compile_info, message_context, &parser)) < 0) + ret = (compile_info->source_type == VKD3D_SHADER_SOURCE_DXBC_DXIL) + ? vkd3d_shader_sm6_parser_create(compile_info, message_context, &parser) + : vkd3d_shader_sm4_parser_create(compile_info, message_context, &parser); + if (ret < 0) { WARN("Failed to initialise shader parser.\n"); return ret; @@ -1156,6 +1167,7 @@ int vkd3d_shader_scan(const struct vkd3d_shader_compile_info *compile_info, char switch (compile_info->source_type) { + case VKD3D_SHADER_SOURCE_DXBC_DXIL: case VKD3D_SHADER_SOURCE_DXBC_TPF: ret = scan_dxbc(compile_info, &message_context); break; @@ -1240,7 +1252,10 @@ static int compile_dxbc_tpf(const struct vkd3d_shader_compile_info *compile_info struct vkd3d_shader_parser *parser; int ret; - if ((ret = vkd3d_shader_sm4_parser_create(compile_info, message_context, &parser)) < 0) + ret = (compile_info->source_type == VKD3D_SHADER_SOURCE_DXBC_DXIL) + ? vkd3d_shader_sm6_parser_create(compile_info, message_context, &parser) + : vkd3d_shader_sm4_parser_create(compile_info, message_context, &parser); + if (ret < 0) { WARN("Failed to initialise shader parser.\n"); return ret; @@ -1309,6 +1324,7 @@ int vkd3d_shader_compile(const struct vkd3d_shader_compile_info *compile_info, switch (compile_info->source_type) { + case VKD3D_SHADER_SOURCE_DXBC_DXIL: case VKD3D_SHADER_SOURCE_DXBC_TPF: ret = compile_dxbc_tpf(compile_info, out, &message_context); break; @@ -1526,6 +1542,7 @@ const enum vkd3d_shader_source_type *vkd3d_shader_get_supported_source_types(uns VKD3D_SHADER_SOURCE_DXBC_TPF, VKD3D_SHADER_SOURCE_HLSL, VKD3D_SHADER_SOURCE_D3D_BYTECODE, + VKD3D_SHADER_SOURCE_DXBC_DXIL, }; TRACE("count %p.\n", count); @@ -1564,6 +1581,7 @@ const enum vkd3d_shader_target_type *vkd3d_shader_get_supported_target_types( switch (source_type) { + case VKD3D_SHADER_SOURCE_DXBC_DXIL: case VKD3D_SHADER_SOURCE_DXBC_TPF: *count = ARRAY_SIZE(dxbc_tpf_types); return dxbc_tpf_types; diff --git a/libs/vkd3d-shader/vkd3d_shader_private.h b/libs/vkd3d-shader/vkd3d_shader_private.h index 0f294d21..132ac622 100644 --- a/libs/vkd3d-shader/vkd3d_shader_private.h +++ b/libs/vkd3d-shader/vkd3d_shader_private.h @@ -147,6 +147,15 @@ enum vkd3d_shader_error VKD3D_SHADER_ERROR_D3DBC_OUT_OF_MEMORY = 7004, VKD3D_SHADER_WARNING_D3DBC_IGNORED_INSTRUCTION_FLAGS= 7300, + + VKD3D_SHADER_ERROR_DXIL_OUT_OF_MEMORY = 8000, + VKD3D_SHADER_ERROR_DXIL_INVALID_SIZE = 8001, + VKD3D_SHADER_ERROR_DXIL_INVALID_CHUNK_OFFSET = 8002, + VKD3D_SHADER_ERROR_DXIL_INVALID_CHUNK_SIZE = 8003, + VKD3D_SHADER_ERROR_DXIL_INVALID_BITCODE = 8004, + + VKD3D_SHADER_WARNING_DXIL_INVALID_BLOCK_LENGTH = 8300, + VKD3D_SHADER_WARNING_DXIL_INVALID_MODULE_LENGTH = 8301, }; enum vkd3d_shader_opcode @@ -810,6 +819,7 @@ struct vkd3d_shader_desc { const uint32_t *byte_code; size_t byte_code_size; + bool is_dxil; struct shader_signature input_signature; struct shader_signature output_signature; struct shader_signature patch_constant_signature; @@ -1161,6 +1171,8 @@ int vkd3d_shader_sm1_parser_create(const struct vkd3d_shader_compile_info *compi struct vkd3d_shader_message_context *message_context, struct vkd3d_shader_parser **parser); int vkd3d_shader_sm4_parser_create(const struct vkd3d_shader_compile_info *compile_info, struct vkd3d_shader_message_context *message_context, struct vkd3d_shader_parser **parser); +int vkd3d_shader_sm6_parser_create(const struct vkd3d_shader_compile_info *compile_info, + struct vkd3d_shader_message_context *message_context, struct vkd3d_shader_parser **parser); void free_shader_desc(struct vkd3d_shader_desc *desc); @@ -1333,6 +1345,7 @@ static inline void *vkd3d_find_struct_(const struct vkd3d_struct *chain, } #define VKD3D_DXBC_HEADER_SIZE (8 * sizeof(uint32_t)) +#define VKD3D_DXBC_CHUNK_ALIGNMENT sizeof(uint32_t) #define TAG_AON9 VKD3D_MAKE_TAG('A', 'o', 'n', '9') #define TAG_DXBC VKD3D_MAKE_TAG('D', 'X', 'B', 'C') -- GitLab https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/44
From: Conor McCarthy <cmccarthy(a)codeweavers.com> --- include/vkd3d_shader.h | 5 ++++ libs/vkd3d-shader/dxbc.c | 36 ++++++++++++++++++++++++ libs/vkd3d-shader/vkd3d_shader_main.c | 22 +++++++++++++-- libs/vkd3d-shader/vkd3d_shader_private.h | 3 ++ libs/vkd3d/state.c | 4 +-- 5 files changed, 66 insertions(+), 4 deletions(-) diff --git a/include/vkd3d_shader.h b/include/vkd3d_shader.h index 64ad59dc..d101231a 100644 --- a/include/vkd3d_shader.h +++ b/include/vkd3d_shader.h @@ -615,6 +615,11 @@ enum vkd3d_shader_source_type * the format used for Direct3D shader model 6 shaders. \since 1.8 */ VKD3D_SHADER_SOURCE_DXBC_DXIL, + /** + * Autodetect VKD3D_SHADER_SOURCE_DXBC_TPF or VKD3D_SHADER_SOURCE_DXBC_DXIL. + * \since 1.9 + */ + VKD3D_SHADER_SOURCE_DXBC_AUTODETECT, VKD3D_FORCE_32_BIT_ENUM(VKD3D_SHADER_SOURCE_TYPE), }; diff --git a/libs/vkd3d-shader/dxbc.c b/libs/vkd3d-shader/dxbc.c index 738e1388..55181902 100644 --- a/libs/vkd3d-shader/dxbc.c +++ b/libs/vkd3d-shader/dxbc.c @@ -524,6 +524,42 @@ void free_shader_desc(struct vkd3d_shader_desc *desc) shader_signature_cleanup(&desc->patch_constant_signature); } +static int autodetect_handler(const struct vkd3d_shader_dxbc_section_desc *section, + struct vkd3d_shader_message_context *message_context, void *ctx) +{ + enum vkd3d_shader_source_type *type = ctx; + + if (section->tag == TAG_SHDR || section->tag == TAG_SHEX) + *type = VKD3D_SHADER_SOURCE_DXBC_TPF; + else if (section->tag == TAG_DXIL) + *type = VKD3D_SHADER_SOURCE_DXBC_DXIL; + + return VKD3D_OK; +} + +int shader_detect_dxbc_source_type(const struct vkd3d_shader_code *dxbc, + struct vkd3d_shader_message_context *message_context, enum vkd3d_shader_source_type *type) +{ + int ret; + + *type = VKD3D_SHADER_SOURCE_NONE; + if ((ret = for_each_dxbc_section(dxbc, message_context, NULL, autodetect_handler, type)) < 0) + { + ERR("Failed to parse source type.\n"); + return ret; + } + + if (*type == VKD3D_SHADER_SOURCE_NONE) + { + ERR("Failed to find a code chunk.\n"); + vkd3d_shader_error(message_context, NULL, VKD3D_SHADER_ERROR_DXBC_MISSING_CODE_CHUNK, + "DXBC does not contain a TPF or DXIL code chunk."); + return VKD3D_ERROR_INVALID_SHADER; + } + + return VKD3D_OK; +} + int shader_extract_from_dxbc(const struct vkd3d_shader_code *dxbc, struct vkd3d_shader_message_context *message_context, const char *source_name, struct vkd3d_shader_desc *desc) { diff --git a/libs/vkd3d-shader/vkd3d_shader_main.c b/libs/vkd3d-shader/vkd3d_shader_main.c index d767a1a6..c87fb2eb 100644 --- a/libs/vkd3d-shader/vkd3d_shader_main.c +++ b/libs/vkd3d-shader/vkd3d_shader_main.c @@ -1114,10 +1114,17 @@ static int scan_with_parser(const struct vkd3d_shader_compile_info *compile_info static int scan_dxbc(const struct vkd3d_shader_compile_info *compile_info, struct vkd3d_shader_message_context *message_context) { + enum vkd3d_shader_source_type source_type = compile_info->source_type; struct vkd3d_shader_parser *parser; int ret; - ret = (compile_info->source_type == VKD3D_SHADER_SOURCE_DXBC_DXIL) + if (source_type == VKD3D_SHADER_SOURCE_DXBC_AUTODETECT + && (ret = shader_detect_dxbc_source_type(&compile_info->source, message_context, &source_type)) < 0) + { + WARN("Failed to autodetect source type.\n"); + return ret; + } + ret = (source_type == VKD3D_SHADER_SOURCE_DXBC_DXIL) ? vkd3d_shader_sm6_parser_create(compile_info, message_context, &parser) : vkd3d_shader_sm4_parser_create(compile_info, message_context, &parser); if (ret < 0) @@ -1167,6 +1174,7 @@ int vkd3d_shader_scan(const struct vkd3d_shader_compile_info *compile_info, char switch (compile_info->source_type) { + case VKD3D_SHADER_SOURCE_DXBC_AUTODETECT: case VKD3D_SHADER_SOURCE_DXBC_DXIL: case VKD3D_SHADER_SOURCE_DXBC_TPF: ret = scan_dxbc(compile_info, &message_context); @@ -1249,10 +1257,17 @@ static int vkd3d_shader_parser_compile(struct vkd3d_shader_parser *parser, static int compile_dxbc_tpf(const struct vkd3d_shader_compile_info *compile_info, struct vkd3d_shader_code *out, struct vkd3d_shader_message_context *message_context) { + enum vkd3d_shader_source_type source_type = compile_info->source_type; struct vkd3d_shader_parser *parser; int ret; - ret = (compile_info->source_type == VKD3D_SHADER_SOURCE_DXBC_DXIL) + if (source_type == VKD3D_SHADER_SOURCE_DXBC_AUTODETECT + && (ret = shader_detect_dxbc_source_type(&compile_info->source, message_context, &source_type)) < 0) + { + WARN("Failed to autodetect source type.\n"); + return ret; + } + ret = (source_type == VKD3D_SHADER_SOURCE_DXBC_DXIL) ? vkd3d_shader_sm6_parser_create(compile_info, message_context, &parser) : vkd3d_shader_sm4_parser_create(compile_info, message_context, &parser); if (ret < 0) @@ -1324,6 +1339,7 @@ int vkd3d_shader_compile(const struct vkd3d_shader_compile_info *compile_info, switch (compile_info->source_type) { + case VKD3D_SHADER_SOURCE_DXBC_AUTODETECT: case VKD3D_SHADER_SOURCE_DXBC_DXIL: case VKD3D_SHADER_SOURCE_DXBC_TPF: ret = compile_dxbc_tpf(compile_info, out, &message_context); @@ -1543,6 +1559,7 @@ const enum vkd3d_shader_source_type *vkd3d_shader_get_supported_source_types(uns VKD3D_SHADER_SOURCE_HLSL, VKD3D_SHADER_SOURCE_D3D_BYTECODE, VKD3D_SHADER_SOURCE_DXBC_DXIL, + VKD3D_SHADER_SOURCE_DXBC_AUTODETECT, }; TRACE("count %p.\n", count); @@ -1581,6 +1598,7 @@ const enum vkd3d_shader_target_type *vkd3d_shader_get_supported_target_types( switch (source_type) { + case VKD3D_SHADER_SOURCE_DXBC_AUTODETECT: case VKD3D_SHADER_SOURCE_DXBC_DXIL: case VKD3D_SHADER_SOURCE_DXBC_TPF: *count = ARRAY_SIZE(dxbc_tpf_types); diff --git a/libs/vkd3d-shader/vkd3d_shader_private.h b/libs/vkd3d-shader/vkd3d_shader_private.h index 132ac622..1d2aec8a 100644 --- a/libs/vkd3d-shader/vkd3d_shader_private.h +++ b/libs/vkd3d-shader/vkd3d_shader_private.h @@ -70,6 +70,7 @@ enum vkd3d_shader_error VKD3D_SHADER_ERROR_DXBC_INVALID_CHUNK_SIZE = 6, VKD3D_SHADER_ERROR_DXBC_OUT_OF_MEMORY = 7, VKD3D_SHADER_ERROR_DXBC_INVALID_SIGNATURE = 8, + VKD3D_SHADER_ERROR_DXBC_MISSING_CODE_CHUNK = 9, VKD3D_SHADER_ERROR_TPF_MISMATCHED_CF = 1000, VKD3D_SHADER_ERROR_TPF_INVALID_REGISTER_RANGE = 1001, @@ -1176,6 +1177,8 @@ int vkd3d_shader_sm6_parser_create(const struct vkd3d_shader_compile_info *compi void free_shader_desc(struct vkd3d_shader_desc *desc); +int shader_detect_dxbc_source_type(const struct vkd3d_shader_code *dxbc, + struct vkd3d_shader_message_context *message_context, enum vkd3d_shader_source_type *type); int shader_extract_from_dxbc(const struct vkd3d_shader_code *dxbc, struct vkd3d_shader_message_context *message_context, const char *source_name, struct vkd3d_shader_desc *desc); int shader_parse_input_signature(const struct vkd3d_shader_code *dxbc, diff --git a/libs/vkd3d/state.c b/libs/vkd3d/state.c index c964ea8f..2755c5f2 100644 --- a/libs/vkd3d/state.c +++ b/libs/vkd3d/state.c @@ -1978,7 +1978,7 @@ static HRESULT create_shader_stage(struct d3d12_device *device, compile_info.next = shader_interface; compile_info.source.code = code->pShaderBytecode; compile_info.source.size = code->BytecodeLength; - compile_info.source_type = VKD3D_SHADER_SOURCE_DXBC_TPF; + compile_info.source_type = VKD3D_SHADER_SOURCE_DXBC_AUTODETECT; compile_info.target_type = VKD3D_SHADER_TARGET_SPIRV_BINARY; compile_info.options = options; compile_info.option_count = ARRAY_SIZE(options); @@ -2019,7 +2019,7 @@ static int vkd3d_scan_dxbc(const struct d3d12_device *device, const D3D12_SHADER compile_info.next = descriptor_info; compile_info.source.code = code->pShaderBytecode; compile_info.source.size = code->BytecodeLength; - compile_info.source_type = VKD3D_SHADER_SOURCE_DXBC_TPF; + compile_info.source_type = VKD3D_SHADER_SOURCE_DXBC_AUTODETECT; compile_info.target_type = VKD3D_SHADER_TARGET_SPIRV_BINARY; compile_info.options = options; compile_info.option_count = ARRAY_SIZE(options); -- GitLab https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/44
From: Conor McCarthy <cmccarthy(a)codeweavers.com> --- programs/vkd3d-compiler/main.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/programs/vkd3d-compiler/main.c b/programs/vkd3d-compiler/main.c index 52488e86..f2215d21 100644 --- a/programs/vkd3d-compiler/main.c +++ b/programs/vkd3d-compiler/main.c @@ -73,6 +73,13 @@ source_type_info[] = "d3dbc", "Legacy Direct3D byte-code.\n" " This is the format used for Direct3D shader model 1, 2, and 3 shaders.\n", true, VKD3D_SHADER_TARGET_SPIRV_BINARY}, + {VKD3D_SHADER_SOURCE_DXBC_DXIL, + "dxbc-dxil", "A 'DirectX Intermediate Language' shader embedded in a DXBC container.\n" + " This is the format used for Direct3D shader model 6 shaders.\n", + true, VKD3D_SHADER_TARGET_SPIRV_BINARY}, + {VKD3D_SHADER_SOURCE_DXBC_AUTODETECT, + "dxbc-auto", "Autodetect a 'TPF' or 'DXIL' shader embedded in a DXBC container.\n", + true, VKD3D_SHADER_TARGET_SPIRV_BINARY}, }; static const struct target_type_info @@ -671,7 +678,7 @@ int main(int argc, char **argv) { memcpy(&token, info.source.code, sizeof(token)); if (token == TAG_DXBC) - options.source_type = VKD3D_SHADER_SOURCE_DXBC_TPF; + options.source_type = VKD3D_SHADER_SOURCE_DXBC_AUTODETECT; else if ((token & 0xfffe0000) == 0xfffe0000) options.source_type = VKD3D_SHADER_SOURCE_D3D_BYTECODE; else -- GitLab https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/44
From: Conor McCarthy <cmccarthy(a)codeweavers.com> --- libs/vkd3d-shader/dxil.c | 248 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 244 insertions(+), 4 deletions(-) diff --git a/libs/vkd3d-shader/dxil.c b/libs/vkd3d-shader/dxil.c index eed95e20..5f67bb20 100644 --- a/libs/vkd3d-shader/dxil.c +++ b/libs/vkd3d-shader/dxil.c @@ -53,6 +53,15 @@ enum bitcode_block_abbreviation UNABBREV_RECORD = 3, }; +enum bitcode_abbrev_type +{ + ABBREV_FIXED = 1, + ABBREV_VBR = 2, + ABBREV_ARRAY = 3, + ABBREV_CHAR = 4, + ABBREV_BLOB = 5, +}; + struct dxil_record { unsigned int code; @@ -69,6 +78,10 @@ struct dxil_block unsigned int length; unsigned int level; + /* The abbrev, block and record structs are not relocatable. */ + struct dxil_abbrev **abbrevs; + size_t abbrev_capacity; + size_t abbrev_count; unsigned int blockinfo_bid; struct dxil_block **child_blocks; @@ -91,6 +104,19 @@ struct sm6_parser struct vkd3d_shader_parser p; }; +struct dxil_abbrev_operand +{ + uint64_t context; + bool (*read_operand)(struct sm6_parser *sm6, uint64_t context, uint64_t *operand); +}; + +struct dxil_abbrev +{ + unsigned int count; + bool is_array; + struct dxil_abbrev_operand operands[]; +}; + static struct sm6_parser *sm6_parser(struct vkd3d_shader_parser *parser) { return CONTAINING_RECORD(parser, struct sm6_parser, p); @@ -280,6 +306,208 @@ static enum vkd3d_result sm6_parser_read_unabbrev_record(struct sm6_parser *sm6) return ret; } +static bool sm6_parser_read_literal_operand(struct sm6_parser *sm6, uint64_t context, uint64_t *op) +{ + *op = context; + return !sm6->p.failed; +} + +static bool sm6_parser_read_fixed_operand(struct sm6_parser *sm6, uint64_t context, uint64_t *op) +{ + *op = sm6_parser_read_bits(sm6, context); + return !sm6->p.failed; +} + +static bool sm6_parser_read_vbr_operand(struct sm6_parser *sm6, uint64_t context, uint64_t *op) +{ + *op = sm6_parser_read_vbr(sm6, context); + return !sm6->p.failed; +} + +static bool sm6_parser_read_char6_operand(struct sm6_parser *sm6, uint64_t context, uint64_t *op) +{ + *op = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789._"[sm6_parser_read_bits(sm6, 6)]; + return !sm6->p.failed; +} + +static bool sm6_parser_read_blob_operand(struct sm6_parser *sm6, uint64_t context, uint64_t *op) +{ + int count = sm6_parser_read_vbr(sm6, 6); + sm6_parser_align_32(sm6); + for (; count > 0; count -= 4) + sm6_parser_read_uint32(sm6); + FIXME("Unhandled blob operand.\n"); + return false; +} + +static enum vkd3d_result dxil_abbrev_init(struct dxil_abbrev *abbrev, unsigned int count, struct sm6_parser *sm6) +{ + enum bitcode_abbrev_type prev_type, type; + unsigned int i; + + abbrev->is_array = false; + + for (i = 0, prev_type = 0; i < count && !sm6->p.failed; ++i) + { + if (sm6_parser_read_bits(sm6, 1)) + { + if (prev_type == ABBREV_ARRAY) + { + WARN("Unexpected literal abbreviation after array.\n"); + return VKD3D_ERROR_INVALID_SHADER; + } + abbrev->operands[i].context = sm6_parser_read_vbr(sm6, 8); + abbrev->operands[i].read_operand = sm6_parser_read_literal_operand; + continue; + } + + switch (type = sm6_parser_read_bits(sm6, 3)) + { + case ABBREV_FIXED: + case ABBREV_VBR: + abbrev->operands[i].context = sm6_parser_read_vbr(sm6, 5); + abbrev->operands[i].read_operand = (type == ABBREV_FIXED) ? sm6_parser_read_fixed_operand + : sm6_parser_read_vbr_operand; + break; + + case ABBREV_ARRAY: + if (prev_type == ABBREV_ARRAY || i != count - 2) + { + WARN("Unexpected array abbreviation.\n"); + return VKD3D_ERROR_INVALID_SHADER; + } + abbrev->is_array = true; + --i; + --count; + break; + + case ABBREV_CHAR: + abbrev->operands[i].read_operand = sm6_parser_read_char6_operand; + break; + + case ABBREV_BLOB: + if (prev_type == ABBREV_ARRAY || i != count - 1) + { + WARN("Unexpected blob abbreviation.\n"); + return VKD3D_ERROR_INVALID_SHADER; + } + abbrev->operands[i].read_operand = sm6_parser_read_blob_operand; + break; + } + + prev_type = type; + } + + abbrev->count = count; + + return sm6->p.failed ? VKD3D_ERROR_INVALID_SHADER : VKD3D_OK; +} + +static enum vkd3d_result sm6_parser_add_block_abbrev(struct sm6_parser *sm6) +{ + struct dxil_block *block = sm6->current_block; + struct dxil_abbrev *abbrev; + enum vkd3d_result ret; + unsigned int count; + + if (block->id == BLOCKINFO_BLOCK) + { + FIXME("Unhandled global abbreviation.\n"); + return VKD3D_ERROR_INVALID_SHADER; + } + + count = sm6_parser_read_vbr(sm6, 5); + if (!vkd3d_array_reserve((void **)&block->abbrevs, &block->abbrev_capacity, block->abbrev_count + 1, sizeof(*block->abbrevs)) + || !(abbrev = vkd3d_malloc(sizeof(*abbrev) + count * sizeof(abbrev->operands[0])))) + { + ERR("Failed to allocate block abbreviation.\n"); + return VKD3D_ERROR_OUT_OF_MEMORY; + } + + if ((ret = dxil_abbrev_init(abbrev, count, sm6)) < 0) + { + vkd3d_free(abbrev); + return ret; + } + + block->abbrevs[block->abbrev_count++] = abbrev; + + return VKD3D_OK; +} + +static enum vkd3d_result sm6_parser_read_abbrev_record(struct sm6_parser *sm6, unsigned int abbrev_id) +{ + enum vkd3d_result ret = VKD3D_ERROR_INVALID_SHADER; + struct dxil_block *block = sm6->current_block; + struct dxil_record *temp, *record; + unsigned int i, count, array_len; + struct dxil_abbrev *abbrev; + uint64_t code; + + if (abbrev_id >= block->abbrev_count) + { + WARN("Invalid abbreviation id %u.\n", abbrev_id); + return VKD3D_ERROR_INVALID_SHADER; + } + + abbrev = block->abbrevs[abbrev_id]; + if (!(count = abbrev->count)) + return VKD3D_OK; + if (count == 1 && abbrev->is_array) + return VKD3D_ERROR_INVALID_SHADER; + + /* First operand is the record code. The array is included in the count, but will be done separately. */ + count -= abbrev->is_array + 1; + if (!(record = vkd3d_malloc(sizeof(*record) + count * sizeof(record->operands[0])))) + { + ERR("Failed to allocate record with %u operands.\n", count); + return VKD3D_ERROR_OUT_OF_MEMORY; + } + + if (!abbrev->operands[0].read_operand(sm6, abbrev->operands[0].context, &code)) + goto fail; + if (code > UINT_MAX) + FIXME("Truncating 64-bit record code %#"PRIx64".\n", code); + record->code = code; + + for (i = 0; i < count; ++i) + if (!abbrev->operands[i + 1].read_operand(sm6, abbrev->operands[i + 1].context, &record->operands[i])) + goto fail; + record->operand_count = count; + + /* An array can occur only as the last operand. */ + if (abbrev->is_array) + { + array_len = sm6_parser_read_vbr(sm6, 6); + if (!(temp = vkd3d_realloc(record, sizeof(*record) + (count + array_len) * sizeof(record->operands[0])))) + { + ERR("Failed to allocate record with %u operands.\n", count + array_len); + ret = VKD3D_ERROR_OUT_OF_MEMORY; + goto fail; + } + record = temp; + + for (i = 0; i < array_len; ++i) + { + if (!abbrev->operands[count + 1].read_operand(sm6, abbrev->operands[count + 1].context, + &record->operands[count + i])) + { + goto fail; + } + } + record->operand_count += array_len; + } + + if ((ret = dxil_block_add_record(block, record)) < 0) + goto fail; + + return VKD3D_OK; + +fail: + vkd3d_free(record); + return ret; +} + static enum vkd3d_result dxil_block_init(struct dxil_block *block, const struct dxil_block *parent, struct sm6_parser *sm6); @@ -327,8 +555,9 @@ static enum vkd3d_result dxil_block_read(struct dxil_block *parent, struct sm6_p break; case DEFINE_ABBREV: - FIXME("Unhandled abbreviation definition.\n"); - return VKD3D_ERROR_INVALID_SHADER; + if ((ret = sm6_parser_add_block_abbrev(sm6)) < 0) + return ret; + break; case UNABBREV_RECORD: if ((ret = sm6_parser_read_unabbrev_record(sm6)) < 0) @@ -339,8 +568,12 @@ static enum vkd3d_result dxil_block_read(struct dxil_block *parent, struct sm6_p break; default: - FIXME("Unhandled abbreviated record %u.\n", abbrev_id); - return VKD3D_ERROR_INVALID_SHADER; + if ((ret = sm6_parser_read_abbrev_record(sm6, abbrev_id - 4)) < 0) + { + WARN("Failed to read abbreviated record.\n"); + return ret; + } + break; } } while (!sm6->p.failed); @@ -372,6 +605,7 @@ static enum vkd3d_result dxil_block_init(struct dxil_block *block, const struct struct sm6_parser *sm6) { enum vkd3d_result ret; + unsigned int i; block->parent = parent; block->level = parent ? parent->level + 1 : 0; @@ -387,6 +621,12 @@ static enum vkd3d_result dxil_block_init(struct dxil_block *block, const struct if ((ret = dxil_block_read(block, sm6)) < 0) dxil_block_destroy(block); + for (i = 0; i < block->abbrev_count; ++i) + vkd3d_free(block->abbrevs[i]); + vkd3d_free(block->abbrevs); + block->abbrevs = NULL; + block->abbrev_count = 0; + return ret; } -- GitLab https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/44
From: Conor McCarthy <cmccarthy(a)codeweavers.com> --- libs/vkd3d-shader/dxil.c | 102 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 96 insertions(+), 6 deletions(-) diff --git a/libs/vkd3d-shader/dxil.c b/libs/vkd3d-shader/dxil.c index 5f67bb20..a9f3df9f 100644 --- a/libs/vkd3d-shader/dxil.c +++ b/libs/vkd3d-shader/dxil.c @@ -83,6 +83,7 @@ struct dxil_block size_t abbrev_capacity; size_t abbrev_count; unsigned int blockinfo_bid; + bool has_bid; struct dxil_block **child_blocks; size_t child_block_capacity; @@ -101,6 +102,10 @@ struct sm6_parser struct dxil_block root_block; struct dxil_block *current_block; + struct dxil_global_abbrev **abbrevs; + size_t abbrev_capacity; + size_t abbrev_count; + struct vkd3d_shader_parser p; }; @@ -117,6 +122,12 @@ struct dxil_abbrev struct dxil_abbrev_operand operands[]; }; +struct dxil_global_abbrev +{ + unsigned int block_id; + struct dxil_abbrev abbrev; +}; + static struct sm6_parser *sm6_parser(struct vkd3d_shader_parser *parser) { return CONTAINING_RECORD(parser, struct sm6_parser, p); @@ -236,6 +247,12 @@ static bool dxil_block_handle_blockinfo_record(struct dxil_block *block, struct if (record->operands[0] > UINT_MAX) WARN("Truncating block id %"PRIu64".\n", record->operands[0]); block->blockinfo_bid = record->operands[0]; + if (block->parent && block->blockinfo_bid == block->parent->id) + { + FIXME("Unhandled nested blockinfo block id.\n"); + return false; + } + block->has_bid = true; break; case BLOCKNAME: case SETRECORDNAME: @@ -403,6 +420,40 @@ static enum vkd3d_result dxil_abbrev_init(struct dxil_abbrev *abbrev, unsigned i return sm6->p.failed ? VKD3D_ERROR_INVALID_SHADER : VKD3D_OK; } +static enum vkd3d_result sm6_parser_add_global_abbrev(struct sm6_parser *sm6) +{ + struct dxil_block *block = sm6->current_block; + unsigned int count = sm6_parser_read_vbr(sm6, 5); + struct dxil_global_abbrev *global_abbrev; + enum vkd3d_result ret; + + assert(block->id == BLOCKINFO_BLOCK); + + if (!vkd3d_array_reserve((void **)&sm6->abbrevs, &sm6->abbrev_capacity, sm6->abbrev_count + 1, sizeof(*sm6->abbrevs)) + || !(global_abbrev = vkd3d_malloc(sizeof(*global_abbrev) + count * sizeof(global_abbrev->abbrev.operands[0])))) + { + ERR("Failed to allocate global abbreviation.\n"); + return VKD3D_ERROR_OUT_OF_MEMORY; + } + + if ((ret = dxil_abbrev_init(&global_abbrev->abbrev, count, sm6)) < 0) + { + vkd3d_free(global_abbrev); + return ret; + } + + if (!block->has_bid) + { + WARN("Missing blockinfo block id.\n"); + return VKD3D_ERROR_INVALID_SHADER; + } + global_abbrev->block_id = block->blockinfo_bid; + + sm6->abbrevs[sm6->abbrev_count++] = global_abbrev; + + return VKD3D_OK; +} + static enum vkd3d_result sm6_parser_add_block_abbrev(struct sm6_parser *sm6) { struct dxil_block *block = sm6->current_block; @@ -411,10 +462,7 @@ static enum vkd3d_result sm6_parser_add_block_abbrev(struct sm6_parser *sm6) unsigned int count; if (block->id == BLOCKINFO_BLOCK) - { - FIXME("Unhandled global abbreviation.\n"); - return VKD3D_ERROR_INVALID_SHADER; - } + return sm6_parser_add_global_abbrev(sm6); count = sm6_parser_read_vbr(sm6, 5); if (!vkd3d_array_reserve((void **)&block->abbrevs, &block->abbrev_capacity, block->abbrev_count + 1, sizeof(*block->abbrevs)) @@ -580,6 +628,17 @@ static enum vkd3d_result dxil_block_read(struct dxil_block *parent, struct sm6_p return VKD3D_ERROR_INVALID_SHADER; } +static unsigned int sm6_parser_compute_global_abbrev_count_for_block_id(struct sm6_parser *sm6, + unsigned int block_id) +{ + unsigned int i, count; + + for (i = 0, count = 0; i < sm6->abbrev_count; ++i) + count += sm6->abbrevs[i]->block_id == block_id; + + return count; +} + static void dxil_block_destroy(struct dxil_block *block) { unsigned int i; @@ -604,8 +663,8 @@ static void dxil_block_destroy(struct dxil_block *block) static enum vkd3d_result dxil_block_init(struct dxil_block *block, const struct dxil_block *parent, struct sm6_parser *sm6) { + unsigned int i, abbrev_count = 0; enum vkd3d_result ret; - unsigned int i; block->parent = parent; block->level = parent ? parent->level + 1 : 0; @@ -618,10 +677,26 @@ static enum vkd3d_result dxil_block_init(struct dxil_block *block, const struct if (sm6->p.failed) return VKD3D_ERROR_INVALID_SHADER; + if ((block->abbrev_count = sm6_parser_compute_global_abbrev_count_for_block_id(sm6, block->id))) + { + if (!vkd3d_array_reserve((void **)&block->abbrevs, &block->abbrev_capacity, + block->abbrev_count, sizeof(*block->abbrevs))) + { + ERR("Failed to allocate block abbreviations.\n"); + return VKD3D_ERROR_OUT_OF_MEMORY; + } + + for (i = 0; i < sm6->abbrev_count; ++i) + if (sm6->abbrevs[i]->block_id == block->id) + block->abbrevs[abbrev_count++] = &sm6->abbrevs[i]->abbrev; + + assert(abbrev_count == block->abbrev_count); + } + if ((ret = dxil_block_read(block, sm6)) < 0) dxil_block_destroy(block); - for (i = 0; i < block->abbrev_count; ++i) + for (i = abbrev_count; i < block->abbrev_count; ++i) vkd3d_free(block->abbrevs[i]); vkd3d_free(block->abbrevs); block->abbrevs = NULL; @@ -630,11 +705,21 @@ static enum vkd3d_result dxil_block_init(struct dxil_block *block, const struct return ret; } +static void dxil_global_abbrevs_cleanup(struct dxil_global_abbrev **abbrevs, unsigned int count) +{ + unsigned int i; + + for (i = 0; i < count; ++i) + vkd3d_free(abbrevs[i]); + vkd3d_free(abbrevs); +} + static void sm6_parser_destroy(struct vkd3d_shader_parser *parser) { struct sm6_parser *sm6 = sm6_parser(parser); dxil_block_destroy(&sm6->root_block); + dxil_global_abbrevs_cleanup(sm6->abbrevs, sm6->abbrev_count); shader_instruction_array_destroy(&parser->instructions); free_shader_desc(&parser->shader_desc); vkd3d_free(sm6); @@ -724,6 +809,7 @@ static enum vkd3d_result sm6_parser_init(struct sm6_parser *sm6, const uint32_t return VKD3D_ERROR_INVALID_SHADER; } + /* Estimate instruction count to avoid reallocation in most shaders. */ count = max(token_count, 400) - 400; vkd3d_shader_parser_init(&sm6->p, message_context, source_name, &version, &sm6_parser_ops, (count + (count >> 2)) / 2u + 10); @@ -744,6 +830,10 @@ static enum vkd3d_result sm6_parser_init(struct sm6_parser *sm6, const uint32_t return ret; } + dxil_global_abbrevs_cleanup(sm6->abbrevs, sm6->abbrev_count); + sm6->abbrevs = NULL; + sm6->abbrev_count = 0; + length = sm6->ptr - sm6->start - block->start; if (length != block->length) { -- GitLab https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/44
As is often the case in LLVM the source code shows where BLOCKINFO can occur, and which blocks may contain children. -- https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/44#note_33355
From patch 1/5: ```diff @@ -1926,6 +1926,12 @@ enum vkd3d_result vkd3d_dxbc_binary_to_text(const struct vkd3d_shader_instructio shader_get_type_prefix(shader_version->type), shader_version->major, shader_version->minor, compiler.colours.reset);
+ if (shader_version->major >= 6) + { + FIXME("Shader model 6 tracing is not implemented yet.\n"); + return VKD3D_ERROR_NOT_IMPLEMENTED; + } + indent = 0; for (i = 0; i < instructions->count; ++i) { ```
Do we (still) need this? I think tracing DXIL should largely work now.
```diff + if (byte_code[4] < 16 || byte_code[4] >= byte_code_size) + { + WARN("Invalid bitcode chunk offset %#x (data size %zu).\n", byte_code[4], byte_code_size); + vkd3d_shader_error(message_context, &location, VKD3D_SHADER_ERROR_DXIL_INVALID_CHUNK_OFFSET, + "DXIL bitcode chunk has invalid offset %#x (data size %#zx).", byte_code[4], byte_code_size); + return VKD3D_ERROR_INVALID_SHADER; + } + if (byte_code[4] + byte_code[5] > byte_code_size) + { + WARN("Invalid bitcode chunk size %#x (data size %zu, chunk offset %#x).\n", + byte_code[5], byte_code_size, byte_code[4]); + vkd3d_shader_error(message_context, &location, VKD3D_SHADER_ERROR_DXIL_INVALID_CHUNK_SIZE, + "DXIL bitcode chunk has invalid size %#x (data size %#zx, chunk offset %#x).", + byte_code[5], byte_code_size, byte_code[4]); + return VKD3D_ERROR_INVALID_SHADER; + } ```
"byte_code[4] + byte_code[5]" can potentially overflow. Since we've already verified that "byte_code[4]" is smaller than "byte_code_size", we can write the condition above as "if (byte_code[5] > byte_code_size - byte_code[4])" instead. Even better would be to introduce actual variable names instead of using magic array offsets.
```diff + sm6->start = (const uint32_t *)((const char*)&byte_code[2] + byte_code[4]); + if (sm6->start[0] != BITCODE_MAGIC) + WARN("Unrecognised magic number 0x%08x.\n", sm6->start[0]); + + sm6->end = &sm6->start[(byte_code[5] + sizeof(*sm6->start) - 1) / sizeof(*sm6->start)]; + + if ((version.type = version_token >> 16) >= VKD3D_SHADER_TYPE_COUNT) + FIXME("Unrecognised shader type %#x.\n", version.type); ```
I imagine we might as well use vkd3d_shader_error() for these as well.
```diff + if ((ret = dxil_block_init(block, NULL, sm6)) < 0) + { + if (ret == VKD3D_ERROR_OUT_OF_MEMORY) + vkd3d_shader_error(message_context, &location, VKD3D_SHADER_ERROR_DXIL_OUT_OF_MEMORY, + "Out of memory parsing DXIL bitcode chunk."); + else if (ret == VKD3D_ERROR_INVALID_SHADER) + vkd3d_shader_error(message_context, &location, VKD3D_SHADER_ERROR_DXIL_INVALID_BITCODE, + "DXIL bitcode chunk has invalid bitcode."); + else + vkd3d_unreachable(); + return ret; + } ```
Is there any reason we couldn't generate the appropriate errors at the point they occur? (E.g., in dxil_block_read().)
```diff @@ -1088,6 +1090,12 @@ static int scan_with_parser(const struct vkd3d_shader_compile_info *compile_info vkd3d_shader_trace(&parser->instructions, &parser->shader_version); }
+ if (parser->shader_version.major >= 6) + { + FIXME("DXIL scanning is not implemented yet.\n"); + return VKD3D_ERROR_NOT_IMPLEMENTED; + } + for (i = 0; i < parser->instructions.count; ++i) { instruction = &parser->instructions.elements[i]; ```
Like further above, do we still need to explicitly handle that here?
```diff @@ -1109,7 +1117,10 @@ static int scan_dxbc(const struct vkd3d_shader_compile_info *compile_info, struct vkd3d_shader_parser *parser; int ret;
- if ((ret = vkd3d_shader_sm4_parser_create(compile_info, message_context, &parser)) < 0) + ret = (compile_info->source_type == VKD3D_SHADER_SOURCE_DXBC_DXIL) + ? vkd3d_shader_sm6_parser_create(compile_info, message_context, &parser) + : vkd3d_shader_sm4_parser_create(compile_info, message_context, &parser); + if (ret < 0) { WARN("Failed to initialise shader parser.\n"); return ret; @@ -1156,6 +1167,7 @@ int vkd3d_shader_scan(const struct vkd3d_shader_compile_info *compile_info, char
switch (compile_info->source_type) { + case VKD3D_SHADER_SOURCE_DXBC_DXIL: case VKD3D_SHADER_SOURCE_DXBC_TPF: ret = scan_dxbc(compile_info, &message_context); break; ```
I don't really see the point of handling DXIL in scan_dxbc(); the DXIL and TPF paths are about as different as the TPF and D3DBC paths. That largely applies to the change to compile_dxbc_tpf() as well, with the added point that that change makes to function misnamed.
From patch 2/5: ```diff + /** + * Autodetect VKD3D_SHADER_SOURCE_DXBC_TPF or VKD3D_SHADER_SOURCE_DXBC_DXIL. + * \since 1.9 + */ + VKD3D_SHADER_SOURCE_DXBC_AUTODETECT, ```
Do we need this? And perhaps more importantly, if we do need this, do we need this right away? For the purpose of vkd3d-compiler and vkd3d, we can just use vkd3d_shader_parse_dxbc() now, without making this part of the public API.
From patch 3/5: ```diff + {VKD3D_SHADER_SOURCE_DXBC_AUTODETECT, + "dxbc-auto", "Autodetect a 'TPF' or 'DXIL' shader embedded in a DXBC container.\n", + true, VKD3D_SHADER_TARGET_SPIRV_BINARY}, ```
Along a similar vein, does this add much over the existing autodetection we have in vkd3d-compiler? -- https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/44#note_33525
participants (3)
-
Conor McCarthy -
Conor McCarthy (＠cmccarthy)
-
Henri Verbeet (＠hverbeet)