[PATCH vkd3d] vkd3d-shader/hlsl: swizzle types of srcs can now be specified on sm4_instruction

[Francisco Casas]

Some dxbc instructions require swizzle types that differ from what is
usual for the register type.

e.g.
gather4 requires a register with register type VKD3D_SM4_RT_SAMPLER but
a swizzle type VKD3D_SM4_SWIZZLE_VEC4 instead of the usual
VKD3D_SM4_SWIZZLE_NONE.

The new fields of sm4_instruction can be used to handle these
exceptions.

Also, they shouldn't change the current behaviour of the rest of
the code since they are currently initalized with zeroes everywhere a
sm4_instruction is initialized.

Signed-off-by: Francisco Casas 
---
 libs/vkd3d-shader/hlsl_sm4.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/libs/vkd3d-shader/hlsl_sm4.c b/libs/vkd3d-shader/hlsl_sm4.c
index 9d45e163..e148c821 100644
--- a/libs/vkd3d-shader/hlsl_sm4.c
+++ b/libs/vkd3d-shader/hlsl_sm4.c
@@ -747,6 +747,10 @@ struct sm4_instruction
     {
         struct sm4_register reg;
         unsigned int swizzle;
+        /* If use_custom_swizzle_type is true, then custom_swizzle_type will be used in the
+         * instruction, instead of the default swizzle type obtained from the register type. */
+        bool use_custom_swizzle_type;
+        enum vkd3d_sm4_swizzle_type custom_swizzle_type;
     } srcs[2];
     unsigned int src_count;

@@ -754,7 +758,7 @@ struct sm4_instruction
     unsigned int idx_count;
 };

-static unsigned int sm4_swizzle_type(enum vkd3d_sm4_register_type type)
+static unsigned int sm4_default_swizzle_type(enum vkd3d_sm4_register_type type)
 {
     switch (type)
     {
@@ -913,8 +917,10 @@ static void write_sm4_instruction(struct vkd3d_bytecode_buffer *buffer, const st

for (i = 0; i < instr->src_count; ++i)
     {
+        unsigned int swizzle_type = instr->srcs[i].use_custom_swizzle_type ?
+                instr->srcs[i].custom_swizzle_type : sm4_default_swizzle_type(instr->srcs[i].reg.type);
         token = sm4_encode_register(&instr->srcs[i].reg);
-        token |= sm4_swizzle_type(instr->srcs[i].reg.type) << VKD3D_SM4_SWIZZLE_TYPE_SHIFT;
+        token |= swizzle_type << VKD3D_SM4_SWIZZLE_TYPE_SHIFT;
         token |= instr->srcs[i].swizzle << VKD3D_SM4_SWIZZLE_SHIFT;
         if (instr->srcs[i].reg.mod)
             token |= VKD3D_SM4_EXTENDED_OPERAND;

[Zebediah Figura]

On 11/10/21 2:11 PM, Francisco Casas wrote:
> Some dxbc instructions require swizzle types that differ from what is
> usual for the register type.
>
> e.g.
> gather4 requires a register with register type VKD3D_SM4_RT_SAMPLER but
> a swizzle type VKD3D_SM4_SWIZZLE_VEC4 instead of the usual
> VKD3D_SM4_SWIZZLE_NONE.

You could look at it that way, but I would rather say that "some 
register types do not use a consistent swizzle type." I.e. there's not 
really a "usual" case.

This is particularly true upon noticing that gather4 isn't the only 
"exception". For instance, I believe that single-element writemasks 
should in general result in SCALAR rather than VEC4 for normal temps.

Along those lines, I'd rather see sm4_swizzle_type() just go away, and 
the swizzle type explicitly specified everywhere. Introducing helpers 
like sm4_source_from_node() and sm4_source_from_deref() would make this 
easier.

As an aside, patch subjects should generally be phrased in the 
imperative. E.g. in this case I'd say something like "Allow using a 
custom swizzle type for sm4 source registers", or, if you end up 
following my suggestion, "Explicitly specify the swizzle type for sm4 
source registers".
> The new fields of sm4_instruction can be used to handle these
> exceptions.
>
> Also, they shouldn't change the current behaviour of the rest of
> the code since they are currently initalized with zeroes everywhere a
> sm4_instruction is initialized.

I think these paragraphs are obvious enough that they don't need to be 
in a patch description.