Re: [PATCH vkd3d v2] vkd3d-shader/hlsl: Return the shape of the left operand when appropriate.

[Giovanni Mascellani]

This supersedes 217302. Also, 217464, from the same patch set, is revoked.

Thanks, Giovanni.

Il 19/10/21 09:32, Giovanni Mascellani ha scritto:
> The assumption about the size of matrices is not correct: it is
> legitimate to compose a matrix 2x2 with a vector of length 4, in which
> case it appears that the result has the shape of the first (leftmost)
> operand. Even for matrices 1xN or Nx1, the result is not always a
> vector: in general it has the shape of the first operand again.
> Signed-off-by: Giovanni Mascellani
>
> libs/vkd3d-shader/hlsl.y | 13 +------------
>   1 file changed, 1 insertion(+), 12 deletions(-)
>
> diff --git a/libs/vkd3d-shader/hlsl.y b/libs/vkd3d-shader/hlsl.y
> index bfe2bbb9..c42e372e 100644
> --- a/libs/vkd3d-shader/hlsl.y
> +++ b/libs/vkd3d-shader/hlsl.y
> @@ -971,18 +971,7 @@ static bool expr_common_shape(struct hlsl_ctx *ctx, struct hlsl_type *t1, struct
>       }
>       else
>       {
> -
>    /* Two vectors or a vector and a matrix (matrix must be 1xn or nx1) */
> -
>    unsigned int max_dim_1, max_dim_2;
> -
> -
>    max_dim_1 = max(t1->dimx, t1->dimy);
> -
>    max_dim_2 = max(t2->dimx, t2->dimy);
> -
>    if (t1->dimx * t1->dimy == t2->dimx * t2->dimy)
> -
>    {
> -
>        *type = HLSL_CLASS_VECTOR;
> -
>        *dimx = max(t1->dimx, t2->dimx);
> -
>        *dimy = 1;
> -
>    }
> -
>    else if (max_dim_1 <= max_dim_2)
> -
>    if (t1->dimx * t1->dimy <= t2->dimx * t2->dimy)
>     {
>         *type = t1->type;
>         *dimx = t1->dimx;