Re: [PATCH v4 0/1] MR203: vkd3d-shader/hlsl: Fold constant integral abs().
Francisco Casas (@fcasas) commented about libs/vkd3d-shader/hlsl_constant_ops.c:
+ { + switch (type) + { + case HLSL_TYPE_FLOAT: + case HLSL_TYPE_HALF: + dst->value.u[k].f = fabsf(src->value.u[k].f); + break; + + case HLSL_TYPE_DOUBLE: + dst->value.u[k].d = fabs(src->value.u[k].d); + break; + + case HLSL_TYPE_INT: + /* abs(INT_MIN) is undefined, leave this expression alone */ + if (src->value.u[k].i == INT_MIN) + return false; This won't get the absolute value of components that come after the component that is INT_MIN. Consider an int vector such as `(-1, INT_MIN, -1, -1)`; This would end up as `(1, INT_MIN, -1, -1)`.
I think it would be better to guard the assignment: ```c case HLSL_TYPE_INT: /* abs(INT_MIN) is undefined, leave this expression alone */ if (src->value.u[k].i != INT_MIN) dst->value.u[k].i = abs(src->value.u[k].i); break; ``` -- https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/203#note_33133
participants (1)
-
Francisco Casas (@fcasas)