Re: [PATCH v2] d3dx9: Reduce rounding error in D3DXQuaternionToAxisAngle

[Matteo Bruni]

2017-11-09 17:34 GMT+01:00 Alex Henrie <alexhenrie24(a)gmail.com>:
> Signed-off-by: Alex Henrie <alexhenrie24(a)gmail.com>
> ---
> Fixes test failure on 64-bit Linux.
>
> v2: Change 2.0f to simply 2.0 to make it clear that truncation is done
> after multiplication.
> ---
>  dlls/d3dx9_36/math.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/dlls/d3dx9_36/math.c b/dlls/d3dx9_36/math.c
> index 8f5f5b2413..1b4edf82cf 100644
> --- a/dlls/d3dx9_36/math.c
> +++ b/dlls/d3dx9_36/math.c
> @@ -1651,7 +1651,7 @@ void WINAPI D3DXQuaternionToAxisAngle(const D3DXQUATERNION *pq, D3DXVECTOR3 *pax
>          paxis->z = pq->z;
>      }
>      if (pangle)
> -        *pangle = 2.0f * acosf(pq->w);
> +        *pangle = 2.0 * acos(pq->w); /* acosf has too much rounding error */
>  }
>
> /*_________________D3DXVec2_____________________*/
> --
> 2.14.2

I think just relaxing the test a tiny bit, allowing 1 ULP of error
margin, is preferable in this case.