Float printf precision

[Stefan Dösinger]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Henri,

While overhauling  the patch to hardcode local constants in the shader
code I came across the TRACEs in wined3d_device_set_{vs,ps}_consts_f,
which print the floats with %.8e. You changed this from %f in
979e3744f7721bef9e88fd671dfd7591c9f85392.

I was curious how you ended up with 8 digits. My thinking was that 32
bit floats have a 23 bit mantissa, and one digit in the printout
encodes lb(10) bits - so you need ~6,9 digits to encode the mantissa,
which rounds up to 7. Thinking further, %.6e should do the job as
well, as there's one additional digit leading the decimal.

Is my thinking correct, or am I missing something?

Cheers,
Stefan
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJRWAiyAAoJEN0/YqbEcdMw+4AQAIW57gxqJJcPEKzS35K4R++Q
aMy1rma7JV4/cUmRIqkmOROc6Y3T2bz7x3e12mNpLVvI9YUBdaZKsofuBJk9QCtL
R6nWL266dLc3aRdIFazwjxK6GmohqjxoLidpCAPSXGkekjw48DqhA+5fdXO2YYm/
2g1Fhyy/cpi6BN2AUh760YNy3ojk/x9eH25oTmifFxCUbWlhYNGsIVt8OTTW9YBE
RnQyyoO2lPjNi5mlNIqoSkkYriG5m97nS8bSMXvu9BjKW8rSmmErLGPf2zDL7Big
wdibKxbXaSSx74upJaOTLjoOx04sqoi4SuPnX/55ftFKj7s2SeoFJaOUwgdNJoN+
UxKcd0HzfaXBi/VikYvtfMd0+SLA8wFIVjrJsWeBwgNU9cEAA3OiRFPWyizZCPqu
4UQOSnxuK6LZ7Ke58ikcG6JqMVTye8iTeEmr6r3NhkRdmCKrFbVGmJ1POb8Q26Hu
q2yQBHq2L5CZ0Vx7fruvqq9Ap8auI/l5B6tneDM/RUdvpWxQTJneVRpSqIE50ZmW
U3XH2cx4eLaEGJnSnHBgxnqgSAmcVTJEkFHtd8tCiqsPcnkw2g3Ng3gQZhjkDJ45
ZxvXNxVUl3B899Y1CwFd3FVJhKQq2jFS0NQvNQYZXhx0djZ2ok8rvtmoJzTaNN1s
SMfTtZDHxHFtluASd6zJ
=7CAL
-----END PGP SIGNATURE-----

[Henri Verbeet]

On 31 March 2013 11:58, Stefan Dösinger <stefandoesinger(a)gmail.com> wrote:
> I was curious how you ended up with 8 digits. My thinking was that 32
> bit floats have a 23 bit mantissa, and one digit in the printout
> encodes lb(10) bits - so you need ~6,9 digits to encode the mantissa,
> which rounds up to 7. Thinking further, %.6e should do the job as
> well, as there's one additional digit leading the decimal.
>
> Is my thinking correct, or am I missing something?

It's been a while since I actually did the calculation, but IIRC you
also need to take the implicit extra bit in the significand into
account, so you'd end up with log(2^24) = ~7.2, which would give %.7e.
But I just rounded up to the next "round" number.