-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi Henri,
While overhauling the patch to hardcode local constants in the shader code I came across the TRACEs in wined3d_device_set_{vs,ps}_consts_f, which print the floats with %.8e. You changed this from %f in 979e3744f7721bef9e88fd671dfd7591c9f85392.
I was curious how you ended up with 8 digits. My thinking was that 32 bit floats have a 23 bit mantissa, and one digit in the printout encodes lb(10) bits - so you need ~6,9 digits to encode the mantissa, which rounds up to 7. Thinking further, %.6e should do the job as well, as there's one additional digit leading the decimal.
Is my thinking correct, or am I missing something?
Cheers, Stefan
On 31 March 2013 11:58, Stefan Dösinger stefandoesinger@gmail.com wrote:
I was curious how you ended up with 8 digits. My thinking was that 32 bit floats have a 23 bit mantissa, and one digit in the printout encodes lb(10) bits - so you need ~6,9 digits to encode the mantissa, which rounds up to 7. Thinking further, %.6e should do the job as well, as there's one additional digit leading the decimal.
Is my thinking correct, or am I missing something?
It's been a while since I actually did the calculation, but IIRC you also need to take the implicit extra bit in the significand into account, so you'd end up with log(2^24) = ~7.2, which would give %.7e. But I just rounded up to the next "round" number.
-
Henri Verbeet -
Stefan Dösinger