31 Mar
2013
31 Mar
'13
10:45 a.m.
On 31 March 2013 11:58, Stefan Dösinger stefandoesinger@gmail.com wrote:
I was curious how you ended up with 8 digits. My thinking was that 32 bit floats have a 23 bit mantissa, and one digit in the printout encodes lb(10) bits - so you need ~6,9 digits to encode the mantissa, which rounds up to 7. Thinking further, %.6e should do the job as well, as there's one additional digit leading the decimal.
Is my thinking correct, or am I missing something?
It's been a while since I actually did the calculation, but IIRC you also need to take the implicit extra bit in the significand into account, so you'd end up with log(2^24) = ~7.2, which would give %.7e. But I just rounded up to the next "round" number.