Am Sonntag, den 13.07.2008, 01:14 +0200 schrieb Michael Karcher:
Am Samstag, den 12.07.2008, 23:56 +0100 schrieb Reece Dunn: If you are brave enough to really find out what happens (might be different depending on Windows version, instruction sets available and things like that), also try the identity matrix scaled by 2^66 and 2^(-66). The square of 2^66 overflows a 32 bit floating point number.
Sorry, forgot denormals. The square of 2^(-66) *is* exactly represenatable as float! Use 2^(-80) to be sure that the square underflows.
Regards, Michael Karcher