On 14 September 2010 03:00, Chris Robinson chris.kcat@gmail.com wrote:
On Monday, September 13, 2010 12:55:43 pm Owen Rudge wrote:
- if (*v < -1.0f) d = -128;
- if (*v < 0.0f) d = *v * 128.0f;
- if (*v >= 0.0f) d = *v * 127.0f;
- if (*v > 1.0f) d = 127;
The way these checks are written out can cause problems, particularly with the first two because the if(*v < 0.0f) occurs after the if(*v < -1.0f) check, so it won't properly clamp values below -1.0. Something like this:
if (*v < -1.0f) d = -128; else if (*v < 0.0f) d = *v * 128.0f; else if (*v > 1.0f) d = 127; else if (*v >= 0.0f) d = *v * 127.0f;
should do fine (along with the 16- and 32-bit variants; the 24-bit one looks fine). You also don't need to keep the float as a pointer, though I imagine the optimizer can handle that.
Is that correct in the first place though? This will produce a non-uniform mapping for positive and negative values. IIRC the common way to do these kinds of mappings is "((x * (2^n - 1)) - 1) / 2", which would be "x * 127.5f - .5f" for an 8 bit signed format. The main disadvantage of that method is that integer 0 doesn't correspond exactly to floating-point 0.0f, but in practice few people seem to care. Of course it all depends on how the target format is actually defined.