[cairo] RFC: More accurate color conversion

[Wolfgang Draxinger]

On Sun, 06 Oct 2013 03:38:43 +0200
sandmann at cs.au.dk (Søren Sandmann) wrote:

> Wolfgang Draxinger <wdraxinger.maillist at draxit.de> writes:
> 
> > I'd like to throw in this little paper, discussing the problem of
> > floating point <-> integer range conversion at depth: 
> >
> > http://kaba.hilvi.org/homepage/programming/range/RangeConversion.pdf
> 
> In terms of that paper, f(x) is fixed to be x / 65535.0, 

I don't see where you pulled that particular 2**16 - 1 from. The paper
discusses range conversions [0, 1] <-> [0, 2**n - 1] which is the
typical problem when confronted with conversion between floating point
and integer.

> hopefully x == g(f(x)) for all x.

Not if you're converting between floating point and integers. That's
numerics 101. The numbers represented by floats are m * 2**p.
But that can be trivially rewritten as m << p. Floating point means
nothing else than floating shift when it comes to the binary
representation. Which means, that for any exponent p > 0 there are 2**p
different integers that map to the same float and that for any exponent
p < 0 there are 2**p floating point values that map to the same integer.

Your typical float has 23 bits for the mantissa and 8 bits for the
exponent. Which means that for integer ranges [-2**23 + 1, 2**23 - 1]
you can have a bijective mapping. Outside of that you need an exponent
=/=0, so good luck.

You can quickly test it with this program, which doesn't even do a
range conversion:

#include <stdio.h>
#include <limits.h>

#define RANGE (1<<25)

void float_int(float f, int const i)
{
	if( (int)f != i ) {
		fprintf(stderr, "%f =/= %d\n", f, i);
	}
}

int main(int const argc, char const * const argv[])
{
	int i;
	for( i = -RANGE + 1; i < RANGE - 1; i++ ) {
		float_int(i, i);
	}
	return 0;
}

Typical output

-33537332.000000 =/= -33537331
-33537328.000000 =/= -33537329
-33537328.000000 =/= -33537327
-33537324.000000 =/= -33537325
-33537324.000000 =/= -33537323
-33537320.000000 =/= -33537321
-33537320.000000 =/= -33537319
-33537316.000000 =/= -33537317
-33537316.000000 =/= -33537315
-33537312.000000 =/= -33537313
-33537312.000000 =/= -33537311
-33537308.000000 =/= -33537309
-33537308.000000 =/= -33537307
-33537304.000000 =/= -33537305
-33537304.000000 =/= -33537303
-33537300.000000 =/= -33537301
-33537300.000000 =/= -33537299
-33537296.000000 =/= -33537297
-33537296.000000 =/= -33537295
-33537292.000000 =/= -33537293
-33537292.000000 =/= -33537291
-33537288.000000 =/= -33537289
-33537288.000000 =/= -33537287
-33537284.000000 =/= -33537285
-33537284.000000 =/= -33537283
-33537280.000000 =/= -33537281
-33537280.000000 =/= -33537279
-33537276.000000 =/= -33537277
-33537276.000000 =/= -33537275
-33537272.000000 =/= -33537273
-33537272.000000 =/= -33537271
-33537268.000000 =/= -33537269
-33537268.000000 =/= -33537267
-33537264.000000 =/= -33537265

As you can see that each some integer values are "bordered" by +1, -1
of them, which however map to the same floating point value. Which
suggests that the least significant bit gets dropped. Which is exactly
what happens, the mantissa of those values is > 2**23 


Wolfgang