[cairo] rewriting libpixman

[Bill Spitzak]

Here's my take on what he is saying. Let's say we want to convert 
between 8 and 4 bit representations, which allows everything to be 
written in hex:

The floating point value for 0xXY in 8 bits is 0xXY/255 which is, when 
written in floating-point hex, 0x0.XYXYXYXYXY..... This is actually true 
even when 0xXY is 0xFF, as 1.0 can also be written 0x0.FFFFFFF....

The floating point value of 0xX in 4 bits is 0xX/31 which is written in 
hex as 0x0.XXXXXX....

Setting these equal it is clear that the correct conversion from 0xX in 
4 bits to an 8 bit value is to turn it into 0xXX.

The opposite conversion is not so clear, you can make an argument that 
0xXY should turn into 0xX+1 if Y is greater than 8 (ie round). However 
this has the unfortunate effect of messing up the 1.0 case. Even if you 
special-case that, the result is that all 23 numbers from 0xE9 to 0xFF 
turn into 0xF, while only 8 numbers from 0x00 to 0x08 turn into 0x0 and 
all other numbers get 16 numbers each. This results in the whole image 
getting brighter. If instead you do a floor, exactly 16 input numbers 
are converted to each output number, so there is no overall shift.

So by that argument the correct conversion from 0xXY in 8 bits to 4 bits 
is 0xX.

These arguments apply to convert between any number of bits and any 
other number, but it has to be written in binary to see it.

This has the huge advantage that it is symmetric (ie invert the image, 
convert, invert produces the same result as conversion). Also the 
majority of other software is doing exactly this conversion and it is 
probably important that Cairo matches. I know from personal experience 
that when people paint an overlay in Photoshop that blends "perfectly" 
there, but the edges appear when composited in your own software, as far 
as they are concerned your software is WRONG, and no amount of 
mathematical arguing will convince them otherwise.

James Cloos wrote:
>>>>>> "Jeff" == Jeff Muizelaar <jeff at infidigm.net> writes:
> 
> Jeff> I still don't follow this logic, specifically the last sentence.
> 
> Then I didn't explain it well enough....
> 
> I was actually investigating conversions between floats in [0.0,1.0]
> to and from ints in [0,2ⁿ), for any n ∈ ℤ⁺.
> 
> In that case, if you mupliply the float by 2ⁿ (saturating at 2ⁿ-1 if
> the float was ≥ 1.0) you get sane results.
> 
> Also, if you divide the ints by 2ⁿ-1.0 you get results which are
> evenly spaced between 0.0 and 1.0 and you generate a repeating
> fraction which fills the width of the float...