[cairo] Video Hackfest conclusions

[Siarhei Siamashka]

On Tuesday 24 November 2009, Benjamin Otte wrote:
> Hi,
>
> As you may be aware, we held a video hackfest last week in Barcelona.
> Developers met to discuss how best to improve GPU support for video
> applications. See http://gstreamer.freedesktop.org/wiki/VideoHackfest
> for more details. In particular, you might be interested in the notes
> some people took while the hackfest was going on. These can be found
> at http://gstreamer.freedesktop.org/wiki/VideoHackfest/Notes
>
> What follows is the results we arrived at after the hackfest. They are
> taken from http://gstreamer.freedesktop.org/wiki/VideoHackfest/Conclusions
> but certainly deserve wide circulation. We're interested in any
> comments (or questions) you might have about them, so please don't
> heistate to reply.
>
> If you do reply, please make sure to reduce the recipient list
> appropriately.

[...]

> == YUV in pixman ==
> It has been a goal for a while to make video a first-class citizen in
> the image stack. A concrete API proposal on how to integrate video
> into pixman was reached and reviewed.
> Links:
>  * http://gstreamer.freedesktop.org/wiki/VideoHackfest/YCbCr_Formats
>  * http://gstreamer.freedesktop.org/wiki/VideoHackfest/pixman
> Actions:
>  * GStreamer developers to provide a specification of important YUV formats
>  * Write implementation
>
> == YUV in Cairo ==
> To make it easy for applications to use these different formats, the
> Cairo API needs to be extended to allow them. A proposal does exist
> and has been reviewed, details need to be finalized.
> Links:
>  * http://gstreamer.freedesktop.org/wiki/VideoHackfest/Cairo
> Actions:
>  * Finalize proposal
>  * Write implementation

I would like to have some clarification regarding forcing BILINEAR filter for
nonscaled YCbCr images.

What happens if some YCbCr image is used as both source and destination if we
want to draw some RGB stuff on top of video frame? In this case the operation
will be split into fetching YCbCr data into temporary scanline buffer (and
converting it to a8r8g8b8 format), doing the operation and writing data back,
converting it to YCbCr. For this kind of operation, any distortions in the
original image are probaby not desired. A test can involve trying to do OVER
compositing operation with fully transparent RGB source and YCbCr as
the destination. Optimistically, the destination image should not change at
all. But in reality, some of the pixels have to be clamped to [0, 255] range
when converting to RGB. So the requirement can be relaxed to requiring
YCbCr -> RGB -> YCbCr roundtrip to only be allowed to modify the values which
are clamped during this conversion. It can be treated somewhat similar to
handling top 8 bits in x8r8g8b8 format. Some different YCbCr pixel values map
to the same a8r8g8b8 value, but so do x8r8g8b8 pixels.

Using bilinear filter on YCbCr -> RGB -> YCbCr roundtrip would introduce some
blur unless I'm mistaken. But this filter may still be a good idea when
generating a final RGB picture. BTW, introducing a dithering filter for doing
final conversion into r5g6b5 format may be a good idea too.

-- 
Best regards,
Siarhei Siamashka