[cairo] API proposal: Damage Tracking

[Soeren Sandmann]

Behdad Esfahbod <behdad at behdad.org> writes:

> > Another thing we may want to track:
> > 
> >     - Keep track of "solid" areas that are guaranteed to be completely
> >       covered with an alpha=1 source, and therefore obscure the
> >       underlying pixels.
> 
> This one is more delicate, but sure, has its uses.  But then wouldn't you also
> need another request for "the rest"?  Or you want to compute the rest using
> region calculus?

Yeah, region calculus was what I had in mind. Ie., during drawing the
widget, you track both damage (D), and solid-damage (SD). When the
widget is moved somewhere else, create translated versions (TD and
TSD) of the regions, invalidate (TD + D) - TSD, then use
XCopyArea() to copy SD to TD. So the "rest" (D - SD) is not really all
that interesting by itself.

However, this only works if you know that the SD area has not been
painted over by someone else, so I'll need to track that as
well. Since I don't know how I'll do that yet, I don't need the
solid-damage tracking right now. But I most likely will at some point.

> > It seems that the things you might want to track are all orthogonal:
> > 
> >     BOUNDING_BOX   /* Track bounding box */
> >     UDPATES        /* Track region covering updates */
> >     HIT_MASK       /* Track input region */
> >     SOLID          /* Track solid painting */
> 
> How would we do hit masking?  Based on alpha?  May make sense to add an api
> like what Keith added to Xrender, that takes a min/max alpha that is
> considered hit.

An alternative might be to specify it as "the pixels
for which cairo_in_fill/stroke() would have returned true, had you
called them instead of the actual fill/strokes you did".

But I'm fine with alpha-thresholds as well. I'd also be fine with
simply saying more than 50% is a hit.

> > Allowing the user to pass in the region solves both the hierarchy
> > problem
> 
> What's the hierarchy problem?

I forgot what I meant by that exactly, but it had to do with the
original proposal of having begin/end_damage() that would be tied to
the save/restore hierarchy. Possibly something about how to track
damage across unbalanced save/restore calls.

> > and allows the application to keep one update region around
> > and have it updated several times.
> 
> Not sure what that would be used for.

Just that it's very convenient to simply take the application's update
region and have cairo update it, rather than having to create new
regions and union them onto an existing one.

> > This is similar in spirit to the X11 Damage extension.
> 
> I suggest something like this instead:
>
[...]
> 
> This is essentially exactly what you propose, with cairo_tracking_t being the
> opaque cairo_region_t.  I like how this way of doing it obviates the need for
> a separate _BOUNDING_BOX mode, as that's simply _extents on UPDATES.  However,
> it will be wasteful as it will keep track of the full region internally.  One
> day we may add _copy_path() too.  Maybe we can change the tracker constructor
> to tell it what kind of formats we're interested in.  Options can be:
> 
>   bounding-box
>   rectangle-list
>   path
>   surface A1
>   surface A8
>   best
> 
> If you choose path, you can later copy_path and get the path for all
> operations.  This will do what _cairo_meta_surface_get_path does now.  If you
> set surface-A8, you can later copy_surface and get a A8 mask of all the
> updates, etc.  If you use best, it will automatically use the most compact
> format needed to fully represent the update areas, like what the clipping code
> does currently (path clipping, mask clipping, ...).

I think this API makes a lot of sense if we are absolutely not going
to have public regions. But I think it would be a mistake not to have
them, because

        - Any serious consumer of cairo is going to need regions

        - cairo itself needs regions

so if cairo doesn't expose them, the code will have to be
duplicated. And empirically, duplicating region code is a quite common
thing to do. If we can avoid it the future, I think that would be a
win.

With the tracker API, for what I need, I would have to first cut and
paste the region code, and then to actually track damage:

        - Create a new tracker object
      
        - Attach tracker object to surface

        - Draw things (optionally with updates turned off)

        - Copy floating point rectangles out of tracker object

        - Detach tracker object

        - Destroy tracker object

        - Convert floating point rectangles to integer rectangles

        - Create new temporary region from those integer rectangles

        - Free floating point rectangles
        
        - Free integer rectangles

        - Union temporary region with existing update region

        - Delete temporary region

Much of this could be encapsulated in a helper function, but it's
still inefficient and requires duplication of non-trivial code. With
regions, the same task would amount to:

        - Attach update region to surface

        - Draw things (optionally with updates turned off)

        - Detach update region from surface

Having regions would also allow something like a cairo_clip_region()
which would avoid a similar gratuitously inefficient conversion:

        - Get integer rectangles from app_region_t

        - Convert integer rectangles to double rectangles

        - Add rectangles to path

        - cairo tessellates rectangles to traps

        - cairo discovers traps are actually just region

        - cairo converts traps to cairo_region_t

So I definitely hope we can have public regions and use them for
tracking.

> > A separate 
> > 
> >     cairo_set_updates (cairo_t *cr, cairo_bool_t updates); 
> > 
> > would be required so that you could get the regions without actually
> > painting anything.
> 

[...]

> Consider this example: you have a canvas implementation, and I'm implementing
> a thumbnailing canvas item.  The thumbnailing canvas item takes another item
> (which can be a hierarchy of multiple items itself), measures it, then sets a
> cairo_scale() and draws it as a 100x100 box.
> 
> Then, your canvas disables updates to get hit tracking bounds/mask/etc, then
> calls into my item's _draw.  My _draw method itself then disables updates and
> calls into _draw method of the item it's thumbnailing, to get its bounding box
> such that it can compute the scale factor, then scales, enables updates, and
> calls the other item's _draw.
> 
> Now, in the above situation, the hit-tracker that your canvas installs should
> not see the drawing operations performed by the other item, that I was using
> for bounding box calculation.

That's a good point; I hadn't thought of that.

> Not sure how to fix this.  Maybe when we disable updates, any tracker
> installed before this point should not receive events anymore, until the
> matching enable.

That seems right to me. It can even be documented in a fairly
straightforward way: "Disabling updates also disables existing
trackers."

I can't think of any situation where this is not what you want.


Thanks,
Soren