[cairo] Vector screen capture with Cairo

[Peter Clifton]

On Wed, 2007-04-04 at 14:07 -0700, Carl Worth wrote:
> On Wed, 04 Apr 2007 21:23:26 +0100, Peter Clifton wrote:
> > > I want to get a vector-based a window capture of a program for
> > inclusion
> > > in user-documentation. This would produce a nice scalable, (and
> > > hopefully memory efficient) way to embed the window capture.
> 
> Hi Peter,
> 
> I think this is a really interesting thing that you are looking into,
> and I'm looking forward to hearing more from you as you make progress.
> 
> > Each X11 drawable the app wants to paint has a cairo surface created for
> > it, and subsequently a graphics context.
> 
> I'm curious how many of these you are seeing. My understanding is that historically
> GTK+ used a separate X11 Window for every widget, and so each
> application window would have a fairly deep corresponding hierarchy of
> X11 Windows. But I also understand that in recent versions of GTK+
> that it uses X in a much more flat style, with several layers of GTK+
> widget hierarchy drawing to the same X11 Window.

I'll be able to comment more on that once I've done some simple test
apps. I've been firing up "any old" GTK app for now, and seeing a
partial trace of some cairo functions which I've instrumented with
"printf" in the shim layer.

It does seem to create and destroy these farily often - e.g. mousing
over a toolbar, I'm seeing things like:

cairo_xlib_surface_create( 0x8124770, 60818447, 0x8128cb8, 900, 650 ) =
0x833a648
cairo_create( 0x833a648 ) = 0x83382e8
cairo_destroy( 0x83382e8 )
cairo_surface_destroy( 0x833a648 ).

By the size of the drawable, I can tell that this is the GdkDrawing area
in the program. It uses gdk calls to paint its self. (The toolbar raises
a tool-tip which partly over-laps this, so I'd expect some expose events
to occur on it).

> > Can I have multiple cairo contexts painting to different areas of
> > my .pdf surface concurrently?
> 
> What do you mean by "concurrently" exactly? You can definitely create
> multiple cairo contexts that all target the same surface. And for each
> context, you can setup a different translation matrix and even a
> different clip so that each is painting to a different area.
> 
> As for concurrence, cairo would not be happy if you were modifying the
> contents of the same surface, (or any other user-created cairo object
> for that matter), from multiple threads concurrently. But otherwise,
> you should be fine.

Ok - Multiple cairo drawing contexts, painting on the one surface. That
is fine - kind of what I expected, but I couldn't see it explicitly in
the docs.

I didn't  mean threaded - I think the last time I used a thread was many
years ago!

> > Can I make a surface backed pattern which can be "painted" onto the .pdf
> > - or will this rasterize the graphics?
> 
> Yes, you can create a pattern from one PDF surface,
> (cairo_pattern_create_for_surface, or even implicitly with just
> cairo_set_source_surface). And you can then paint with this pattern to
> another PDF surface.
> 
> *BUT*, as you anticipated, this will currently trigger rasterization
> of the source pattern.
>
> This is certainly something that could be fixed in cairo, and it would
> be extremely useful, (and Kristian I think even had some functional
> support for this kind of thing in an early version of cairo's PDF
> surface).
>
> If you want to look into the issue, the function of interest is:
> 
> 	_cairo_pdf_surface_emit_surface_pattern
> 
> Notice the comment that reads:
> 
> 	/* XXX: Should do something clever here for PDF source surfaces ? */
> 
> and notice that the next operation is to call
> _cairo_surface_acquire_source_image on the source pattern's surface,
> (which explicitly asks for a rasterized version of the source
> surface). So, instead, emitting a vector version of the source
> surface, (which should be straightforward as cairo already does know
> how to emit vector PDF after all), is what should happen there---to
> form a native PDF pattern rather than an image pattern in the
> resulting PDF.
> 
> Also, one thing you should be aware of if you're going to get into
> this, is that what I've been calling a "PDF source surface" won't
> actually be a PDF surface inside of cairo, but will be a "meta
> surface" that can be replayed as native PDF, (this is exactly what's
> already happening under the hood for all PDF surfaces anyway).

I've probably not got enough time to dig deeply into that, but meta
surfaces do sound like they could be key to this kind of problem, the
still vector based "final" product of a series of drawing operations?

> Oh, and to create the little intermediate, sub-surfaces that you want,
> the best thing is to just call cairo_surface_create_similar for each,
> passing the final PDF surface as the template, (since you don't want
> to actually create intermediate PDF surfaces, since that requires a
> destination file name).
> 
> And here, there's another current limitation which is that there
> currently isn't a _cairo_pdf_create_similar function. This means that
> your "similar" surface will be an image surface from the beginning
> instead of a meta-surface like you want to get into the code paths I
> described above.
> 
> I know that Adrian Johnson has already started looking into making
> create_similar return a meta-surface for the PDF backend, (note that
> cairo already returns a meta-surface for the SVG backend, for
> example). And he may also be planning on doing all the work that I'm
> suggesting above, (quite likely in fact, as getting a meta-surface fro
> create_similar isn't really useful without it).
> 
> So I'll let him comment on his progress as far as that goes.
> 
> > I'm realising this is far from "easy", but it still looks possible. Any
> > further comments?
> 
> Don't forget that if it were all easy, it wouldn't be nearly so much
> fun!
> 
> Also, as has been mentioned earlier in the thread, GTK+ doesn't
> currently draw everything with cairo---many GDK functions still bypass
> cairo. So you'll probably need to address that as well to get your
> code to work reliably.

I believe it does gdk calls though - and it looks fairly easily replace
the Xlib primitives with cairo ones.

> And finally, you talked about starting up the program to be
> screenshotted and informing it to arrange itself as desired. A much
> more interesting approach would involve putting an unmodified
> application into the desired state through whatever means, then
> signalling it in some fashion, and having GTK+ respond to the signal
> by setting up the PDF-targeting cairo context[*] and forcing a full
> expose to the application.
> 
> -Carl
> 
> [*] One annoyance here is that GTK+ doesn't actually create and pass a
> context to the application, but expects the application to create the
> cairo context when it's exposed. But it does do that with
> gdk_cairo_create, so you do at least have a place to hook in and
> ensure the application gets a context targeting the cairo surface that
> you really want. But, presumably you've already figured this much out
> anyway.

That bit isn't so bad.

Right now, I'm writing some hooks to keep track of the cairo objects the
application uses, and to attempt duplicating some of these operations
into a .pdf surface.

I'll keep the list appraised if I make any useful progress.

Regards,

-- 
Peter Clifton

Electrical Engineering Division,
Engineering Department,
University of Cambridge,
9, JJ Thomson Avenue,
Cambridge
CB3 0FA

Tel: +44 (0)7729 980173 - (No signal in the lab!)