[cairo] PDF memory usage?

[Adrian Johnson]

Ian Britten wrote:
> Hi all,
> I'm researching some possible problems/solutions trying to
> export our data to PDF via Cairo, and wondered if anyone could
> fill in some background information for me?
> 
> - Does the PDF surface keep all data in memory until the final
>    finish/flush is called?

Data is kept in memory for the current page. When show_page() is called 
the data is written to the PDF file and freed from memory.

> 
> - If yes, is there any way to progressively write out part/all
>    of the data, mostly to free up the memory?
> 
> We produce large maps, which typically contain a mixture of
> both vector data, and high-resolution image data.  I'm trying
> to get this data written out to a PDF, while preserving the
> vector data as vector in the PDF.  However, I'm concerned that
> Cairo is going to try to carry the large image(s) in memory
> while the PDF is being created, and may exhaust the memory.
> 
> For example, a 32-bit image that completely fills an A0 sheet
> at 1200dpi would need over 8G of memory.  Now, worst case,
> imagine several overlapping/transparent images, plus a slew
> of vector data - Phew!
> [ No, I have no idea what the users/customers expect to do
> with files like these - I'm just trying to create them... ]

PDF has a file size limit of about 10GB due to the use of a 10 digit 
number for the byte offset of each object. Cairo is currently limited to 
2GB on 32-bit architectures, and 10GB on 64-bit. I would not be 
surprised if many PDF consumers do not handle file sizes greater than 
the 2/4GB limit of 32-bit signed/unsigned integers.

> The typical approach for dealing with large output like this
> seems to be to try and chunk/tile the data.  However, with
> the target being PDF, I'm not sure if this is possible (And
> whether the Cairo interfaces support it).  My basic PS
> knowledges makes me think it could/should be possible to just
> keep writing to a PDF file and let the viewer worry about
> rendering the final data, but I'm not certain...
> So, I figured I'd ask some of the clever people here!  :)

Cairo does not support this. My recommendation is to use a 64-bit 
machine with enough memory for the job.

> Many thanks for any info, suggestions, etc!
> Ian
> _______________________________________________
> cairo mailing list
> cairo at cairographics.org
> http://lists.cairographics.org/mailman/listinfo/cairo