[cairo-commit] papers/opengl_freenix04 opengl_freenix04.tex,NONE,1.1

[Carl Worth]

Committed by: cworth

Update of /cvs/cairo/papers/opengl_freenix04
In directory pdx:/tmp/cvs-serv14842

Added Files:
	opengl_freenix04.tex 
Log Message:
Initial commit of paper as sent by David and Peter

--- NEW FILE: opengl_freenix04.tex ---
\documentclass[twocolumn,10pt]{paper}

\usepackage{paper}
\usepackage{url}        
\usepackage{isolatin1}  

\ifpdf
\usepackage[pdftex]{epsfig}
\else
\usepackage{epsfig}
\fi

\begin{document}

  \title{Hardware Accelerated Image Compositing using OpenGL}

%  \docstatus{Submitted to USENIX Conference 2004}

  \author{
    \authname{Peter Nilsson}
    \authaddr{Department of Computing Science}
    \authaddr{Umeå University, Sweden}
    \authurl{\url{c99pnn at cs.umu.se}}
    \and
    \authname{David Reveman}
    \authaddr{Department of Computing Science}
    \authaddr{Umeå University, Sweden}
    \authurl{\url{c99drn at cs.umu.se}}
  }

  \maketitle


  \section{Introduction}
  The X Render Extension (Render)~\cite{render:2000} has now been around for a 
  few years and has widely been accepted as the new rendering model for the X 
  Window System~\cite{x}. Some of the features that Render today supports are 
  alpha compositing, anti-aliasing, sub-pixel positioning, polygon rendering, 
  text rendering and image transformations. The core of Render is its image 
  compositing model which borrows fundamental notions from the Plan 9 window 
  system~\cite{pike:draw}. Render provides a unified rendering operation which 
  supports the Porter-Duff~\cite{porterduff:1984} style compositiing operators.
  All pixel manipulations are done through this operation. This provides for 
  a simple and consistent model throughout the rendering system. 

  Although Render can be accelerated by graphics hardware using XAA~\cite{xaa} 
  to some extent, it requires graphics driver developers to add support for this
  in each driver. It's probably fair to say that so far, the few drivers which 
  do support this only do so with modest results.

  In the past years modern 2D graphics applications like window systems 
  tend to use more demanding graphic features like alpha blending, image 
  transformations and anti-aliasing.
  Even with todays powerful computers these tasks constitute a heavy burden on 
  the CPU. To relief the CPU, modern window systems have developed 2D-graphics
  engines which utilize the high performance rendering capabilities inherent in 
  todays graphics hardware. In fact, much of the eye candy we have seen in these
  systems wouldn't even be feasible without hardware accelerated rendering.
  The one that has probably attracted the most attention is Apple's Quartz 
  Extreme~\cite{quartzextreme} compositing engine used in Mac OS X. Rumor also 
  says that Microsoft will add something similar to their upcoming product, at 
  this time called Longhorn~\cite{longhorn}.

  The fact that this is a field in which the X Window system and the open source
  community doesn't quite seem to keep up is what originally lead up to the 
  ideas behind this project. The main idea is to investigate the potential 
  usefulness of a 2D graphics library that can be 
  accelerated by graphics hardware to create a rendering environment similar 
  to those used in the other window systems mentioned above. How would one 
  go about realizing these ideas in the simplest and most efficient manner?

  What we need is a complete 2D graphics API that is accelerated by
  graphics hardware. Much of work has already been done as we are very 
  fortunate to have the Cairo library~\cite{cairo}, an open source 
  library which provides a PDF-like 2D graphics API. One 
  thing missing this far in Cairo is the ability to accelerate the rendering 
  process  with todays high performance graphics hardware. 
  The Cairo library which is a modern cross-platform 2D graphics API designed 
  for multiple output formats with Render semantics and provides a powerful 
  vector based drawing environment. So what we need is an output format for 
  Cairo that can be fully hardware accelerated. OpenGL is the most widely used 
  and supported graphics API available today, it has great support for hardware 
  acceleration and is very portable. 

  Is it possible to create an interface on top of OpenGL which would provide 
  the same consistent rendering model as Render? And can the interface be 
  implemented in such a way that it can take advantage of the OpenGL hardware 
  acceleration provided by modern graphics cards?

  The proposed paper will present results from
  extensive research to find and evaluate the possibilities of OpenGL
  hardware acceleration of operations needed for the creation of a 
  Render-like interface on top of OpenGL. This research is concluded with 
  the implementation of an OpenGL 2D compositing library. The semantics of the
  library is designed to precisely match the specification of the 
  X Render extension. Having the same semantics as Render allows for seamless 
  integration with the Cairo library that then will provide an attractive
  environment for developing new graphical applications. The paper will 
  describe the solutions we have chosen for the library and the problems that
  we have encountered in the development process. Furthermore the paper will 
  also include test and benchmarking results that points out the performance and
  accuracy of the library on various hardware. 

  The development of the 2D compositing library and the other implementation
  parts are made in an entirely open fashion, input from the open source 
  community are regarded.


  \section{Implementation Issues}
  As OpenGL layers are available for various platforms and systems, the library 
  is designed so that it isn't OpenGL layer specific. Different
  layers can be used by plugging them in to the core of the library through a
  virtualized backend. Each backend only needs to provide a few functions 
  which will allow for the core of the library to perform its rendering 
  operations unaware of the structure of the underlaying GL layer. Having a 
  virulized backend instead of just choosing a suitable layer at compile 
  time has the advantage of allowing the library to be compiled for use with
  multiple GL layers.

  As of now the GLX backend is the only backend implemented but on request
  backends for other GL layers will be added.

  For any usable 2D graphics API, offscreen drawing is required. This is 
  something GL has lacked efficient support for until recently. However as this 
  is a relatively new feature in the OpenGL world, it is not yet supported by 
  all hardware and all drivers. When support for offscreen drawing is missing 
  the program using the library will have to handle this on its own. In the 
  Cairo library for example, image buffers will be used instead when offscreen 
  drawing can not be performed by our library.

  All rendering operations are performed on and with so called surfaces, which 
  can be of type onscreen, offscreen or intermediate. Onscreen surfaces are 
  usually portions of the frame buffer. Offscreen surfaces are textures stored 
  in the graphic card's texture memory. Intermediate surfaces are only used 
  internally by the library and are offscreen areas that can be rendered to. It 
  is up to the backend to decide on the type of intermediate surface, however 
  pbuffers are most likely to be used, so is at least the case for the GLX 
  backend.

  The general composite operation is the fundamental rendering part of the 
  library. It takes two source surfaces and one destination
  surface, where the second of the two source surfaces is the optional 
  mask surface.
  If a mask surface is supplied, an intermediate surface
  is created and the first source is rendered on it using the SRC operator. 
  In the next step the mask surface is then blended on top of it with the IN 
  operator. 
  A texture is created from the resulting surface which can finally be 
  composited on the destination surface and we're all done. 
  Every time drawing is to be done onto an offscreen surface, an intermediate 
  needs to be created so that the drawing can be done to this and the result can
  then be copied back to the texture.

  Most parts of the Render specification has been implemented but there are
  still some issues that need to be addressed. The most obvious one being the 
  implementation of an anti-aliasing model. There are several ways to go, some 
  of them require specific hardware support and some do not. Part of the problem
  is that when used with Cairo, primitives are drawn in immediate mode, which 
  means that they are drawn in the order which the end application wishes. This 
  means some complications when drawing correctly anti-aliased polygons with 
  OpenGL in the old fashion way. The final paper will have a detailed 
  description of the model we have chosen and why we choose that model.


  \section{Results and Conclusion}
  Right now the library hasn't really been tested that much in real applications
  as it is relatively early in the development process. Some small test 
  utilities for Cairo have been ported to use the new GL backend though. The 
  results have been satisfying both with respect to performance and accuracy. 
  The figures shows a part of the output from the cairo-demo application, the 
  output image has been cut down only to save some space in this paper. 
  Figure 1 shows output from cairo-demo using xrender and figure 2 shows the 
  corresponding output using the GL backend for rendering.
  Off course these pictures cannot be expected to show any great details of the 
  result but it might give a ruff idea about the accuracy of the output. 
  Note that no anti-aliasing is applied to the image rendered by GL as it is not
  yet implemented in a stable fashion. Yet you can se that the image rendered by
  GL compares rather well to that rendered by xrender. 

  \begin{figure}[htbp]
    \begin{centering}
      \epsfig{file=cairo-demo-xrender.eps, width=3.1in}
      \small\itshape
      \caption{\small\itshape Output from cairo-demo when run with anti-aliased 
        xrender output.}
      \label{fig1}
    \end{centering}
  \end{figure}

  \begin{figure}[htbp]
    \begin{centering}
      \epsfig{file=cairo-demo-glc.eps, width=3.1in}
      \small\itshape
      \caption{\small\itshape Output from cairo-demo when run with aliased GL 
        output.}
      \label{fig2}
    \end{centering}
  \end{figure}

  A simple benchmark application was also written to compare the rendering 
  performance with Cairo, using the different output formats including the new 
  GL backend developed in this project. The benchmark application currently 
  contains two separate tests. The first one animates a semi transparent, stoked
  and filled Cairo path made up of several bezier curves that moves randomly 
  over a simple background image. This test shows the speed at which lots of 
  transparent trapezoids can be drawn using Cairo paths. 

  The second test scales a background image and translates an other semi 
  transparent image randomly across the screen, testing image compositing and 
  transformation performance.  

  On all the systems we currently have run these tests, performance has 
  increased multiple times when rendering with the new GL backend. 
  All these testing and benchmarking utilities can be downloaded from the 
  project web site (referenced in Section 7). The site also contains complete 
  screen shots of these applications in action. A lot more testing will be done 
  and the results will be presented in the final paper.  


  \section{Future Work}
  As of today the existing implementation of the library supports all the basic 
  functionality initially set up for the project. But still the software is 
  in an early stage of development and lots of work remain in making it stable 
  and optimized with regards to performance and accuracy. A deadline for the 
  project has been set up in March 2004. The aim is to have a useful library by 
  that time as well as complete documentation of the project. A presentation of 
  the work will be held at Umeå University at that time but development and 
  maintaining of the software will certainly stretch beyond that date. 

  Off course the future will demand new features from the library as it is an 
  area of continuous development. For example in the near future new functions
  for the creation and compositing of glyph sets probably need to be added for
  more efficient rendering of text.

  Special functions for more efficient rendering of gradients could also be
  added at some level.


  \section{Visions}
  The Render compositing model has one major weakness when used with OpenGL, and
  that is that all compositing operations that uses a mask surface requires 
  that OpenGL creates an intermediate surface and performs the compositing 
  in two steps. If an extension for GL would exist that allows for the 
  blending of two textures with each other and then onto a drawable in one 
  operation the Render compositing model would be much more efficient. 

  The X desktop seems to be going into a new era and Cairo is definitely the
  2D graphics API which will be used in tomorrows X applications. The 
  support for hardware accelareted surfaces in Cairo might then be of big 
  importance. Plans on the creation of a X server that will use OpenGL for
  all rendering is currently being made and our library or the work behind the
  library can hopefully be usable for this purpose. 


  \section{Acknowledgments}
  We would like to thank Carl Worth and Keith Packard and along with all of the 
  people involved in the development of Cairo for being helpful and encouraging.
  We would also like to thank our internal supervisor along with the staff at 
  the department of Computing Science at Umeå University for supporting us in 
  this project. Thereby approving the project for financial funding in terms of 
  study allowances. 


  \section{Availability}
  All source code related to this project is free software distributed under the
  MIT license.

  The source can be retrieved via anonymous access from the Cairo CVS 
  (\url{anoncvs at cvs.cairographics.org:/cvs/cairo}). Just check out the module 
  called libglc. There is also a simple web site that includes some information 
  about the project at \url{http://www.cs.umu.se/~c99pnn/thesis}. A patch for 
  the Cairo library along with some test and benchmark utilities can also be 
  found at this location.

  \bibliography{paper}
  \bibliographystyle{plain}

\end{document}