[cairo] Cairo Buildbot

[Andrea Canciani]

There has been some interest into setting up automated builds to check
the status of cairo.
I started experimenting with buildbot (http://buildbot.net/) and
configured a server on
http://ranma42.dyndns.info:8010/ controlling two buildbots on my
laptop (one on MacOS X,
one on Ubuntu Lucid).
I have no experience in setting up this tools, so most of my
configuration is based on the
default options. I like this system, but I am open to changing it for
better ones.
One thing I think is quite important is to keep the client-side
configuration simple,
so that it is easy to setup additional bots.


Configuring a buildbot client is really easy.
You need to setup your development environment with everything you
need to checkout
cairo (git), configure it (from the repository, so at least autotools
and a shell), compile it
(make, gcc, depencencies) and whatever you would normally need (if you
want to test
pdf, poppler etc etc). Uh, you will obviously also need buildbot
I might be forgetting some tools, but as long as you can manually get
and compile cairo, your
dev env should be ok.

The minimum required to set up a client is:
buildbot create-slave <basedir> <master> <name> <passwd>
In my case, I ran (something like):
buildbot create-slave /Users/ranma42/Code/buildbot/nirvana-macosx
ranma42.dyndns.info:9989 nirvana-macosx my_secret_password

After doing this, you simply have to start your buildbot:
buildbot start <basedir>
In may case:
buildbot start /Users/ranma42/Code/buildbot/nirvana-macosx)

and (if the master knows about it), a connection will be established.

If you need special environment settings (LD_PRELOAD, DISPLAY, etc), I
think that the
best way to set them in environment you use to start your buildbot,
but it is not a requirement
since they can also be set by the master.

The only other configuration I did is set up
$basedir/info/{host,admin} like this:
==> admin <==
Andrea Canciani <ranma42 at gmail.com>
==> host <==
MacOS X Snow Leopard (10.6) running on a MacBook Pro

This way people can get some information about the buildbots and who
is their administrator
(probably in case they want to blame him/her because they are not online). See
http://ranma42.dyndns.info:8010/buildslaves/nirvana-macosx


Ok, so the client (aka slave) configuration is really simple. Then the
server is probably a little
harder. This is in fact true and the main reason is that it handles
all the logic of the buildbot
system (when do we start a build? what passes does the build consist
of? who will run them?)

The attachment contains a diff between the sample configuration and
what is currently running
on my server (ok, I changed the passwords, but it doesn't really matter).

The first change is the list of buildslaves (two buildslaves with
default options).
Buildslaves are the entities that actually run the builds.

The second change I did is to set up the schedulers. There are two
schedulers following branch
master and triggering full-* builds when the branch has been stable
(received no commits) for
30*60 seconds and fast-* builds when it has been stable for 5*60 seconds.
There are also other types of schedulers (for example to perform
nightly builds) which trigger
builds upon other events.

Then I set up the build factories. They are the sequence of passes to
be performed by the buildbots.
fFull is the slow-but-complete sequence performing a full build from
nothing to test:
1. checkout from git the revision you want to build and copy it to a
working directory
2. configure with default options (i.e. run "./autogen.sh")
3. Compile the library (run "make")
4. Compile the testsuite (run "make cairo-test-suite" in the test directory)
5. Test (run "make test") ignoring stderr and stdout (normally it
would log them, but I found out they
were too big, probably because of some problems with forking) and get
test/cairo-test-suite.log as
the logfile instead.

fFast is the fast sequence that should ideally catch commits that
break compilation as early as possible.
1. checkout the revision you want to build
2. build it in place (running "make")
This means that autotools are not run unless "make" decides so, that
object files are preserved so
compilation of small changes will be quite fast and other nice things like that.
It also means that unfortunately sometimes to fix the compilation you
have to manually run "./autogen.sh"
(from the client or from the server, it's the same) to get the
compilation working (this is of course needed
for the first compilation, but apparently autotools sometimes break so
you have to run it again).

The builders are then the associations between buildslaves and
factories. It is possible to have many
buildslaves in a single builder (which means that only one of them
will get to run the factory when the
builder is scheduled (improving availability reducing the average load
on the slaves)) and/or repeat
buildslaves in different builders (which obviously means that those
buildslaves can be used to perform
different types of builds).

The last few changes are to set up the authentication on the server
(on my server unauthenticated users
can only see things, to perform actions you have to authenticate) and
the hyperlinks that appear on the
pages.


Hidden but still doing something is a small script in the hooks
directory of the repo on my server that get
run when I pull. Ideally this should go on the fd.o repository and
should be run when somebody pushes
to it.


Comments, suggestions, security concerns (attaching a buildbot to an
untrusted master is a bad idea,
since it will (try to) run whatever the master says) etc are appreciated.

Andrea
-------------- next part --------------
A non-text attachment was scrubbed...
Name: my_master.diff
Type: application/octet-stream
Size: 4559 bytes
Desc: not available
URL: <http://lists.cairographics.org/archives/cairo/attachments/20100907/9fa7922b/attachment.obj>