Managing free software projects
Andrew.Tridgell@anu.edu.au
Abstract
Over the years I have had the pleasure of managing several free
software projects such as Samba, rsync and KnightCap. I have also
contributed to other projects such as Linux and Dosemu. The projects
involved contributors spread all over the world working on a single
set of quickly changing source code.
In this talk I will describe some of the problems encountered with
this sort of distributed development and the solutions I have found
that work well as well as those that don't. I will particularly talk
about a web based bug tracking tool called JitterBug that I have
recently developed to make the task a little easier. Other tools that
I will describe include cvs/ssh, anonymous cvs and the FAQ-omatic.
Although I will emphasise issues relating to free software development
I hope that some of the ideas will be of interest to commercial
software developers.
Introduction
Running a free software project is time very consuming but can also be
very rewarding. Free software projects typically have very fast
ongoing development schedules and often need to deal with issues
beyond those required by commercial software developers.
In this paper I will concentrate on giving references to some useful
software packages that make the job of running a free software project
a little easier. In my talk I will expand on other issues involved in
free software projects as well as giving demonstrations of the
packages mentioned here.
Source code control
As soon as you get more than one developer working on a software
project you start finding that a good source code control becomes
essential. Even with only one developer such a system can save you a
lot of time.
The essential features of a source code control system for free
software projects are that it is simple to use, flexible and able to
easily accomodate widely distributed developers. The best system to
date seems to be CVS. CVS is the de-facto standard source code control
system within the free software community and it has certainly proved
itself extremely valuable within the projects I have worked on. The
particular features that I have found useful are:
- remote access. This is really important but only supported by a
few systems. By combining CVS with ssh any of the package developers
can work from anywhere in the world.
- optimistic (non-locked) operation. Source code control systems
tend to be optimistic or pessimistic. Optimistic systems allow
multiple developers to work on the same piece of source code at the
same time then semi-automatically merges the result. This fits in well
with the 'casual developer' that typifies free software programmers.
- branching. The ability to define branches and merge between them
is very important for the often speculative development that happens
in free software projects. In Samba each major release gets a branch
which can be updated independently of new developments, allowing
simple patch releases. We also use branches for radical new
developments (such as the addition of opportunistic locking) so that
these radical changes don't hold up the other developers while they
are being coded.
This only touches the surface of what CVS can do. The basic message is
that if you are a free software developer then you should really
consider storing your sources in CVS.
More information on CVS can be obtained from http://www.cyclic.com/.
Distribution
Everyone needs to distribute their free software! For small packages a
common method is to upload a .tar.gz file to a incoming directory on
major ftp archives such as sunsite.unc.edu. If you want to go further
than this then you will need to organise your own distribution. There
are a number of distribution channels that you could setup.
- anonymous ftp. This is the main way that free software packages
are distributed. Setting up a anonymous ftp site isn't hard and gives
you direct control over what is available and updates.
- anonymous cvs. This is becoming far more common for free software
packages. I now distribute all my source code in this way. The big
advantage is that users can get the very latest updates immediately by
just typing "cvs update" in their source directory. This gets rid of the
vexing question of when to release an update and also means that it is
trivial for users to maintain their own local changes without having
to reedit the code every time they download. I have also found that
anonymous cvs users tend to provide better patches as they can use the
CVS facilities for patch generation. Some info on anonymous cvs is
available from http://samba.anu.edu.au/cvs.html.
- cvsweb. cvsweb isn't strictly a distribution mechanism but it is
extremely useful for users and developers. It allows anyone with a web
browser to browse through your source code and see what changes were
made when and by who. See the above anonymous cvs reference for an
example.
- mirrors. If your package is popular then you will soon start
receiving offers from people to mirror your web or ftp site on their
servers. This can be good and bad. The advantage is that your package
gets more exposure and is available faster to many people. The
disadvantage is that mirrors so often get out of sync or just stop
being maintained at all.
- web pages. Distributing software by the web is becoming more and
more popular, rivalling ftp. Just make sure you keep your web and ftp
copies in sync!
- by email. A large section of the internet is still only connected
via email. If your package is popular you will inevitably gets emails
asking for a copy from people without ftp or web access. At first you
might just keep a uuencoded copy of your package handy to help these
people. If that gets tedious then you might like to setup an automated
fetch-by-email system. We have done this using the capabilities built
into listproc for Samba and it works well.
Bug Tracking
Apart from the actual coding the most time consuming part of running a
free sofware project is answering queries and managing bug rpeorts. In
fact, for many projects this is much more time consuming than the
actual coding!
There are several packages available that try to make this task a bit
easier. After using a simple email system for a long time I decided
last year to setup one of these systems of the Samba Team. I looked at
the available packages and unfortunately found that none really met
our needs so I started writing a new one. The result is JitterBug, and
it is the success of JitterBug that really prompted me to offer this
paper.
The Samba Team receives a huge number of bug reports, queries and
patches. Nearly all arrive via email. In the last 3 months we have
received around 4000 emails totalling more than 40 MB. This is more
feedback than most commercial software packages and it puts a huge
load on the volunteers that work on Samba.
JitterBug is a web based bug tracking system. It allows all Samba Team
members to view bug reports, respond to them, send or create FAQ
replies and set notifications for messages all without leaving their
web browser.
JitterBug has now been released as free software and is being used by
a large range of other projects. If your have a free software project
that has outgrown your ability to provide simple email support then I
suggest you have a look at JitterBug. See http://samba.anu.edu.au/jitterbug/.
for more details.
There are other systems available, of course. A good list is available
from http://linas.org/linux/pm.html.
Remote Login
If your development team is widely distributed then you may find it
necessary to give people remote logins to a central machine in order
for them to do releases, manage documentation or setup useful services
(such as CVS). Unfortunately, just giving them a normal telnet login
can be a very bad idea as for some inexplicable reason free software
projects appear to receive more than their fair share of interest from
malicious hackers. We have had an amazing number of attacks on our
system, as evidenced by our firewall logs.
To make things a bit more secure we insist that all logins are only
done via ssh and we only allow a very limited access list through a
firewall. Each users source IP address (or address range) is added
individually to the firewall rules giving them access to only port 22
as needed for ssh. New project members need to send their ssh public
key along with the IP address they will be connecting from.
Releases
If your project is typical of most free software projects then you
will probably end up doing quite a few releases as bugs are found and
fixed. You will quickly find this quite tedious unless you automate the
process. For the software projects that I manage I have created a
simple release script that does the following:
- prompts user for version number (auto suggesting next number)
- updates the version.h file
- updates documentation with new version number
- commits the above changes in CVS
- tags the release in CVS so that you can refer to a particular
release by name
- create creates the release package and puts it up for ftp
- creates a set of patches against the previous release and makes
that available via ftp
- puts the latest release into the fetch-by-email system
before this script was created many of the releases were stuffed up
because some detail was missed. It has saved us a lot of trouble!