At the heart of the system is the wanna-build database, which keeps track of package versions and states. quinn-diff compares the package lists for the target architecture against the list of source packages every day and feeds a list of packages that need re-compilation into the database where they enter state Needs-Build.
All the build daemons (there can be more than one) query the database regularly for such packages and take some of them so that they go into state Building. Of course, humans also can take packages, e.g. in special cases where automatic compilation isn't possible. Here we also see the second purpose of wanna-build: It ensures that the same version of a package won't be built twice.
If everything goes well, a finished package can be uploaded later, which is another state Uploaded. After that it will eventually be installed into the Debian archive so it appears in the updated package list for the target architecture. This list will be merged into the database, so the package will go to state Installed and remains there until the next version of the source package.
There are several other states; they include: Failed is for packages that failed to build due to errors in the sources, and the errors are expected to be fixed in a successor version (after reporting the problem, of course). So a new version will directly enter Needs-Build, but with a warning that something was wrong with the previous version. Along with this state an error description is stored. State Dep-Wait is used when a package needs some other packages to be compiled but those aren't available yet and must be built before. This state stores a list of required packages and maybe versions, and if all of them are known to be installed the state changes back to Needs-Build.
As we have already seen, the build daemon takes packages from the database for compiling them. Let's look a bit closer: If it has some packages to build, it uses sbuild for the actual compilation process, and for each build a log is mailed to the maintainer of the daemon. He reviews the log and decides what to do with the package: upload it, set it to Failed or Dep-Wait, make some additions to the source dependency list and retry it, etc... If a positive acknowledge is received, the daemon moves it to an upload directory, from where all packages are uploaded by a cron job.
Looking at the log files is the only human intervention in the whole process if no errors happen. There are two good reasons for not further automating this: First, sometimes builds end with an ``OK'' result but the build nevertheless failed for reasons that are invisible to the machine. And second, directly uploading would require to automatically PGP-sign the resulting files with a key without passphrase on the build machine. I considered this an unacceptable security hole.
The build script sbuild more or less just calls some standard Debian tools to compile the sources. It also helps with some common tasks and bookkeeping, but the really special thing about it are the source dependencies. Often packages need other packages installed for compilation, for example compilers and libraries. It is not practical to have all these packages installed all the time, and often it's not even possible because of conflicts. The source dependencies now simply tell sbuild for each package which other packages are needed. It can then automatically install them before the build and remove them again afterwards.
The source dependency list can partially be generated automatically, too, by looking at the dependencies of the binary packages generated by the source. This is andrea's job, which analyses the i386 package list for dependencies and maps library packages to development package names. It also merges the results with manual additions for things that cannot be auto-generated, like compilers or special tools.
Content developed by Roman Hodek for the 6th International Linux-Kongress 1999
