Wanna-build states: an explanation
This page tries to explain what every wanna-build state means
and what will happen to a package when it's in that state. It's
target audience are Debian package maintainers that try to
understand why their package has, or has not, been built for a
specific architecture. Also, an explanation of the different log
results is given.
Finally, a flowchart version of the wanna-build states is
available, but do note that it doesn't talk
about everything mentioned in this document.
The wanna-build states
For every Debian-supported architecture, there's a wanna-build
database installed on buildd.debian.org, with all packages and their current
compilation state. There are 7 states: needs-build,
building, uploaded, dep-wait,
failed, not-for-us, and
installed.
Their meaning is as follows:
- needs-build
- A package marked needs-build has seen an upload of
a new version by its maintainer, but for a different
architecture than the one this wanna-build database is for; as
such, it needs a rebuild. If the state is
needs-build, it has not been picked up by an
autobuilder yet, but it will be (once one is available at a
time the specific package is near the top of the list). People
commonly say
a package is queued for rebuild
when they are
talking about a package in the needs-build state.
It may be interesting to note that the needs-build
queue is not a FIFO queue; rather, the ordering used is based
on the following criteria:
- Packages' previous compilation states; packages that
have been built previously are given priority over new
packages.
- priorities (packages with required priority are
built before packages with extra priority)
- The section a package is in. This ordering is based on what
packages are deemed more important; e.g., section games is
built after section base, and section libs is
built before devel.
- an asciibetical ordering on the package name.
Additionally, under certain conditions, it may happen that a buildd
will not take packages at the head of the queue; for instance,
when a buildd can't find the source of a given package, it will
put it back in the queue (where it will then again be put at its
previous position, i.e. the head of the queue), but it will
ignore the package for a few hours. Another example where this
might happen is when an architecture has multiple autobuilders;
in that case, the architecture's porters may choose to build
larger packages on their faster autobuilders, and leave the
smaller ones for the slower machines in the pool. A buildd can
theoretically also explicitly request a different section ordering,
but that is not usually done.
There could be other situations where the queue order seems to
be ignored; but note that they are all exceptions.
- building
- A package is marked building from the moment an
autobuilder picks it from the top of the wanna-build queue
until the moment the autobuilder admin replies to the log. As
packages are not picked one by one, this means a package can
be (and usually is) marked building before the build has
actually started; however, as buildd builds packages in its
local queue on a FIFO basis, it should not take too long
anymore. Also, note that the state of a package is
not modified once the build is complete; only
when the autobuilder admin comes around to replying to the
logs.
- uploaded
- When a build attempt was successful, a build log is sent to
the autobuilder admin and to buildd.debian.org. The
autobuilder maintainer will then sign the .changes file which
is embedded in the build log, and send it to the
autobuilder. In reaction, the autobuilder will upload the
package and set its state to uploaded. As such, a
package in this state can be found in the incoming queue
(somewhere).
An autobuilder will not touch a package anymore once it's
state is uploaded, at least not until the next upload
or until a porter manually modifies the state of a package.
- dep-wait
- When a package fails due to missing build-time dependencies,
the autobuilder maintainer will send a mail to the
autobuilder, instructing it to remove the package sources and
to mark the package as dep-wait on the missing
build-dependencies. A package in such a state will
automatically, without human intervention, be marked
needs-build once said dependencies are available.
Originally, a package had to see a build attempt before the
maintainer would manually place it in the dep-wait
state. However, in august 2005 some code was added to
wanna-build which will make a package move from the installed state directly to the
dep-wait state, if that is appropriate.
There are two specific cases in which it may happen that a
package is marked dep-wait forever; these are when a typing
error happened by specifying the dep-wait dependencies
(so that the package is marked dep-wait on a package that does
not and will not ever exist) and when a build-time dependency is
declared on a package which is marked not-for-us, or
which is in the packages-arch-specific list.
As an example to the latter, consider three packages: a package
foo, which exists for i386 only; a package
bar, which exists for m68k only (and which roughly
performs the same function); and a package baz, which can be
built with one of foo or bar. Should the maintainer
of the baz package forget to add bar to the
Build-Depends, and should he or she add it when it is noticed that
baz is dep-waiting on a non-existing foo for
m68k, then the dep-wait state for m68k will
have to be manually lifted by the m68k porters.
- failed
- If a build attempt failed, and the autobuilder maintainer
decides it is really a failure that should not be retried, a
package is marked as failed. A package will not leave
this state until a porter decides it should do so, or until a
new version is available. However, when a new version of a
package is available which was marked as failed in
the previous version, the autobuilder will ask its admin
whether or not the package should be retried; this is so that
packages which will obviously fail again will not waste buildd
time. Although failing a package before trying a build is
hardly ever the right thing to do, the option is available to
the autobuilder admin.
Note that a package will never be marked
failed without human intervention.
- not-for-us
- Certain specific packages are architecture-specific; for
instance, lilo, an i386 boot loader, should not be
rebuilt on alpha, m68k, or s390. However, wanna-build
does not look at the control file of a package when creating its
database; only at the packages' name and section, the previous
build state, and its priority. As such, by the first upload of
an architecture-specific package which should not be built on
other architectures, a build attempt is tried none the less (but
fails even before the build-time dependencies are downloaded
and/or installed)
Since autobuilders should not waste time trying to build
packages that aren't required for their architecture, there's
need for a way to list packages for which even an attempt to
build isn't required. The first solution to this problem was
not-for-us; however, as that is difficult to
maintain, not-for-us is deprecated nowadays;
autobuilder maintainers should use
packages-arch-specific instead, which is a list of
packages specific to one or more architectures instead of a
wanna-build state.
A package in not-for-us or
packages-arch-specific will not
leave this state automatically; if your package specifically
excluded a given architecture in its control file previously,
but now includes more architectures, it must be
manually requeued.
If you ever find yourself in the position that you have to ask
for this to happen, you can do so by asking the relevant buildd
maintainer. They can be reached at $arch@buildd.debian.org.
- installed
- As the name suggests, a package marked installed is
compiled for the architecture the wanna-build database is
for. Before Woody was released, a package's state changed from
uploaded to installed after the daily katie
runs. With the implementation of Accepted-autobuild,
however, this is no longer true; nowadays, a package goes
from state uploaded to installed when it is
accepted into the archive. This means that a package is
usually marked installed after 15 minutes, on
average.
In addition to these seven states, wanna-build also
knows two -removed states, which are really corner cases. These
two states are dep-wait-removed and
failed-removed. They relate to their respective plain
state as follows: when a package in state failed or
dep-wait doesn't appear in a new Packages file which is
fed to wanna-build – when it appears it has been
removed – the information about that package isn't thrown
away, as it could be that the package not appearing in the
Packages file is just a temporary glitch, or that the package is
temporarily removed for some reason (but that it will
reappear in the archive, given time). Instead, in such a case, a
package is moved to a -removed state, so that the
information on why it failed or what it's waiting for can be
retained. Should the package reappear in a following Packages file
which is fed to wanna-build, it will then be moved from
failed-removed back to failed, or from
dep-wait-removed back to dep-wait before further
processing.
It is not possible to access the wanna-build database directly;
this database is installed on ftp-master.debian.org, which is a
restricted host, and only autobuilders have an SSH key which
allows them to access the wanna-build database of their
architecture. This has been the case even before ftp-master was
restricted; As wanna-build does a database-level lock when
accessing, even reading, the data, you had to be in the right
group (wb-<arch>) to be able to directly access a
wanna-build database.
That said, you can see what state a package is in by
going to the buildd
stats page, except if it is in state installed
(well, not unless you don't mind digging through the
multi-megabytes "<arch>-all.txt" files...).
Alternatively, there are some web pages available through
crest.debian.org, which
are based on parsing of the <arch>-all.txt files, and give
an easier to read (at least for humans) way to see what state a
package is in.
The build log results
When a package is built by sbuild (the buildd component which
does the actual building), a log with the build result is sent,
by mail, to the autobuilder admin and to logs@buildd.debian.org
(so that it can end up at http://buildd.debian.org). The build
log result can be one successful, failed,
given-back, or skipped. Note that, at the buildd log overview
page, the prefix maybe- is added, because among
other things, the fact that a build can be marked
failed there for things that aren't really a
failure has caused confusion in the past (or, the other way around,
sometimes a package which apparently built successful is really broken
and needs to be rebuilt).
The meaning of
the log results is as follows:
- successful
- The build was successful. When the autobuilder maintainer
receives this log, he will extract the embedded
.changes file, sign it, and send it back to the
autobuilder, which will cause the package to be uploaded.
- failed
- The build failed. As there can be a number of reasons why a
build fails, enumerating all them would be tedious, so no
attempt is done here. If a package of yours is marked
(maybe-)failed, you will want to read the above, and
check its current wanna-build state.
- given-back
- The build failed due to a temporary problem with the
autobuilder; examples include network problems, the
unavailability of the packages' source with the current
sources.list, low disk space, and others.
A package which is given-back is marked as
needs-build again; as
such, it will be automatically picked up by a different
autobuilder once one is ready.
- skipped
- In the time between the package was picked by the/an
autobuilder and marked building and the build attempt, a new
version for this package was uploaded, or a porter manually
modified the wanna-build state for another reason. When that is
done, a mail is sent to the autobuilder, which will mark the
package as not to be built; sbuild sees this, and will skip the
build (although a build log with this result is sent, describing
the fact that this happened).
To illustrate the above, we have also provided a flowchart-version of this procedure. Again, note
that it does not contain everything mentioned in this document.
