Proposal: incremental release process (the package pool)
This was supposed to be just a draft of the proposal. I'd like
to get it more polished before I submitted it, but considering
the timeline that might not be possible.
My draft comments are enclosed in [brackets].
PROPOSAL: NEW (`Incremental') RELEASE PROCESS --- DRAFT
This message proposes a new release process for Debian. I'm
proposing that this is discussed for the standard period [is
this 2 weeks?] and then voted. The ballot should contain the
options: "SWITCH to the new release process", "KEEP the current
release process", and "Further discussion".
Abstract:
This is a new release process for the Debian operating system.
Currently we try to refine a "unstable" version into a "stable"
release by going trough a period of "freeze"; it is my belief
that this method isn't adequate anymore due to the size of the
system and the project itself.
This new process was first proposed in 98 by Marcus Brinkmann [is
this right?], and since then been periodically discussed but
never voted on.
Rationale:
This new release process, called for some "the package
pool", but which I prefer to call "Incremental release process",
aims to cope with some basic problems that make Debian releases
a tough job:
1: Packages evolve at different speeds. Some packages have new
versions every day, others only change once a year. Some
packages have very unstable development releases, others are
always usable regardless of version. So while a 4-month freeze
may be barely enought for some packages, it will make others
hopelessly outdated (and this is not a matter of "wanting to be
on the bleeding edge" - most times the newer version is a lot
more usable).
2: Debian has grown beyond our own expectatives. The number of
packages and the number of developers are reason of pride for
some (we use these numbers in our Freshmeat ad - "most packages,
most people, the biggest is still the best...") and despair for
others. But the freeze process can become a real nightmare if we
have to deal with almost 4000 packages.
3: An unstable version of a package is an unstable version,
period. Some months of "freeze" will not magically decant it
into a stable version, no matter what.
Just to make sure I won't get flamed, this is how the
Incremental process addresses each of these problems:
1: Slower packages can move at their own pace, the freezes will
always catch the latest usable version. The (hopefully) shorter
freeze period will benefit faster packages.
2: Each maintainer has control over the "stability" of his/her
packages, by "promoting" the most stable version. This
distributes the workload.
3: The middle layer ("working") only holds stable, usable
versions of packages. So the "freeze" process is only a matter
of fine-tuning, dependency sanity checking and polishing.
Presentation of the three layers:
The process consists basically in creating three stability
``layers'': ``pool'', ``working'' and ``stable''.
The three layers are not too different than the system we have
today, only formalized in a different way. Currently we have
"experimental", for software we think won't work too well,
"unstable", for software that may or may not work, and "stable",
which is our released version. By simple logic interpolation, we
can see that we could use a different layer for software that
we're very sure that will work.
This proposal includes erradication of the "experimental" area,
because very few maintaiers use it, because it's "out of the
way" for people to download from it, and because it will be
redundant with the "pool" layer.
In this new release process, all uploads go to a lower layer
called "pool". There may be more than one version of the same
package in the pool.
If a maintainer is very sure about the stability of a given
version of a package, the maintainer "promotes" that version to
the middle layer, called "working". At any moment, someone could
in theory burn a CD of "working" and it would be, surprise, a
"working" operating system. Apt users may safely keep their
systems up-to-date with "working"; it will change less often
than our current "unstable" (thus less downloads), and it will
always work reasonably.
For a package version to be promoted, it must have all of its
dependencies resolvable withing "working". This may be checked
automatically, the necessary code is already in apt-get. If this
requirement isn't met, the "promote" command goes to a queue to
be reprocessed daily; this queue is processed as a whole, so
that tough situations like the recent perl->perl-5005 upgrade
would be dealt with automatically (just send "promote" for each
package, and the whole bunch will get processed when all
packages with "perl" dependencies in "working" have newer
versions in the queue with a fixed dependency).
At any moment, the Project Leader or a delegate (Release
Manager, as we have today, or Release Team perhaps someday) may
declare a freeze. This means copying all packages in "working"
to a codenamed area (like "buzz", "rex", "bo", "hamm", "slink",
"potato"). This snapshot will be thoroughly tested, and when
it's deemed fully stable by the Project Leader or delegate, with
the help from the Testing Team and QA Group, it is released.
Note that this freeze process doesn't require participation from
all developers. Those not interested in releases may just get on
with their lives, continue uploading to "pool" and promoting
stable versions without even bothering about the freeze.
On the other hand, the QA Group (or Release Team if such a thing
is created) may modify any package in the frozen area as they
see fit; this includes fetching newer versions from "working",
manually modifying the package (effectively an NMU), pulling
_older_ versions from previous releases, or even removing the
package from the frozen area.
Maintainers who _are_ interested in the release/freeze process
may, of course, stay in touch with the QA Group/Release Team, or
perhaps even join one of the groups, and take care of his/her
own packages.
Additional burdens:
This proposal admittedly [sp?] places some extra burdens on the
maintainers. These are the worst ones:
1: Maintainers will have to worry about "promote" commands. I
think we can live with that, it's easier than the BTS.
2: Maintainers will have to worry about the versions of the
package present in the pool. Not a show-stopper either.
3: Maintainers will have to worry about which version of their
packages are working. Honestly, I think this is a bonus.
Another possible additional burden placed by this proposal is on
the dinstall managers, if maintainers start to make more (and
possibly buggy) uploads. This would have to be dealt with.
Technical details:
1: We need to be able to keep several versions of a package in
the pool. Packages should be deleted automatically; when one
version is promoted, all older versions are deleted from Pool.
These multiple versions kept are upstream versions; only one
"Debian Revision" is kept for each version. So if I upload
freeciv_1.8.3-4 it will delete freeciv_1.8.3-3, but if I then
upload freeciv_1.8.3+cvs19991010 it will _not_ delete any of the
1.8.3 packages.
2: Promoting should be done either by sending a signed message
to a special automated address (similar to *@bugs.debian.org) or
by running a special script at master. This program must
process, besides the "promote" command, a "remove" command to
purge a package from Working and a "clean" command to remove one
given upstream version from Pool.
[Food for thought: should we keep more than one version in
Working too, so that the maintainer may "change his mind" about
the stability of a version?]
3: To be nicer on mirrors, Working should at all times be simply
a forest of symlinks into Pool, because mirrors can't handle the
movement of a file (they just delete it from the old location
and download it again for the new one).
4: dpkg-scanpackages, or whatever is used to generate Packages
files for apt, must be fixed so that when multiple versions are
found, the newest one is used (currently it uses the first one
found, which will give filesystem-dependent results).
Implementation:
When the current Unstable (potato) is frozen, instead of
creating a new Unstable area, we will create the Pool and
populate it with a copy of potato; plus, create an empty Working
area and wait for maintainers to start populating it; plus,
delete the "project/experimental" area. Of course the promotion
automating software must be working and tested by then.
[]s,
|alo
+----
--
I am Lalo of deB-org. You will be freed.
Resistance is futile.
http://www.webcom.com/lalo mailto:lalo@webcom.com
pgp key in the web page
Debian GNU/Linux -- http://www.debian.org
Reply to: