DEP17 - /usr-merge - what has happened - what will happen - what you can do to help
Hello developers,
yeah, I know, this is annoying to many. Still I hope that we can close
this chapter by trixie with your help.
# What has happened?
Since the unstable buildds have been updated to be merged-/usr, the file
move moratorium has been officially delegated to
https://wiki.debian.org/UsrMerge and in that course has been reduced.
The debootstrap uploads to bullseye p-u and bookworm p-u have been
reviewed accepted and will be part of the next stable point release.
usr-is-merged now enforces a merged layout in trixie and unstable. If
you are faced with failures from debootstrap consider updating your
debootstrap from bullseye-updates or bookworm-updates.
dh_movetousr and dh-sequence-movetousr is a thing and can be used to
convert packages.
dh_installsystemd and dh_instalinit now install units to /usr. This
renders 11 packages rc-buggy and patches for all instances have been
filed. On the flip side, more than 500 source packages will complete the
transition in their next upload or binNMU.
While systemd.pc still points the unit directory below /lib, the change
is prepared and patches have been filed for all resulting bugs. The
change is still being deferred, because it would cause 19 rc bugs as of
this writing.
systemd (255) has removed support for the split layout in unstable.
I have sent patches moving shared libraries in essential from /lib to
/usr/lib.
# What will happen next?
I have spent quite some effort on ensuring that most of systemd units
would move with little impact. As a result, I hope that we can resolve
more than half of them using no-change uploads or binNMUs. These will
not be scheduled now, because there is little urgency yet and your
regular uploads will make such binNMUs unnecessary in many cases.
I now change focus away from systemd units towards the essential set.
This is a tricky affair as it risks breaking bootstrap and the
debian-installer. As such, I intend to send patches for affected
packages and have started doing so already. There are quite a few more
patches to come. Within three months we need to reach the point where
essential is fully converted with the exception of those packages that
cannot be converted without breakage. At that point, I'll coordinate a
synchronized NMU of the remaining packages. Affected maintainers have
received a mail while ago. And then we hopefully return to the simpler
pre-/usr-merge bootstrap protocol where packages describe what makes up
Debian. I hope to have this completed by the end of March 2024.
My next focus will be difficult cases. There are two problem categories
that I already know will require non-trivial patches. One is udev rules
in Multi-Arch: same packages and the other is diversions. These probably
all need patches and extensive testing.
What remains is converting the remaining packages and we ideally get
that done by trixie. This is the point where we consider binNMUs for
systemd units. As we progress, we'll also encounter more and more
problems caused by concurrent file move and restructuring (DEP17 P1) and
will get a better understanding of how the approach using Conflicts
impacts upgrades from bookworm to trixie. Possibly, we'll have to
convert some of those Conflicts (DEP17 M7) into protective diversions
(DEP17 M8) in order to unbreak upgrades.
As much as I'd like to say we're done then, there will still be a number
of tasks remaining. The release-notes will likely need an update.
External repositories need help adapting to Debian's changes.
Derivatives will need help setting up their own monitoring. We'll also
notice some broken pieces. For most of us though, problems should fade
away.
# What you can do to help?
Please do apply /usr-merge related patches in a timely manner. I try to
give you time by sending them in a useful order and leaving at least
two weeks before they become urgent.
Please move files from / to /usr yourself.
https://wiki.debian.org/UsrMerge has instructions on whether your
package is eligible and what to do. I will not be able to send patches
for each and every package. Neither from a funding pov nor does my day
have sufficient hours. Getting this done must be a collective effort.
Please upload restructuring changes to experimental. If you rename
binary packages (e.g. adding a "t64" suffix for the 2038 transition) or
move a file from one package to another, please upload to experimental.
This advice is valid for the entire trixie cycle. In doing so, dumat is
enabled to spot /usr-merge related problems in your package and report
them to you rather than you having to check for them. Alternatively,
run[1] dumat locally before upload.
If you want to support this effort beyond your own packages, help is
appreciated with writing patches. Especially the ones for udev rules are
relatively mechanical and just need someone doing them. I'm happy to get
you started and reviewing them. Consider stopping by in
OFTC#debian-usrmerge.
I hope that this all makes sense to you. In case it does not, don't
hesitate to talk. There probably are omissions and errors still. Your
previous feedback has been very valuable and I hope it all has found its
way into the process.
Thanks for your support
Helmut
[1] Running dumat locally is not a trivial affair. You may operate it
from a git clone. You need a current database and unless you want to
import it yourself, you may use a current snapshot from
https://subdivi.de/~helmut/dumat.sql.zst. Once imported into a 1.4GB
sqlite3 database, please generate a reference analysis using
./analyze.py -d dumat.db. Then, you may import your updated package
using ./import_mirror.py -d dumat.db --changes somefile.changes and
reperform the analysis using ./analyze.py -d dumat.db. Compare the
output to the reference generated earlier and try to understand the
possible difference.
Reply to: