Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var
On Wed, 7 Jun 2023 at 04:40, Russ Allbery <rra@debian.org> wrote:
>
> Luca Boccassi <bluca@debian.org> writes:
>
> > diff --git a/policy/ch-files.rst b/policy/ch-files.rst
> > index b34c183..30ce013 100644
> > --- a/policy/ch-files.rst
> > +++ b/policy/ch-files.rst
> > @@ -722,6 +722,43 @@ The name of the files and directories installed by binary packages
> > outside the system PATH must be encoded in UTF-8 and should be
> > restricted to ASCII when it is possible to do so.
> >
> > +.. _s-tmpfiles.d:
> > +
> > +tmpfiles.d
> > +----------
> > +
> > +Packages might need additional files or directories to implement their
> > +functionality. Directories that are located under ``/var/`` or
> > +``/etc/``, and files that are located under ``/var/``, must not be
> > +created manually via maintainer scripts, but instead be declaratively
> > +defined via the `tmpfiles.d
> > +<https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html>`_
> > +interface.
>
> This is an oddly specific list of directories and not at all the
> directories that I would have expected to be handled by tmpfiles.d. My
> naive expectation would be that the most common path handled by tmpfiles.d
> would be /run, /tmp, and /var/tmp. I read through the other messages in
> this bug but I didn't see the explanation for why those directories in
> particular.
>
> Given that neither /var (apart from /var/tmp) nor /etc are transient, I
> would have expected packages to simply ship the directories under those
> paths that they need in the package. Why is that not the right approach?
> Could you explain when tmpfiles.d should be used instead of shipping the
> directory in the package?
It's not only for transient hierarchies as in memory based, but also
for hierarchies that can be blown away and the system is supposed to
recover from. /var is state, and if random things under it are lost,
it should be possible to recover without having to throw away the
whole installation and start over. Of course there will be some
limits, but for the really trivial stuff (ie: "I need to store some
cached data somewhere so I need a directory under /var") it should be
possible to cleanly re-set up in an automated fashion after a reboot.
Not only this, but tmpfiles.d help also enforce that the right
metadata (dac/mac/acl) are set, again without having to reinstall
packages. Also more specifically, for image-based systems we want to
be able to have the vendor tree (ie: content shipped by packages)
restricted to /usr and optionally /etc. One of the goals we have for
Trixie is to no longer ship anything under /var (and /boot) hard-coded
in packages.
So the list of directories there is provided as an example, I can make
it more or less specific as needed.
> > +The ``tmpfiles.d`` file format is defined by the ``systemd`` project, and is
> > +guaranteed to be stable. Ideally, such definitions should be defined upstream
> > +where applicable, and shipped as they are by Debian packages.
> > +
> > +Details about the syntax and installation paths for ``tmpfiles.d`` are defined
> > +by its `reference implementation's documentation,
> > +<https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html>`_ and will
> > +not be redefined here.
>
> I'm clearly missing something, but I was naively expecting this section to
> start with something more like this:
>
> Packages that need to create files or directories in file systems that
> may be deleted on each reboot (for example, ``/run``, ``/tmp``, and
> ``/var/tmp``) should do so via configuration files in the
> ``/usr/lib/tmpfiles.d`` directory. The syntax of those files is
> defined by the `systemd tmpfiles.d documentation
> <https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html>`__.
>
> However, reading that documentation, it sounds like most of the cases for
> other directories are handled by other systemd unit configuration
> directives. We should say that explicitly here and reproduce the list of
> other directories that should be handled directly by the unit file if
> that's what we want people to do.
I'd be more than delighted to recommend using
StateDirectory=/RuntimeDirectory= and friends as the first choice, and
can certainly do so. However, note that those work best for cases
where there is a service which is the clear owner (they can be shared,
with some caveats that we are trying to solve upstream if ephemeral
users are used, but still needs an owner), and this is important for
lifetime management (ie: when to delete). There are also cases where
there is no clear service that is the owner and to whose lifecycle the
directory should be tied to, and for that tmpfiles.d is the solution.
> What's the plan for creating directories in /run on non-systemd systems?
> I had assumed that tmpfiles.d would be used for those directories so that
> we can use the same mechanism for both systemd and non-systemd systems,
> but it looks like that's not ideal for systemd systems. Is it safe to use
> *both* (e.g.) RuntimeDirectory= *and* tmpfiles.d for the same directory so
> that tmpfiles.d is used for non-systemd systems?
I don't think it's safe as they could clash, there are specific
semantics tied to usage of ephemeral users that make it different
enough. Other init systems will simply have to be adapted if we go
that way, eg: the init script should create the directory before
starting the service.
> > +``tmpfiles.d`` snippets should be detected at package build time by
> > +tools such as ``debhelper``, packaged, and the appropriate snippet to
> > +call them on installation, upgrade, removal, purge and other steps as
> > +required, should be automatically added by helpers such as
> > +``dh_installtmpfiles``.
>
> Policy should not say things like this. It's the job of Policy to define
> what those snippets are and be specific. It's Policy-compliant to not use
> debhelper at all (we've had some discussions of changing that, but we
> haven't yet). debhelper is one implementation of Policy (and
> additional non-Policy stuff), so we should be defining (at a high level)
> what it needs to do and what any non-debhelper parallel implementation
> should do.
Ok, it was meant as a way to say "please don't go call things by hand
in your maintainer script and use the integrated tools instead", which
I think is something we do want to say in policy? Any specific wording
you'd prefer? One of the goals here is to reduce custom maintainer
scripts code, if the change results in people adding more to call
tools by hand it's kinda counterproductive.
> Also, isn't the most obvious way to implement this to use triggers? That
> runs into the problem that Policy still doesn't have any documentation of
> triggers, but (unlike debhelper) we can just require that implementations
> of the tmpfiles.d mechanism pick up new files via triggers and do the
> right thing, which feels appealing.
If you mean that the upstream tools should learn to use triggers, then
obviously that cannot happen, as dpkg triggers are a debianism. If you
mean that debhelper should implement this via triggers - perhaps, but
there is one case that I am slowly working to implement, running
tmpfiles to clean things on purge, where that wouldn't work. I think
I'd prefer to leave this open for now, and see what we come up in the
end on the debhelper side. My intention here was pushing toward using
integrated tools and not writing maintainer scripts by hand, that's
all.
> > Packages shipping ``tmpfiles.d`` snippets should depend on the
> > +appropriate virtual packages in the following order:
> > +``default-systemd-tmpfiles | systemd-tmpfiles``.
>
> I read the discussion of this and agree this is the best approach.
>
> > +Init systems are required to integrate with ``tmpfiles.d`` and run the
> > +service that applies them on boot, and regularly for cleanup
> > +purposes. The documentation for the reference implementation,
> > +`systemd-tmpfiles,
> > +<https://www.freedesktop.org/software/systemd/man/systemd-tmpfiles.html>`_
> > +explains how to call the program so that the appropriate ``tmpfiles.d``
> > +snippets are applied at the appropriate time.
>
> I don't really like being this nonspecific. Can't we just say
> specifically what init systems need to do? I assume it's something like
> "call systemd-tmpfiles --create --boot on boot and systemd-tmpfiles
> --clean periodically on some schedule" (what schedule?).
New command lines options do get added and can be useful, so I'd
rather not specify the exact invocation in policy, as that's going to
go out of date pretty soon.
> Also, based on the previous discussion, it sounds like we should also say
> something about how non-systemd init systems must depend on an
> implementation of the tmpfiles.d mechanism. That's arguably a consequence
> of the requirement they integrate with it, but given that the exact
> dependencies matter a lot for ensuring nothing weird happens during system
> package installation, I think we should spell it out.
Ok, will add something about that.
> > +Instead, :ref:`s-tmpfiles.d` snippets should be shipped, being ideally
> > +provided by the upstream sources, if any. For more details about the
> > +``tmpfiles.d`` interface, see :ref:`s-tmpfiles.d`.
>
> I personally lean against commenting on what upstream should do. I think
> any maintainer already knows that ideally as much is upstreamed as
> possible, it's not Policy's role to say stuff like that (that's more of a
> Maintainer's Guide thing), and it tends to prompt people to file weird
> bugs or create weird Lintian checks that are unactionable for maintainers
> of packages whose upstreams have no intention of providing these files for
> whatever reason.
>
> Policy is only about Debian packages. We're not writing policy for
> upstream; there's a wiki page that tries to collect that sort of advice.
What I meant to say here is, if upstream provides a tmpfiles.d, use
that instead of writing a debian-specific one where possible, to avoid
unnecessary debianisms. Any suggestion on how to you want to see this
worded?
Reply to: