Re: init system daemon readiness protocol
Vincent Bernat:
> All of them are relying on the fact that the monitored process won't
fork.
> They are therefore not able to handle readiness and dependencies.
Also untrue. Handling dependencies has nothing to do with forking, and
it's an error to think that anyone handles readiness. Readiness is not
something that any service manager does reliably. There's just not the
consistent mechanism for it, yet. The RabbitMQ server on one of my
machines, to pick one example, can take bloody ages to start up when it
so happens that it has a lot of persistent messages to scan back in from
disc. And it doesn't even bind its listening sockets, let alone
communicate, until it's done. The end result is that no AMQP client
program can connect, and just (say) stall waiting for the login
response. No service manager fixes this. RabbitMQ server doesn't fork
and exit, and it doesn't yet speak any notification protocol. The
service managers think that it's up and ready, so they go ahead and
start all of the AMQP client services, that are all quite properly
marked as depending from it and that are all quite properly not started
until after the server is. The clients then repeatedly fail to connect().
(This is not the only problem with RabbitMQ and service managers. There
are at least two startup races with the erlang runtime and the erlang
port mapper that I know of. It would be nice if, for starters, the
server could at least be passed a listening socket file descriptor.
Yes, there's "rabbitmqctl". But that's the point, of course. Major
softwares still work on the "Use only our package-specific XYZctl
massive russian-doll-like scripts for service management." basis.)
Russ Allbery, Ian Jackson, and others all discussed this, and made these
same points, back in December 2013 and January 2014. I highly recommend
reading the debian-ctte mailing list discussion. You'll see discussion
of the lack of standardization of readiness protocols, the absence of
any widely adopted readiness protocol, and (yes) both M. Jackson and M.
Allbery in favour of discouraging the idea of forking in order to
"daemonize" something that is already a daemon. (As M. Allbery knows,
I've been encouraging people not to uselessly fork for ... ahem! ...
some years. I recently revisited my FGA with an observation from my
on-going project to make 155 nosh service bundles for the BSDs. A lot
of BSD programs, in recent years, have quietly gained "don't fork"
command line options of one sort or another.)
On the subject of readiness protocols: To see another
already-established mechanism, perhaps avoid reinventing the wheel, and
once again see the error of thinking that all this was only just thought
of and entirely missing a whole history of tools development, see
Laurent Bercot's fifodir mechanism
(http://skarnet.org/software/s6/fifodir.html).
On the subject of forks: It's interesting to observe that an
EVFILT_PROC kevent() with NOTE_TRACK works reliably on FreeBSD/PC-BSD
for tracking across forking daemons. This is not to say that forking is
suddenly alright. But it gives the lie to any claim that
daemontools-family service managers lose track when daemons fork. The
nosh service-manager is quite happy with the old
you-have-to-turn-an-undocumented-debug-option-on-to-stop-it forking
Vixie cron on FreeBSD, for example, and is perfectly aware of and
tracking the forked child process after the parent exits. You'll notice
that I didn't put a "fghack" in nosh. It's not needed on FreeBSD, and
should be equally well unneeded on Debian kFreeBSD. I must remember to
give M. Bercot, M. Pape, M. Guenter, and M. Sampson some good news.
And as I said, dependencies have nothing to do with forking.
Dependencies are a matter of having some way of specifying dependencies
in service configurations, and topologically sorting the low-level
start/stop tasks. Felix von Leitner's minit was doing dependencies back
in 2000.
Reply to: