hi jonathan,
thanks for your feedback on this and the previous patches.
before going any further, i'd like to point out that i'm still waiting
for feedback from guillem, and i don't plan on doing any further work
on this until i hear at least something from him indicating that he's
interested in moving forward with this.
to put things as politely as possible i find the current situation
"demotivating" with regards to the amount of time and effort i've put
in and the level of feedback recieved. but i'll leave that there for now.
On Wed, Nov 04, 2009 at 03:17:00AM -0600, Jonathan Nieder wrote:
> I finally got around to reviewing your patch to maintain the conffile
> database during unpack/configure/remove stages. I am still worried
> that the “commit” operation does not seem to be atomic like it should
> be. Could you look at my description of your patch and let me know
> where I go wrong?
i think most of what you say below is correct.
> Okay, so here’s what your patch does:
>
> * Before unpacking, clean out the staged ("new") version of a
> package’s conffile database.
>
> * When unpacking archives, let conffiles pass through the staged
> (new) conffile database before copying to their installed location
> elsewhere in the filesystem.
>
> * When configuring, “commit” the package’s staged conffile database
> (rename it to "current").
>
> * When purging a package’s configuration files, remove its conffile
> database (right after removing the conffiles themselves).
this is all correct.
> Invariants maintained (ignoring a possible tmp conffile db):
>
> * When not-installed, a package has no conffile db.
>
> * In config-files state, a package has a "current" conffile db
> and no other.
correct. there might be a little "*" attached to this invariant though
that it might be possible that a config-files or purged package may be
left with an empty conffile db... at least it was possible at some point
in the evolution of these changes but i can't say for sure whether it's
still the case.
> * When half-installed, a package has no conffile db.
not completely sure on this. i should say that i haven't fully explored
the various corner cases of the package configuration workflow, which is
what i was hoping to get from discussions like these.
> * Once unpacked, a package may or may not have a "current"
> conffile db, which remembers the distributed configuration files
> for the last configured version; and a package will certainly have
> a "new" conffile db, which holds the distributed configuration
> files for the unpacked version.
correct.
> * In half-configured state, everything breaks. It is possible
> to forget the "current" conffile db here. One problem is that
> committing the new conffile db requires multiple steps:
>
> rm -r tmp/pkg
> mv current/pkg tmp/pkg
> mv new/pkg current/pkg
> rm -r tmp/pkg
yeah, this is the only place afaik where the atomicity breaks down, largely
due to not being able to atomically replace one directory with another. i'm
not sure what the best alternative here is...
> If dpkg is interrupted in the middle, the "current" configuration
> can be lost.
>
> $ dpkg --configure pkg
> rm -r tmp/pkg
> mv current/pkg tmp/pkg
> <dpkg is interrupted here>
> $ # okay, better try again.
> $ dpkg --configure pkg
> rm -r tmp/pkg
> ...
it's a pretty small window for failure, and i designed it to do the least
amount of stuff possible during said window of non-atomicity (i.e. one
directory rename). but yes it's hypothetically possible that a power outage
some other externality (unrelated memory errors, i/o errors, etc) could
trigger what you describe. one remedy not requiring any significant changes
would be to have a final "flag" file of some kind to indicate that it's
been successful, which doesn't solve the atomicity problem but could solve
the "how to detect when we previously screwed up" matter.
> One (crazy?) idea would be to overwrite entries in current/pkg
> until they all match new/. That is, run the equivalent of
>
> ln -f new/pkg/afd9834287dc... current/pkg/afd9834287dc...
>
> for each configuration file until their stat information matches.
>
> Upside: I imagine something similar to this would be useful anyway
> for remembering which configuration files have been brought up to
> date. Downside: after this operation, the old distributed
> configuration file is gone. There is no turning back and deciding
> to upgrade the conffile a different way.
it's also quite a bit more complicated, and more stuff could go wrong
during this time (i.e. running out of inodes, etc). with regards to losing
the old files, if those were similarly link-shuffled off to a tmp location
it might be possible to keep them in case of error, though.
another idea: what if we just abandoned the concept of "current"/etc entirely
and used package versions instead, or maybe had them as symlinks? a symlink
can be updated atomically, and if there were some kind of "byversion" dir
then they wouldn't need to cross paths on the filesystem at all.
> - Once installed, the "current" conffile db should remember
> the distributed configuration files and there should be no
> "new" conffile db.
>
> - triggers-awaited and triggers-pending are just like the
> installed state here.
correct.
thanks,
sean
Attachment:
signature.asc
Description: Digital signature