On robustness of maintainer scripts

[Jakub Wilk <jwilk@debian.org>]

Maintainer scripts generated by our Python helper assume more about 
consistency of installed packages that dpkg can actually guarantee. This 
leads to upgrade failures in some scenarios (not only purely 
theoretical, but also observed by piuparts, and by our users). Here's my 
analysis what is wrong, and how it could be fixed.

For simplicity I'll concentrate mainly on dh_python2, but the other 
helpers have similar problems.


preinst
=======

stdeb used to generate the following preinst script:

if [ -e /var/lib/dpkg/info/<package>.list ] && which pycentral >/dev/null 2>&1
then
	pycentral pkgremove <package>
fi

This is wrong: just because /usr/bin/pycentral exists, doesn't mean you 
can use it. Its dependencies might be temporarily not satisfied, for 
example.

Test case (in a wheezy chroot):
# dpkg --unpack python-central_0.6.17_all.deb
# dpkg --unpack cpuset_1.5.6-2_all.deb
# dpkg --unpack cpuset_1.5.6-2_all.deb # yes, second time

Solution: just remove this snippet from your packages. They were only 
needed for lenny->squeeze upgrades.


postinst
========

dh_python2 generates the following postinst script:

if which pycompile >/dev/null 2>&1; then
	pycompile -p <package>
fi

This is all right provided that the package depends on python (or 
python-minimal). However, is some rare cases such dependency is not 
desirable; debconf is the most prominent example. If the dependency is 
missing, then /usr/bin/pycompile might exist but not be usable (for 
example, because python-minimal dependencies are not installed yet).

Test case (in a wheezy chroot):
# dpkg --unpack python-minimal_2.7.3-4_all.deb
# dpkg -i debconf_1.5.49_all.deb

But, things get worse when you look how pycompile is implemented. What 
it does is: for each supported Python version, check whether 
/usr/bin/python2.X exist, and if it does, run it with -m py_compile. The 
problem is, again: just because the executable exist doesn't mean it 
works.

Now, the assumption "if /usr/bin/pythonX.Y exist, it works" would be 
okayish if set of packages required to run non-default Python version 
was a subset of the default python(3) dependencies. Luckily this used to 
be often the case, so we've been getting away with this bug.

Test-case (in a squeeze chroot, with wheezy added to sources.lists):
# dpkg --unpack cpuset_1.5.6-2_all.deb
# apt-get install -t squeeze python2.6
# apt-get install -t wheezy python gcc-4.4
# dpkg --force-depends -r libssl0.9.8
# dpkg -i python-pkg-resources_0.6.24-1_all.deb

Real-world failures: #680930, #704593
(Yes, the multi-arch package split in src:python3.3 makes the situation 
much worse...)

Proposed solution:
1) Wait until #671711 is fixed.
3) Make pycompile a shell script that does only two simple things:
- write options it was called with to a file, say, 
/var/lib/python/pyX.Ycompile.todo;
- use dpkg-trigger to trigger pythonX.Y.
2) Add code to pythonX.Y's postinst that'll process stuff from 
/var/lib/python/pyX.Ycompile.todo. Note that you don't need to move any 
complicated logic to the interpreter packages. All that the postinst 
would have to do is to call some script shipped in python-minimal.


prerm
=====

dh_python2 generates the following prerm script:

if which pyclean >/dev/null 2>&1; then
	pyclean -p <package>
else
	dpkg -L <package> | grep \.py$ | while read file
	do
		rm -f "${file}"[co] >/dev/null
	done
fi

But dpkg only guarantees that the dependencies are unpacked. (And even 
less in some error situation, but we shouldn't worry about it now.) So 
it could be that /usr/bin/pyclean exists, but it doesn't work.

Test case (in wheezy chroot):
# apt-get install -q python-pkg-resources
# dpkg --force-depends -r libssl0.9.8
# dpkg -r python-ipaddr

Real-world failure: #706758

Proposed solution:
Get rid of /usr/bin/pyclean, so that the fallback code is activated. \o/
Just kidding. :) Not only that would almost certainly have unintended
consequences, but also the fallback code doesn't look complete: it 
doesn't take care of namespaces.

Actual proposed solution:
Rewrite /usr/bin/pyclean in shell.

--
Jakub Wilk