Re: Package testing with Salsa CI for new packages
On Fri, Aug 18, 2023 at 07:19:18PM +0200, Paul Boddie wrote:
> > > Here, it would seem that the most prudent approach is to use the Salsa CI
> > > service instead of trying to get the test suite to run during the package
> > > build process.
> >
> > You should do both if possible, assuming that by "Salsa CI service" you
> > mean autopkgtests which you can, and IMO should, also run locally.
>
> I'm not really clear on what autopkgtest really is, other than a tool that
> uses some kind of test suite description that may reside in debian/test. I'm
> also not completely clear on what is supposed to invoke it, other than either
> the Salsa CI mechanism or dh_auto_test.
The maintainer is supposed to invoke it before uploading the package, and
the Debian infra invokes it on all packages that are uploaded.
Notably, dh_auto_test is unrelated.
> In the Debian Wiki documentation...
>
> https://wiki.debian.org/Python/LibraryStyleGuide
>
> ...it mentions a field in debian/control:
>
> Testsuite: autopkgtest-pkg-python
>
> My impression is that this calls autodep8 to generate some kind of test suite
Yes.
Though note that it generates a trivial test that only checks a top-level
import (just like the wiki page says).
> description which is then invoked by dh_auto_test.
It's not invoked by dh_auto_test. autopkgtests are not a part of the
package build process and they operate on built binary packages.
> It doesn't help that there
> is an alternative to this that resembles it but behaves differently:
>
> Testsuite: autopkgtest-pkg-pybuild
It just generates a different (better) test.
> > > I have also found it difficult to persuade the tests to run successfully
> > > during the build process. A few of these attempt to invoke the moin
> > > program, but this cannot be located since it is not installed in the
> > > build environment.
> >
> > This should also be fine, unless it's completely impossible to run it
> > without installing into /.
>
> The moin program is made available in setup.py using an entry point. Maybe if
> there were some kind of script instead, it would work.
AFAIK there should be ways to work with this.
Ideally, of course, the upstream test suite should support this, but I
understand that not all upstreams use common practices.
> > > However, one conclusion is that testing a system, as some of the
> > > test cases appear to do, and as opposed to testing library functionality,
> > > is not necessarily appropriate when directed by something like
> > > dh_auto_test.
> >
> > If there are tests that can't be run at build time you can skip those. You
> > can even ask the upstream to provide tool arguments to simplify that.
>
> I may well discuss such matters with them. One challenge that is relevant in
> this situation is that upstream have been working in their own virtualenv (or
> venv, or whatever it is now) world for a few years, using plenty of
> dependencies that were not packaged in Debian.
Which is by itself not a problem from the technical side, as you could
just package those (I understand that this requires effort and may have
other problems, but by itself it's normal).
Reply to: