Re: Bug#344615: missinglib: ftbfs [sparc] *** [test] Bus error
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
skaller wrote:
> I am curious to know Debian policy on testing.
There is no official policy, but several people have been working on
trying to put one together lately. See [1] and [2].
> Felix does include a test suite -- but it takes quite some
> time to run. This will get worse as number of tests goes up,
> and number of combinations of options to test goes up.
Currently, I don't think its too long. For a compiler, I think there is
a little more justification for running a lot of tests. Especially when
it runs on so many architectures.
> Furthermore, some new features require testing that begins to
> look nasty -- for example testing sockets requires opening
> sockets (we're using localhost .. but it isn't clear if even
> that is really permitted),
This should be ok.
> and creation of pthreads
> (which in event of a bug could run away on some platforms ..
> and there is no point in tests that will never catch bugs,
> so one has to assume all tests can and will fail).
I believe the autobuilders are fairly good at cleaning up after
packages... there are plenty of unusual things that can happen that I
feel this is probably fairly well tested. For example, it watches the
output and kills it if the package appears to be stuck.
> Worse may be coming, for example testing killing of
> an application may require launching another process
> with sufficient authority which sends it a kill signal
> (and could kill autobuilder by mistake .. :) Yet the
> build scripts themselves are fairly arbitrary so there
> is already a risk of this.
Well, at least that would be detected quite quickly :)
>
> One technique might be a separate test package which
> build-requires the binary package being tested.
> However if the suite fails, the binary -- not the test
> suite -- should be flagged as bugged: it isn't clear
> the Debian autobuilder etc would handle all that properly.
This kind of system is currently being discussed as a possibility for
Debian packages. See the links below for the actual threads.
> Even trickier .. bootstrapped systems, which build
> depend on themselves .. :)
Fortunately, these are fairly rare.
> Any advice on how far to go in original package would
> be of interest, and how to cope with 'extra testing'
> since the need for that will certainly arise: there
> have to be *some* limits on testing in original package,
> but there should also be a way to automate testing
> and bugging out of packages.
Ideally these would be your full-blown test suite which gets run before
your release a new version of the software. This would test regressions
and ensure the program is producing the correct output. The tests run
by a Debian autobuilder should probably be considerably fewer and only
test things like basic functionality (i.e., does the program segv at
startup?)
- -Mike
1 - http://lists.debian.org/debian-devel/2005/12/msg01034.html
2 - http://lists.debian.org/debian-project/2005/11/msg00072.html
3 - http://lists.debian.org/debian-project/2005/11/msg00073.html
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
iD8DBQFDvFBu7ZPKKRJLJvMRAmn6AJkBonXutuEhzCSkwvx6+MzZHkcREgCfZ6D3
QJrVwh9xVYCtOVuUTuwhwXQ=
=1ue8
-----END PGP SIGNATURE-----
Reply to: