Re: Ideas for additional large scale tests
Hi there,
(We had a small exchange via private email with Lucas and he suggested
we continue the discussion on debian-qa@, hence full text included
below.)
On Mon, Feb 26, 2007, Lucas Nussbaum wrote:
> On 24/02/07 at 16:34 +0100, Loïc Minier wrote:
> > Hi,
> >
> > I've been rebuilding the GNOME stack with a couple of interesting QA
> > tests:
> > - -Os in CFLAGS
> > - -Wl,-O1 in LDFLAGS
> > - -Wl,-z,defs in LDFLAGS
> > - -j2 in MAKEFLAGS
> > - use differents directories for builddir and srcdir
> >
> > Not all these tests are easily achieved with Debian packages. It was
> > easier for me because I didn't dpkg-buildpackage these, I used a
> > GNOME tool named jhbuild instead.
> >
> > These tests would be however interesting for Debian packages:
> > - -Os is useful for embedded platforms
> > - -Wl,-O1 gives better performances and might be promoted for all
> > packages at some point
> > - -Wl,-z,defs checks for missing link flags; this is a very useful test
> > but it breaks with Python extensions
> > - -j2 is interesting to catch errors in Makefiles and might be used in
> > the future to speed up builds
> >
> > One way to easily test these flags might be to use wrappers for cc,
> > gcc, c++, g++..., adding the flags.
>
> The main problem with testing -Os, -Wl,-O1 and -Wl,-z,defs is that they
> could cause the compiler/linker to generate incorrect code instead of
> just failing, and this is much harder to detect.
I think -Os and -Wl,-O1 are more about testing our toolchain, but I
think it's interesting to check whether this works on a large scale or
in e.g. the GNOME stack because this has some direct benefit for people
interested in the use of these flags and because it will improve our
toolchain. I agree it might also produce broken binaries and that it
would be hard to test, but it's a good first step into ensuring we
could provide a "thin client optimized" archive for example.
-Wl,-z,defs shouldn't cause any incorrect binaries; it should simply
prevent the build in interesting cases, that is when some link flags
are missing. The most problematic packages with -z defs are Python
bindings which explicitely avoid linking to -lpython since the python
interpreter is not linked to libpython, but provides the same symbols.
> make -j2 looks really interesting.
Yes; I expect it will expose some rare Makefile races (rare is in the
context of GNOME since all modules are mostly automakized; other
upstreams might have broken Makefiles :-) but it will certainly help us
work toward building with parallel=$i, and I'm sure you would benefit
of that!
> Are you interested in working with me on that ?
Well, I've got the tools to do the work over GNOME (jhbuild), and I'm
busy enough that I'd prefer avoiding putting my finger in more general
QA stuff. O:-) These were mostly suggestions as a followup to your
request during your talk at FOSDEM of what tests you could run; it
seemed you were requesting more ideas to keep more nodes busy. :)
--
Loïc Minier <lool@dooz.org>
Reply to: