Re: Debian ISO Testing
Hi Martin!
On Thu, Jul 27, 2023 at 04:52:29PM +0100, Martin McCarthy wrote:
>
>I'm very pleased to be involved in the Debian project. Thank you for having
>me. Let me introduce myself. I'm Martin, I am a developer based in Manchester
>in the UK and I've recently decided to dive head first into the wonderful
>world of FOSS. I have been in the Microsoft ecosystem for the best part of
>two decades and only ever dabbled in Linux and Debian a little bit during
>that time. I recently took part in the Debian 12.1 ISO release event and I
>had fun, learned a lot of things and met some interesting people (including
>the former DPL Sledge). I had some suggestions based on my experience that I
>would like to share with you.
>
>1) When I was conducting testing of Debian 12.1, I had an idea of writing a
>"Testing Wizard" Bash/Perl/Python script to help with the testing process. A
>simple shell script that will test the ISO against a predefined set of tests
>and gather pertinent information (such as hardware information for the team).
>The tester would run the script, all the automatic code would execute (such
>as gathering hardware information, ISO name, boot time, etc) then the wizard
>would guide the tester through a series of required tests and then possibly
>some random ones thrown in (in order to increase chances of catching bugs
>before publishing). I'd be interested to hear other people's opinions on it
>and possibly we could have a session to discuss it in a bit more detail if
>you're interested. If I'm overthinking it or over egging the pudding here, I
>am also interested to hear your feedback there too. :-)
You're describing something that overlaps quite a bit with what we've
discussed already (your point 3 below). We'd like to have a system
set up to guide testing and help us test most effectively:
* minimise the number of tests we need to cover all the features
* track what hardware people are using
* lead people through the tests if needed, to make things easier
>2) Recognition of testers. Although the small group over on #debian-cd on
>OFTC were thankful for everyone's contributions including my own, it would be
>really great for new testers like myself to have some kind of formal project
>recognition. It may seem like a small thing but it makes a huge difference to
>me personally. There are several ways of recognising the testers...some easy
>some not so easy. Firstly, when publishing the release notes, naming the
>testers (by their chosen names) that helped test the images before
>publication would be a nice way to recognise us. Also, as cheesy as this may
>seem or sound, awarding a certificate to recognise that that person has
>participated in the Debian release testing process. Doesn't have to be
>printed of course, it could be designed in Gimp or whathaveyou. I have mocked
>a design which you can view here - > https://justpaste.it/de4hd (NB: this is
>a heads up. This link is SFW...if you happen to be in the office right now).
As Andy C already mentioned, we're already keen to track and recognise
the efforts of volunteers in Debian. If you look at
https://contributors.debian.org/
you'll see that we track what people are doing in a number of ways. If
you follow the link to
https://contributors.debian.org/source/Release%20image%20testers/
that's the set of data for people who've helped test our images. For
you to appear there too, you just need to set up an account and log in
to claim your data. The data is in the system, we just deliberately do
*not* publically list people who haven't signed up. Privacy,
etc... :-)
>3) Overhaul of the wiki testing process. Rattus proposed some overhauling of
>the ISO testing situation with the wiki. I am happy to work on that with you
>Rattus if you get time and we have something where we don't need to wait for
>the wiki to be released from another user. Although, I'm also happy to keep
>it as is if that's what the community wants. If it's a nod to times of old,
>I'm all for that too. I found the process quaint and reminded me of the old
>days of locking Excel workbooks. :D
*grin*
>4) Automation. I read a thread on this mailing list recently talking about
>openQA and whilst I am not opposed to automatic testing by machines, I would
>insist that the manual testers are still retained as relying solely on
>machines for QA can be problematic (machines are not perfect just like
>humans). Not only that, the testers over on #debian-cd is a community and
>having full automation would mean that community no longer meets regularly to
>test stuff...and thus the community disperses and fades away over time. FOSS
>to me is all about community so I would hope that openQA works alongside the
>ISO testers rather than replace us. This is slightly off-topic but AI is
>doing a lot of damage to corporate tech and I hope that the FOSS projects
>don't fall victim to it too.
ACK.
--
Steve McIntyre, Cambridge, UK. steve@einval.com
< liw> everything I know about UK hotels I learned from "Fawlty Towers"
Reply to: