Re: Package testing with Salsa CI for new packages
Hi,
Quick answer for now...
On 19 Aug 2023 at 19:04:09, Paul Boddie wrote:
> On Friday, 18 August 2023 19:54:55 CEST Carles Pina i Estany wrote:
> >
> > Ha! I wasn't aware of the aptly option
> > (https://salsa.debian.org/salsa-ci-team/pipeline/#using-automatically-built-> apt-repository and SALSA_CI_DISABLE_APTLY=0).
> > I think that I might have re-invented the wheel in a tiny part of
> > Debusine CI/CD.
>
> It is certainly a way of propagating packages to those that might need
> them. However, the instructions indicating how a package might access
> these dependencies appear to be deficient.
I haven't used aptly on salsa but on Monday I want to give it a try, to
see if I can replace my solution...
> It does not appear to be sufficient to merely specify the dependency
> package repositories and mark them as trusted. Doing that will just
> cause the repositories to be regarded as ignored and the GPG keys
> signalled as unrecognised.
>
> So, the GPG keys need to be obtained. This is a hassle because something like
> wget is needed to do that, and then apt has to be persuaded not to fail in an
> opaque way. So the modified recipe is something like this:
>
> before_script:
> - apt-get update
> - NON_INTERACTIVE=1 apt-get install -y wget
> - echo "deb https://salsa.debian.org/moin-team/emeraldtree/-/jobs/4575438/
> artifacts/raw/aptly unstable main" | tee /etc/apt/sources.list.d/pkga.list
> ...
> - wget -q -O /etc/apt/trusted.gpg.d/emeraldtree.asc https://
> salsa.debian.org/moin-team/emeraldtree/-/jobs/4575438/artifacts/raw/aptly/
> public-key.asc
I'm ensure that this is correct. I have:
-----
deb [arch=amd64 signed-by=/usr/share/keyrings/oracle-virtualbox-2016.gpg] https://download.virtualbox.org/virtualbox/debian bookworm contrib
-----
So I thought that the "signed-by" was needed; and that the files that I
have there are binary files (not .asc) of this type:
carles@pinux:~$ file /etc/apt/trusted.gpg.d/skype.gpg
/etc/apt/trusted.gpg.d/skype.gpg: OpenPGP Public Key Version 4, Created Wed Jun 22 09:36:35 2016, RSA (Encrypt or Sign, 2048 bits); User ID; Signature; OpenPGP Certificate
carles@pinux:~$
I don't know if a .asc files are allowed. Actually, from my
.bash_history:
---
wget -O- https://www.virtualbox.org/download/oracle_vbox_2016.asc | sudo gpg --dearmor --yes --output /usr/share/keyrings/oracle-virtualbox-2016.gpg
---
(I was following instructions, I didn't try leaving the .asc file there)
> ...
> - apt-get update
>
> This seems to make the various jobs happy, but the one that I was most
> concerned with remains unhappy! I don't actually need the dependencies
> for anything other than autopkgtest, but that job employs its own
> environment where the above recipe has no effect.
You could try using:
---
variables:
SALSA_CI_AUTOPKGTEST_ARGS: '--setup-commands=ci/setup-the-repo.sh'
---
(and write and ship the ci/setup-the-repo.sh in the repo, do whatever
you need there)
Or, probably even better but less flexible:
---
variables:
SALSA_CI_AUTOPKGTEST_ARGS: '--add-apt-source="deb http://MIRROR SUITE COMPONENT'
---
(I found add-apt-source via "man autopkgtest" in TEST BED SETUP OPTIONS.
I haven't used add-apt-source, I don't know what happens with the gpg
keys... but you could use [trusted=yes] :-| )
For the MIRROR you have salsa variables that might help if you need to
specify the pipeline.
> So, what I now need to do is to find out how I can make the new packages
> available to autopkgtest specifically.
We will get there! (one way or another)
> > I will point out at some things that might safe some time to you
> > (great) or not (ignore! :-) ):
> >
> > debusine's .gitlab-ci.yml:
> > https://salsa.debian.org/freexian-team/debusine/-/blob/devel/.gitlab-ci.yml
>
> This is definitely useful for examples of the CI definition syntax and how
> hidden jobs can be used to provide additional script fragments.
>
> [...]
>
> > The job autopkgtest, via debci command, runs autopkgtest:
> > https://salsa.debian.org/freexian-team/debusine/-/jobs/4574458#L65
>
> The difficult thing appears to be the configuration of this testing
> environment. One approach might involve overriding the existing job
> definitions to incorporate the injection of new packages.
hopefully SALSA_CI_AUTOPKGTEST_ARGS will help you. I added this in
salsa-ci/pipeline last year because I was trying to do a similar thing
to what you are doing (I had to pin a package from backports).
(I'm just a user of salsa-ci/pipeline, not a member of the team neither
I can speak for them!)
> [...]
>
> > It's possible for sure. Other people in this list might come up with a
> > different idea. I don't have almost any experience with Debian
> > packaging, but I have some experience on the salsa CI. So I might be
> > giving you solutions that might be sub-optimal! :-)
>
> I think that you are giving me some good ideas at least. I do feel that I am
> encountering problems that must have been encountered many times before, but I
> could imagine that people do not use these tools in this way or have different
> expectations of what they are able to accomplish.
I don't know if an FAQ, conventions, best practises or something might
help...
Cheers,
--
Carles Pina i Estany
https://carles.pina.cat || Wiktionary translations: https://kamus.pina.cat
Reply to: