[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Optimization of SalsaCI with sbuild [Debugging]



In line with the email thread and email title, The purpose of this email is to start a collaboration on how best we can optimize Salsa CI by migrating from using dpkg-buildpackage to sbuild for building packages. The reason is to reduce the build time it may take for testing packages. Other reasons are referenced in this MR's description. 

So far there has been a lot of progress as reflected in the diffs. However. One major block that we are having is integrating sbuild with ccache. The pipeline generally has 3 stages;
  1. Images - This creates reusable docker containers using these Dockerfiles that are stored in GitLab to be reused anytime a job is called. these images need to be built and availed so that the build time is reduced. 
  2. Build - This fetches the necessary upstream resources for a particular package and uses builds the package depending on the host, architecture and build environment variables. Currently, this process is using dpkg-buildpackage and this is the section we want to integrate with sbuild
  3. Clean - This cleans up the used images for further use. 
We have been thinking of integrating ccache with sbuild but we are having issues with which stage we can implement that so that it runs successfully. I created a long comment that tries to explain some of my analysis of the whole pipeline and why it would be failing. please use it as a reference. The latest build error that I'd also need some enlightenment on is on this line. 

I would have been verbose in my here but I have tried to provide the necessary links and a basic explanation of what the scenario is. 

I hope to collect perspectives and also answer some questions where I can. 

Regards,
Enock


On Fri, Apr 7, 2023 at 6:46 AM Johannes Schauer Marin Rodrigues <josch@debian.org> wrote:
Hi Enoch,

Quoting Enoch Kaxada (2023-04-07 01:32:24)
> I am writing this email to open up a collaboration so that we can work
> together.
>
> We are working on integrating sbuild into salsa CI to reduce the build time
> mostly. Here is the MR
> <https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404>.
> There are several threads of discussions under the MR to see that we set up
> sbuild in the best way possible but we think we need extra eyes.
>
> Our main challenge is using ccache together with sbuild and the latest
> implementation is on this HEAD
> <https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404/diffs>
> .
>
> We are looking so solid explanations as to why the pipelines errors are the
> way they are.

I got recently diagnosed with multiple sclerosis and am still unable to feel my
feet and fingers. The latter makes using my computer very hard and frustrating.
Please be more specific what problem you are having with sbuild so that I can
minimize my time in front of the keyboard.

Also note, that while I probably know subild best, I have no idea about ccache
integration with it. So I might not be of much help.

Thanks!

cheers, josch

Reply to: