Review of debusine's autopkgtest related API
[ Please keep me in CC as I'm not subscribed to debian-ci@lists.debian.org ]
Hello Antonio, Paul, Martin, Simon, and other members of the CI team,
it's been a long time that I have been interested in building some
infrastructure to manage scheduling and distribution of Debian-related
build, QA and data collection tasks to a network of worker machines. I
called this project debusine:
https://salsa.debian.org/freexian-team/debusine/
It has been started a while ago but thanks to external funding, its
development pace is about to increase significantly. It is being developed
by Freexian with the intention of giving people access to a range of
pre-configured tools and workflows running on remote hardware.
We want to make it easy for Debian contributors to leverage all the great
QA tools that Debian provides. We want to improve and modernize Debian's
build infrastructure, while also enabling distribution-wide experiments,
custom package repositories and custom workflows with advanced package
reviews.
Analyzing autopkgtest results is an important step in any serious package
review workflow and as such, our current milestone plans to make it
possible to run autopkgtest on debusine workers. As autopkgtest and debci
experts, we would love to have your feedback on our plans, in particular
on the public interface that we have designed.
To give a bit more context, as a debusine user with an API key, you can
submit "work requests". Each work request has a "task" assigned to it
(e.g. sbuild, autopkgtest, lintian) and some JSON data that gives more
details about the specific task. The structure of the JSON data varies
from task to task but it basically defines the public API that lets users
schedule work requests to debusine. The result of the work request
(including any artifact generated) will then be stored in the database and
made available for consumption by the user through the API.
As an example, here's how the data could look like for a simple
autopkgtest work request:
{
"input": {
"source_artifact_id": 1234, # References a source package
"binary_artifact_ids": [1235, 1236], # References associated binary packages
},
"architecture": "amd64",
"distribution": "debian:unstable",
"backend": "qemu",
"extra_apt_sources": [
"deb https://deb.debian.org/debian experimental main"
]
}
We are currently drafting the expected structure of that JSON data for
autopkgtest (and lintian) work requests in this merge request and we would
appreciate if you could review it:
https://salsa.debian.org/freexian-team/debusine/-/merge_requests/300
I have left some open questions in the document and there are some
unresolved review threads that have interesting questions too.
We are aware that this duplicates features of debci, but we are trying to
achieve a high level of integration and sharing worker setups across
different QA tools, and duplication seems unavoidable in that context.
We plan to provide a debusine.debian.net/org instance accessible to all
Debian developers in the near future, so you will be able to experiment
and make use of the API that you will have helped to shape.
If you have questions about debusine, don't hesitate to ask. If you are
interested to follow along and/or help, you are more than welcome to:
https://freexian-team.pages.debian.net/debusine/devel/contributing.html
Have a nice day,
Raphaël.
PS: You might find it useful to browse the documentation to learn more about the
goals and the high level concepts:
https://freexian-team.pages.debian.net/debusine/devel/why.html
https://freexian-team.pages.debian.net/debusine/design/index.html
--
⢀⣴⠾⠻⢶⣦⠀ Raphaël Hertzog <hertzog@debian.org>
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋ The Debian Handbook: https://debian-handbook.info/get/
⠈⠳⣄⠀⠀⠀⠀ Debian Long Term Support: https://deb.li/LTS
Reply to: