Architecture
The “bigger picture” image below, taken from the LPC 2020 talk, shows the overall KernelCI architecture from a user point of view:
The first thing worth noting is that there are two main categories of tests: the “native” ones are orchestrated by KernelCI itself and stored in the KernelCI backend while others are imported into KCIDB from other autonomous systems.
KernelCI native architecture
KernelCI native tests are orchestrated using the following components:
- Core tools contain all the primitive functions implemented in Python as well as the system configuration. This is how kernels are built, test definitions are generated etc.
- kernelci-backend which provides an API on top of Mongo DB to store all the data. It also performs some post-processing such as generating email reports, detecting regressions and triggering automated bisections.
- kernelci-frontend which provides a web dashboard such as the one hosted on linux.kernelci.org. This makes use of the backend API to retrieve results.
- kernelci-jenkins to run a Jenkins instance and orchestrate all the builds and tests being scheduled. It also relies on Kubernetes provided by Microsoft Azure and Google Compute Engine to run all the kernel builds.
- Test labs, typically using LAVA but not only, are hosted by people and organisations outside of the KernelCI project. They are however connected to KernelCI services to run tests and send results directly to the backend.
There are several instances hosted by the KernelCI project, for different needs as explained in the documentation. Each instance is made up of all the components listed above. It’s possible for anyone to set up their own private instance too. However, developers typically don’t need to set up a full instance but only the components they need to make changes to. Here’s how they all relate to each other:
graph TD
frontend(Web frontend) --> backend(Backend API)
backend --> mongo[(Mongo DB)]
core(Core tools) -.-> backend
core -.-> lava[LAVA labs]
lava -.-> backend
jenkins(Jenkins) --> core
jenkins --> k8s[Kubernetes]
Dotted lines are optional dependencies, and solid lines are required ones. To put this in words:
The Core tools can be used on their own on the command line without anything else installed. They may be used to build kernels locally, submit data to a Backend API, or schedule jobs in test labs such as LAVA. Jobs may be run without any Backend API, but if there is one then results can be sent to it. Then Jenkins uses Core tools to do all that, via Kubernetes for kernel builds. Finally, the Web frontend uses a Backend API but nothing depends on it (apart from end users) so it’s entirely optional.
External systems and KCIDB
Other fully autonomous systems producing their own kernel builds and running their own tests can submit results to KCIDB, which is a BigQuery database to provide a unified kernel test reporting mechanism. Please take a look at this blog post for a comprehensive description: Introducing Common Reporting.
Here’s the list of systems currently submitting data:
The monthly KCIDB engagement reports provide more details about the latest status.