[Mesa-dev] [PATCH v2 0/5] Use GitLab CI to build Mesa

Thu Sep 20 13:50:17 UTC 2018

On Tue, 2018-09-04 at 14:43 -0700, Clayton Craft wrote:
> Quoting Juan A. Suarez Romero (2018-08-29 03:12:33)
> > Hello.
> > 
> > This is a first part, version 2, of a more complete proposal to use GitLab CI to
> > build and test Mesa. This first part just adds the required pieces to build
> > Mesa, using the different supported tools (meson, autotools, and scons).
> > 
> > A second part, to be sent in the future, will use the results of the former to
> > run different tests and verify everything works fine.
> > 
> > An example of the pipeline that will result from this patchset can be seen at
> > https://gitlab.freedesktop.org/jasuarez/mesa/pipelines/3070.
> > 
> > I hope I can explain here all the rationale behind this proposal. Any question
> > is welcomed.
> > 
> > 
> > In this V2 series, we have:
> > 
> > - Addressed all comments in V1, specifically:
> > 
> >   - Reviewed the dependencies installed in the base images
> >   - libdrm is now installed in the base image using Meson
> >   - Some minor changes is done in the base image system to speed up installation
> >   - Fixed a small error in the inspect-images script
> > 
> > - Added support for scheduled builds, so base images can be rebuilt every now
> >   and then to obtain any fix/update in the dependencies.
> > 
> > - Have different base/llvm images per branch. So far we only have a single
> >   base/llvm image, that was used for all Mesa builds. But his can create
> >   conflicts if for instance staging/XX.Y has different dependencies than master:
> >   the base images should be different. To avoid this, each branch uses its own
> >   base/llvm images. This is not a big problem from space POV in the registry, as
> >   if the images are the same for two branches in the registry it will share the
> >   same Docker layers.
> > 
> > 
> > ## Brief introduction
> > 
> > This proposal is based on a setup we are using at Igalia for the different
> > releases we were in charge of. The idea is to build different Docker images
> > containing Mesa with different setups; thus we are build an image using Meson,
> > another using Autotools, and so on. Even more, as the Gallium drivers depend on
> > the LLVM version, we also use different setups with different LLVM versions.
> > 
> > Some of those images are later used to run different test suites in different
> > hardware. So we just save those Docker images for further pulling in a next step
> > in the pipeline. For those that we are not using for testing, we plainly discard
> > them. At least, they helped to verify that the setup builds correctly.
> > 
> > A very complete example of this setup pipeline can be seen in
> > https://gitlab.com/igalia/graphics/mesa/pipelines/26539728
> > 
> > As said before, right now we are just focusing on building the images, rather
> > than testing them. In a second part to be sent we will focus on the testing.
> > 
> > 
> > ## Why Docker
> > 
> > When we started the initial setup, we have something clear: just build
> > everything once, and tests as in many devices as possible. The reasons are
> > different: we have powerful machines for building, but then for testing we have
> > different hosts with different power skills; some low-end NUCS, some with few
> > memory, and so on. And also, we wanted to ensure that the discrepancies when
> > testing were not due the different software configuration (different distros,
> > different versions, different installed packages, ...): use the same software as
> > much as possible. And then we have another requirement: have an easy way to
> > distribute the result in the testing machines.
> > 
> > And Containers is something that really helps to achive this. And inside
> > Containers, Docker is a de-facto, so we started to use Docker. Wich Docker, we
> > can build self-contained images with everything we need to run Mesa, and
> > distribute it very easily. And the fact that it allows to use layers to build
> > the images, makes quite easy to reuse previous results in the future builds.
> > 
> > 
> > ## Why Rocker
> > 
> > As said before, Docker is a great tool, but it has a couple of drawbacks when
> > building the images.
> > 
> > On one side, the Dockerfile syntax is quite rigid, and does not allow for
> > complex configuration. For instance, it's quite difficult to create a Dockerfile
> > where you can allow to use Meson or Autotools to build Mesa, specified as
> > parameter when building the image. Either you create an unreadable Dockerfile,
> > or ends up creating different Dockerfiles, even when most of them shares a lot
> > of common code. And if you start to add more options, like LLVM version, it
> > becomes crazy.
> > 
> > One way to solve this problem is to create a Dockerfile with templates, like
> > Jinja or any similar tool, and then generate the final Dockerfile with the
> > proper options.
> 
> Using docker-compose, you could create a dockerfile that uses an env variable
> for specifying the build system, then one (or more) 'services' in the compose
> file that run the same dockerfile with the env. var set to your desired build
> system. You'd need to add some logic to handle the env var could be a script
> installed into the container that calls meson or autotools.
> Something like this might allow you to re-use the dockerfile without any
> further modifications to support one or more build systems.
> 

Something to explore in the future, indeed.

> > 
> > The other problem with Docker, more complex to build, is how to allow to
> > cache. We are using ccache to speed up the build time, but the problem is that
> > Docker does not allow to mount external directories when building an image. And
> 
> Docker-compose allows bind mounts in containers:
> https://docs.docker.com/compose/compose-file/#volumes
> 

Yes, but I think this is the same as in docker: volumes are mount when running a
container, not when building an image.

Said that, I must say too that Docker in Fedora is modified to allow bind
mounting on build time. I did not used this version because it is not in the
rest of the docker; I would like to allow people not using Fedora to have a
simple way to create images on their own local system.

> > thus, we can't reuse the cache generated in previous builds. There are some
> > proposals like creating different images, or building the image using `docker
> > run` (which allows to mount those directories) plus `docker commit` to generate
> > the final image. But all of them leads to Dockerfiles quite difficult to
> > maintain. So solution is normally just not use cache, with the impact this has.
> > 
> > In our case, we solved both problems using Rocker instead of Docker. Rocker is
> > like a `docker build` with steroids, that fixes the above problems. It is a tool
> > that is able to create Docker images based on Rockerfiles, that are like
> > Dockerfiles but with enriched syntax, that provides the same power as a
> > template. And, which is more important, allows to mount external directories
> > when building the image; we use this feature to mount the ccache results
> > directory.
> > 
> > Unfortunately, Rocker is a tool that is not maintained anymore
> > (https://github.com/grammarly/rocker). We still use it because it provides
> > everything we need, and fortunately we don't need any more features.
> > 
> > Maybe there are other alternatives out there, but we were happy with this and
> > hence our proposal. If we want to use Docker rather than Rocker, then we could
> > use template tools like Jinja, and forget about caching during build, which will
> > impact on the build time.
> > 
> > 
> > ## Involved stages
> > 
> > The dependencies required to build Mesa doesn't change very frequently, so
> > building them everytime is a waste of time. As Docker allows to create images
> > based on the content of other images, we have defined the setup in several
> > stages.
> > 
> > On a first stage a "base" image is built. This image contains almost all the
> > dependencies required to build Mesa. Worth to mention that libdrm is excluded
> > here, as this is a dependency that really changes quite frequently, so we
> > postpone the installation for further stages.
> > 
> > One we have the "base" image, we create different images with the different LLVM
> > compilers. This ensure that when using a specific image we only have that LLVM
> > version, and not any other.
> > 
> > An important point here is that, these builts appears in the pipeline, they are
> > not actually built if not required. That is, in the case of the base image, if
> > the Rockerfile used to create the image has changed with respect to the one used
> > to create the image that is already in the registry, then the image is rebuilt
> > (as this means something changed, very likely some dependency). But if the
> > Rockerfile didn't change, then there is no need to rebuild the image, and just
> > keep using the one already in the registry. This is also done for the LLVM
> > images. This helps to improve the speed, as most of the times they don't need to
> > be built again.
> > 
> > The third stage is the one that builds Mesa itself. Here we just define which
> > tool to use and which LLVM version. This is done by passing the right parameters
> > to the `rocker build` tool. It will pick the right base image, install the
> > missing dependencies (mainly, libdrm), select which drivers should be built
> > (based on the LLVM version and parsing the configure.ac file), and create the
> > image.
> > 
> > As LLVM version only affects the Gallium drivers, and in order to reduce the
> > build time, we added a fake tool "gallium" that just builds only the Gallium
> > drivers based on the LLVM version.
> > 
> > Another worth to mention point is the `distcheck` tool. As it names suggests,
> > this run `make distcheck` to build Mesa, which basically creates a dist tarball,
> > uncompress it, and builds it with autotools. The bad part is that it only builds
> > the dist tarball with autotools, but it does not checks with the other tools
> > (scons and meson). To fix this, this `distcheck` job generates as an artifact
> > the dist tarball, which is used by the "Tarball" stage in the pipeline (4th
> > stage) to uncompress and builds using those tools.
> > 
> > Finally, at this moment we only save one of the images created, the one
> > corresponding to the `autotools` jobs, as this has all the drivers built, and
> > can be downloaded and used for testing. The other ones are just used to verify
> > that everything is built correctly, but discarded afterwards.
> > 
> > 
> > CC: Andres Gomez <agomez at igalia.com>
> > CC: Daniel Stone <daniels at collabora.com>
> > CC: Dylan Baker <dylan.c.baker at intel.com>
> > CC: Emil Velikov <emil.velikov at collabora.com>
> > CC: Eric Anholt <eric at anholt.net>
> > CC: Eric Engestrom <eric.engestrom at intel.com>
> > 
> > 
> > Juan A. Suarez Romero (5):
> >   gitlab-ci: build Mesa using GitLab CI
> >   gitlab-ci: build base images if dependencies changed
> >   gitlab-ci: Build from the released tarballs
> >   gitlab-ci: update base + llvm images with scheduled pipelines
> >   gitlab-ci: use branch/tag in base image names
> > 
> >  .gitlab-ci.yml                    | 210 ++++++++++++++++++++++++++++++
> >  gitlab-ci/Rockerfile.base         | 189 +++++++++++++++++++++++++++
> >  gitlab-ci/Rockerfile.llvm         |  62 +++++++++
> >  gitlab-ci/Rockerfile.mesa         | 133 +++++++++++++++++++
> >  gitlab-ci/inspect-remote-image.sh |  21 +++
> >  5 files changed, 615 insertions(+)
> >  create mode 100644 .gitlab-ci.yml
> >  create mode 100644 gitlab-ci/Rockerfile.base
> >  create mode 100644 gitlab-ci/Rockerfile.llvm
> >  create mode 100644 gitlab-ci/Rockerfile.mesa
> >  create mode 100755 gitlab-ci/inspect-remote-image.sh
> > 
> > -- 
> > 2.17.1
> > 
> > _______________________________________________
> > mesa-dev mailing list
> > mesa-dev at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/mesa-dev