[Mesa-dev] [PATCH v2 1/5] gitlab-ci: build Mesa using GitLab CI
Emil Velikov
emil.l.velikov at gmail.com
Wed Aug 29 13:11:59 UTC 2018
Hi Juan,
I've shared a number of suggestions. I'll leave that to you if they
will be in v3 or patches on top.
On 29 August 2018 at 11:12, Juan A. Suarez Romero <jasuarez at igalia.com> wrote:
> In order to build the images, Rocker is used. This is a tool that
> extends the Dockerfiles with new features that are quite interested
> here. The main features we use is the support for templating, and the
> support for mounting external directories during the image building.
> This help to use tools like ccache to improve the build speed.
>
I think that gitlab-ci supports templating - not sure about mounting
external directories.
But as everyone has said - one could toggle to another tool at a later
stage... if needed.
> v2:
> - Review dependencies (Eric Anholt)
> - Install libdrm in base image with Meson (Eric Anholt)
> - Do system changes before apt to speed up installation (Daniel Stone)
>
> Signed-off-by: Juan A. Suarez Romero <jasuarez at igalia.com>
> Acked-by: Daniel Stone <daniels at collabora.com>
> ---
> .gitlab-ci.yml | 177 ++++++++++++++++++++++++++++++++++++
> gitlab-ci/Rockerfile.base | 185 ++++++++++++++++++++++++++++++++++++++
> gitlab-ci/Rockerfile.llvm | 57 ++++++++++++
> gitlab-ci/Rockerfile.mesa | 132 +++++++++++++++++++++++++++
> 4 files changed, 551 insertions(+)
> create mode 100644 .gitlab-ci.yml
> create mode 100644 gitlab-ci/Rockerfile.base
> create mode 100644 gitlab-ci/Rockerfile.llvm
> create mode 100644 gitlab-ci/Rockerfile.mesa
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> new file mode 100644
> index 00000000000..5cee333dd45
> --- /dev/null
> +++ b/.gitlab-ci.yml
> @@ -0,0 +1,177 @@
> +image: docker:latest
> +
> +services:
> + - docker:dind
> +
> +stages:
> + - base
> + - llvm
> + - mesa
> +
> +variables:
> + DOCKER_IMAGE: $CI_REGISTRY_IMAGE
> + CCACHE_DIR: $CI_PROJECT_DIR/../ccache
> + LLVM: "6.0"
> +
> +cache:
> + paths:
> + - ccache/
> + key: "$CI_JOB_STAGE"
> +
> +before_script:
> + - mkdir -p ccache
> + - rm -fr ../ccache
> + - mv ccache ../
nit: how does this look
rm -rf ../ccache
mkdir -p ../ccache
Although why do we .. in the first place? Mind adding a comment?
> + - export MAKEFLAGS=-j$(nproc)
> + - apk --no-cache add libc6-compat
> + - wget https://github.com/grammarly/rocker/releases/download/1.3.1/rocker-1.3.1-linux_amd64.tar.gz
> + - tar xvf rocker-1.3.1-linux_amd64.tar.gz
> + - rm rocker-1.3.1-linux_amd64.tar.gz
> + - mv rocker ..
> + - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
> +
> +after_script:
> + - mv ../ccache ./
> +
> +.build_llvm: &build_llvm
> + stage: llvm
> + cache: {}
> + script:
> + - ../rocker build -f gitlab-ci/Rockerfile.llvm --var LLVM=$LLVM
> + - docker push $CI_REGISTRY_IMAGE:llvm-$LLVM
> +
I think we can drop most, if not all the special LLVM handling. More
details below.
> +
> +llvm:3.3:
> + variables:
> + LLVM: "3.3"
> + <<: *build_llvm
> +
> +llvm:3.6:
> + variables:
> + LLVM: "3.6"
> + <<: *build_llvm
> +
> +llvm:3.8:
> + variables:
> + LLVM: "3.8"
> + <<: *build_llvm
> +
I'm fairly sure we can drop anything older than 3.9 through the series.
Jose was pretty clear that they're aiming/moved to llvm 5.0
> +RUN apt-get update \
> + && apt-get --no-install-recommends -y install autoconf automake gcc g++ libtool-bin \
> + pkg-config gettext ccache make scons bison flex sudo git wget bzip2 xz-utils \
> + libclc-dev libelf-dev libexpat1-dev libffi-dev libomxil-bellagio-dev \
> + libpciaccess-dev libx11-xcb-dev libxdamage-dev libxml2-dev libxrender-dev \
> + libxvmc-dev libunwind-dev zlib1g-dev python-pip python-setuptools python-wheel \
> + python3-pip python3-setuptools python3-wheel \
> + && rm -fr /var/lib/apt/lists/*
Why do we need the rm after apt-get?
> +USER local
> +
A user with name "local" sounds a bit strange. Is that the recommendation?
> +WORKDIR /home/local
> +
> +ENV LDFLAGS="-L/usr/local/lib $LDFLAGS"
> +
> +{{ if .Env.MAKEFLAGS }}
> +ENV MAKEFLAGS={{ .Env.MAKEFLAGS }}
> +{{ end }}
> +
> +{{ if .Env.CCACHE_DIR }}
> +MOUNT {{ .Env.CCACHE_DIR }}:/home/local/.ccache:Z
> +RUN sudo chown -R local:local /home/local/.ccache
> +ENV PATH=/usr/lib/ccache:$PATH
> +{{ end }}
> +
> +ATTACH [ "/bin/bash" ]
> +
> +RUN wget https://xorg.freedesktop.org/releases/individual/proto/glproto-1.4.14.tar.bz2 \
> + && tar -jxvf glproto-1.4.14.tar.bz2 \
> + && rm glproto-1.4.14.tar.bz2 \
> + && cd glproto-1.4.14 \
> + && ./configure \
> + && make \
> + && sudo make install \
> + && sudo ldconfig \
> + && sudo rm -fr ../glproto-1.4.14
> +
Thinking out loud:
There should be a way to make this a trivial function and feed it the
static data.
...
wget foo
tar filename(foo)
cd basename(foo)
...
> --- /dev/null
> +++ b/gitlab-ci/Rockerfile.llvm
> +{{ if eq .LLVM "3.3" }}
> +RUN wget https://people.igalia.com/jasuarez/packages/llvm-3.3_3.3-+checkinstall1_amd64.deb \
> + && dpkg -i llvm-3.3_3.3-+checkinstall1_amd64.deb \
> + && rm llvm-3.3_3.3-+checkinstall1_amd64.deb
> +ENV LD_LIBRARY_PATH=/usr/lib/llvm-{{ .LLVM }}/lib:$LD_LIBRARY_PATH
> +
With 3.3 gone, there's no need to build our own LLVM package ;-)
> +{{ else }}
> +MOUNT .:/context
> +RUN ./autogen.sh \
> + && make distcheck \
> + && __version=`cat VERSION` \
> + && mkdir -p /context/release-output \
> + && mv /home/local/mesa-head.txt /context/release-output \
> + && mv mesa-$__version.tar.xz /context/release-output \
> + && sudo rm -fr /home/local/mesa
> +
> +{{ else if eq .BUILD "autotools" "gallium" }}
> +
> +RUN export LLVM={{ $llvm_version }}.0\
> + && eval `cat configure.ac | egrep ^LLVM_REQUIRED` \
> + && if dpkg --compare-versions $LLVM ge $LLVM_REQUIRED_GALLIUM ; then GALLIUM_DRIVERS=i915,etnaviv,freedreno,imx,nouveau,pl111,r300,svga,swrast,tegra,v3d,vc4,virgl ; fi \
> + && if dpkg --compare-versions $LLVM ge $LLVM_REQUIRED_R600 ; then GALLIUM_DRIVERS=$GALLIUM_DRIVERS,r600 ; fi \
> + && if dpkg --compare-versions $LLVM ge $LLVM_REQUIRED_RADEONSI ; then GALLIUM_DRIVERS=$GALLIUM_DRIVERS,radeonsi ; fi \
> + && if dpkg --compare-versions $LLVM ge $LLVM_REQUIRED_SWR ; then GALLIUM_DRIVERS=$GALLIUM_DRIVERS,swr ; fi \
/me dreams of a day where only a single LLVM_REQUIRED will be available
> + && DRI_DRIVERS=i915,i965,nouveau,r200,radeon,swrast \
> + && VULKAN_DRIVERS=intel \
> + && if dpkg --compare-versions $LLVM ge $LLVM_REQUIRED_RADV ; then VULKAN_DRIVERS=$VULKAN_DRIVERS,radeon ; fi \
> + && ./autogen.sh \
> + --with-gallium-drivers=$GALLIUM_DRIVERS \
> + {{ if eq .BUILD "gallium" }} --with-dri-drivers="" \
> + {{ else }} --with-dri-drivers=$DRI_DRIVERS \
> + --with-vulkan-drivers=$VULKAN_DRIVERS \
> + --with-platforms=x11,drm,wayland {{ end }} \
Omitting a vulkan/dri/gallium drivers list will default to building
some of them :-\
Perhaps we could set the *DRIVERS variables conditionally, defaulting to "".
> + {{ if ne $llvm_version "0.0" }} --enable-llvm --enable-llvm-shared-libs {{ end }} \
> + {{ if ne $debug_build "false" }} --enable-debug {{ end }} \
> + --enable-glx-tls --enable-gbm --enable-egl \
> + && make \
> + && make check \
> + && sudo make install \
> + && sudo ldconfig \
> + && sudo rm -fr /home/local/mesa
> +
> +{{ else if eq .BUILD "meson" }}
> +
> +RUN meson _build \
Worth adding the gallium/vulkan/dri drivers list here?
HTH
-Emil
More information about the mesa-dev
mailing list