[igt-dev] [PATCH v4 6/6] gitlab-ci: introduce containers with IGT inside

Wed Aug 28 10:13:25 UTC 2019

On Wed, 2019-08-28 at 09:03 +0300, Arkadiusz Hiler wrote:
> On Tue, Aug 27, 2019 at 05:02:02PM +0300, Oleg Vasilev wrote:
> > Currently, the containers are used only to build IGT. We also want
> > to
> > have an option to run IGT inside a container. For this, new build
> > stage
> > is added.
> > 
> > Signed-off-by: Oleg Vasilev <oleg.vasilev at intel.com>
> > ---
> >  .gitlab-ci.yml            | 25 +++++++++++++++++++++++-
> >  .gitlab-ci/docker-help.sh | 41
> > +++++++++++++++++++++++++++++++++++++++
> >  Dockerfile                | 14 +++++++++++++
> >  3 files changed, 79 insertions(+), 1 deletion(-)
> >  create mode 100755 .gitlab-ci/docker-help.sh
> >  create mode 100644 Dockerfile
> > 
> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > index 0ac12318..76862514 100644
> > --- a/.gitlab-ci.yml
> > +++ b/.gitlab-ci.yml
> > @@ -16,6 +16,7 @@ stages:
> >    - build
> >    - test
> >    - deploy
> > +  - containers
> 
> I am not sure it is the right call to make that a stage after deploy.
> 
> deploy:pages is buggy and fails often:
> https://gitlab.freedesktop.org/freedesktop/freedesktop/issues/46
> 
> For pre-merge it should be ok, if it proves to be troublesome we can
> shuffle it around then.

Can I just swap deploy and containers? 

> 
> >  ################# BUILD CONTAINERS #######################
> >  
> > @@ -85,12 +86,15 @@ build-containers:build-fedora:
> >  build:tests-fedora:
> >    stage: build
> >    script:
> > -    - meson $MESON_OPTIONS build
> > +    - meson --prefix=`pwd`/opt-igt $MESON_OPTIONS build
> >      - ninja -C build
> > +    - ninja -C build install
> >      - ninja -C build igt-gpu-tools-doc
> > +    - ls opt-igt
> 
> Leftover ls?
> 
> >    artifacts:
> >      paths:
> >        - build
> > +      - opt-igt
> >  
> >  build:tests-fedora-no-libunwind:
> >    stage: build
> > @@ -273,3 +277,22 @@ pages:
> >    only:
> >      - master
> >  
> > +############## CONTAINERS ####################
> > +
> > +
> > +containers:igt:
> > +  stage: containers
> > +  image: docker:stable
> > +  services:
> > +    - docker:dind
> > +  dependencies:
> > +    - build:tests-fedora
> > +  variables:
> > +    DOCKER_HOST: tcp://docker:2375
> > +    DOCKER_DRIVER: overlay2
> > +    FORCE_REBUILD: 1
> > +  script:
> > +    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN
> > $CI_REGISTRY
> > +    - docker pull $CI_REGISTRY/$CI_PROJECT_PATH/build-
> > fedora:$CI_COMMIT_REF_NAME
> > +    - docker tag $CI_REGISTRY/$CI_PROJECT_PATH/build-
> > fedora:$CI_COMMIT_REF_NAME build-fedora
> 
> You forgot to change those to use commit-$CI_COMMIT_SHA:
> https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/jobs/540405

Right, I'll retest the series over a clear registry to make sure
everything builds correctly. 

> 
> > +    - .gitlab-ci/pull-or-rebuild.sh Dockerfile igt
> > diff --git a/.gitlab-ci/docker-help.sh b/.gitlab-ci/docker-help.sh
> > new file mode 100755
> > index 00000000..6e3f8021
> > --- /dev/null
> > +++ b/.gitlab-ci/docker-help.sh
> > @@ -0,0 +1,41 @@
> > +#!/bin/sh
> > +#
> > +# Copyright © 2019 Intel Corporation
> > +#
> > +# Permission is hereby granted, free of charge, to any person
> > obtaining a
> > +# copy of this software and associated documentation files (the
> > "Software"),
> > +# to deal in the Software without restriction, including without
> > limitation
> > +# the rights to use, copy, modify, merge, publish, distribute,
> > sublicense,
> > +# and/or sell copies of the Software, and to permit persons to
> > whom the
> > +# Software is furnished to do so, subject to the following
> > conditions:
> > +#
> > +# The above copyright notice and this permission notice (including
> > the next
> > +# paragraph) shall be included in all copies or substantial
> > portions of the
> > +# Software.
> > +#
> > +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> > EXPRESS OR
> > +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> > MERCHANTABILITY,
> > +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO
> > EVENT SHALL
> > +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
> > DAMAGES OR OTHER
> > +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> > ARISING
> > +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> > OTHER DEALINGS
> > +# IN THE SOFTWARE.
> > +
> > +TEST_LIST=$(find /opt/igt/libexec/igt-gpu-tools -type f -printf
> > "%f ")
> > +cat << END
> > +This is a docker container, packaged with the igt-gpu-tools.
> > + - igt_runner is in the \$PATH. In order for the results to be
> > available on the
> > +   host system, the directory has to be made available inside,
> > e.g.
> > +     docker run -v results:/tmp/results igt-final igt_runner
> > /tmp/results
> > + - The test lists are in /opt/igt/share/igt-gpu-tools
> > + - The test binaries are in \$IGT_TEST_ROOT=$IGT_TEST_ROOT
> > +
> > +In order for the graphic devices to be available inside the docker
> > container,
> > +those either need to be mapped with --device or the container
> > needs to be run
> > +in --privileged mode.
> > +
> > +Contents of /opt/igt/libexec/igt-gpu-tools: $TEST_LIST
> > +PATH: $PATH
> > +LD_LIBRARY_PATH: $LD_LIBRARY_PATH
> > +IGT_TEST_ROOT: $IGT_TEST_ROOT
> > +END
> > diff --git a/Dockerfile b/Dockerfile
> > new file mode 100644
> > index 00000000..b601aecd
> > --- /dev/null
> > +++ b/Dockerfile
> > @@ -0,0 +1,14 @@
> > +# Version number to bump for force image rebuild: 0
> > +FROM build-fedora
> > +
> > +RUN dnf install -y findutils && \
> > +        dnf clean all
> 
> I think we should move those to the base image - `dnf install` seems
> to
> be the most failure-prone part of this whole endeavor. It's the most
> common reason that igt-ci-tags for pre-merge fails (after the other
> issues were addressed).

Well, using build image as a base for igt container should be only
temporarily. Once better base image is ready, it should then be moved there.

> 
> With the things above fixed, the whole series is:
> Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler at intel.com>
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3261 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/igt-dev/attachments/20190828/59eaa4f7/attachment-0001.bin>