[igt-dev] [PATCH v3 12/12] ci: Add job for testing changes to tests in Qualcomm devices

[Petri Latvala]

On Fri, Mar 11, 2022 at 03:13:00PM +0100, Tomeu Vizoso wrote:
> Will allow for more efortless testing of changes to tests that could
> affect Qualcomm devices.
> 
> With these changes, it should be fairly trivial to add testing on other
> KMS and GPU drivers.
> 
> v2: Update expectations after 9f32d552afd7 ("tests/kms_setmode: Use dynamic subtests")
> 
> v3: Use igt_runner and store a list of subtests for msm CI
> 
> Signed-off-by: Tomeu Vizoso <tomeu.vizoso at collabora.com>
> ---
>  .gitlab-ci.yml            | 49 +++++++++++++++++++
>  ci/msm_results.txt        | 98 ++++++++++++++++++++++++++++++++++++++
>  ci/run_tests.sh           | 14 ++++++
>  tests/msm_ci/msm.testlist | 99 +++++++++++++++++++++++++++++++++++++++
>  4 files changed, 260 insertions(+)
>  create mode 100644 ci/msm_results.txt
>  create mode 100755 ci/run_tests.sh
>  create mode 100644 tests/msm_ci/msm.testlist
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index f820661496c4..ec865bef5e28 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -325,6 +325,55 @@ default:
>        test -e "${CI_JOB_JWT_FILE}" &&
>        export CI_JOB_JWT="$(<${CI_JOB_JWT_FILE})" &&
>        rm "${CI_JOB_JWT_FILE}"
> +
> +.lava-test:arm64:
> +  image: $CI_REGISTRY/$CI_PROJECT_PATH/build-debian:commit-$CI_COMMIT_SHA
> +  variables:
> +    BASE_SYSTEM_HOST_PREFIX: "${MINIO_HOST}/mesa-lava"
> +    FDO_UPSTREAM_REPO: gfx-ci/rootfs
> +    ARCH: arm64
> +    # Tag corresponding to artifacts from https://gitlab.freedesktop.org/gfx-ci/rootfs/
> +    KERNEL_ROOTFS_TAG: "2022-03-11-jq--2022-03-08-bzip2--290b79e0e78eab67a83766f4e9691be554fc4afd"
> +    BASE_SYSTEM_MAINLINE_HOST_PATH: "${BASE_SYSTEM_HOST_PREFIX}/${FDO_UPSTREAM_REPO}/${KERNEL_ROOTFS_TAG}/${ARCH}"
> +    JOB_ARTIFACTS_BASE: ${PIPELINE_ARTIFACTS_BASE}/${CI_JOB_ID}
> +    JOB_RESULTS_PATH: "${JOB_ARTIFACTS_BASE}/results.tar.gz"
> +    MINIO_ARTIFACT_NAME: igt-arm64
> +    BUILD_PATH: "${PIPELINE_ARTIFACTS_BASE}/${MINIO_ARTIFACT_NAME}.tar.gz"
> +    JOB_ROOTFS_OVERLAY_PATH: "${JOB_ARTIFACTS_BASE}/job-rootfs-overlay.tar.gz"
> +    KERNEL_IMAGE_NAME: Image
> +    KERNEL_IMAGE_TYPE: "image"
> +    VISIBILITY_GROUP: "Collabora+fdo"
> +    HWCI_TEST_SCRIPT: "/install/ci/run_tests.sh"
> +  dependencies:
> +    - build:tests-debian-meson-arm64
> +  needs:
> +    - build-containers:build-debian
> +    - build:tests-debian-meson-arm64
> +  script:
> +    - ./ci/lava/lava-submit.sh
> +  artifacts:
> +    name: "igt_${CI_JOB_NAME}"
> +    when: always
> +    paths:
> +      - results/
> +  tags:
> +    - $RUNNER_TAG
> +  after_script:
> +    - wget -q "https://${JOB_RESULTS_PATH}" -O- | tar -xz
> +
> +test:msm:
> +  extends:
> +    - .lava-test:arm64
> +  stage: test
> +  variables:
> +    DEVICE_TYPE: sc7180-trogdor-lazor-limozeen
> +    DTB: sc7180-trogdor-lazor-limozeen-nots
> +    BOOT_METHOD: depthcharge
> +    KERNEL_IMAGE_TYPE: ""
> +    IGT_FORCE_DRIVER: msm
> +  tags:
> +    - mesa-ci-x86-64-lava-rk3399-gru-kevin    # why it doesn't work!? mesa-ci-x86-64-lava-sc7180-trogdor-lazor-limozeen
> +


I'm not sure how to dress these questions into words, I'll try my
best...

What's the aim of this, from the range of "support testing the kernel"
to "just test that IGT doesn't break"? In particular, what's the
kernel used here, how/when is it changed?

Can that runner handle the load induced by merging this? For the
record, IGT forks run the pipelines too, and i915 CI pushes postmerge
tags _and_ premerge tags to its own ci-tags repository fork so it's
quote a bit more load than just for every merge to IGT proper.

If I read that correctly, the results.json is stored in pipeline
artifacts so there's a way to access the logs if something breaks,
ok...

What's the procedure if someone wants to extend this kind of testing
on their own platform?

Is IGT_FORCE_DRIVER necessary? FYI there's also
IGT_DEVICE=pci:vendor=vendorstring along with other types of
IGT_DEVICE filters. The semantics are slightly different but
IGT_FORCE_DRIVER is the older, slightly less capable, device selection
interface from before IGT_DEVICE.


-- 
Petri Latvala