[igt-dev] [PATCH v3 12/12] ci: Add job for testing changes to tests in Qualcomm devices

[Tomeu Vizoso]

On 4/4/22 1:00 PM, Petri Latvala wrote:
> On Fri, Mar 11, 2022 at 03:13:00PM +0100, Tomeu Vizoso wrote:
>> Will allow for more efortless testing of changes to tests that could
>> affect Qualcomm devices.
>>
>> With these changes, it should be fairly trivial to add testing on other
>> KMS and GPU drivers.
>>
>> v2: Update expectations after 9f32d552afd7 ("tests/kms_setmode: Use dynamic subtests")
>>
>> v3: Use igt_runner and store a list of subtests for msm CI
>>
>> Signed-off-by: Tomeu Vizoso <tomeu.vizoso at collabora.com>
>> ---
>>   .gitlab-ci.yml            | 49 +++++++++++++++++++
>>   ci/msm_results.txt        | 98 ++++++++++++++++++++++++++++++++++++++
>>   ci/run_tests.sh           | 14 ++++++
>>   tests/msm_ci/msm.testlist | 99 +++++++++++++++++++++++++++++++++++++++
>>   4 files changed, 260 insertions(+)
>>   create mode 100644 ci/msm_results.txt
>>   create mode 100755 ci/run_tests.sh
>>   create mode 100644 tests/msm_ci/msm.testlist
>>
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> index f820661496c4..ec865bef5e28 100644
>> --- a/.gitlab-ci.yml
>> +++ b/.gitlab-ci.yml
>> @@ -325,6 +325,55 @@ default:
>>         test -e "${CI_JOB_JWT_FILE}" &&
>>         export CI_JOB_JWT="$(<${CI_JOB_JWT_FILE})" &&
>>         rm "${CI_JOB_JWT_FILE}"
>> +
>> +.lava-test:arm64:
>> +  image: $CI_REGISTRY/$CI_PROJECT_PATH/build-debian:commit-$CI_COMMIT_SHA
>> +  variables:
>> +    BASE_SYSTEM_HOST_PREFIX: "${MINIO_HOST}/mesa-lava"
>> +    FDO_UPSTREAM_REPO: gfx-ci/rootfs
>> +    ARCH: arm64
>> +    # Tag corresponding to artifacts from https://gitlab.freedesktop.org/gfx-ci/rootfs/
>> +    KERNEL_ROOTFS_TAG: "2022-03-11-jq--2022-03-08-bzip2--290b79e0e78eab67a83766f4e9691be554fc4afd"
>> +    BASE_SYSTEM_MAINLINE_HOST_PATH: "${BASE_SYSTEM_HOST_PREFIX}/${FDO_UPSTREAM_REPO}/${KERNEL_ROOTFS_TAG}/${ARCH}"
>> +    JOB_ARTIFACTS_BASE: ${PIPELINE_ARTIFACTS_BASE}/${CI_JOB_ID}
>> +    JOB_RESULTS_PATH: "${JOB_ARTIFACTS_BASE}/results.tar.gz"
>> +    MINIO_ARTIFACT_NAME: igt-arm64
>> +    BUILD_PATH: "${PIPELINE_ARTIFACTS_BASE}/${MINIO_ARTIFACT_NAME}.tar.gz"
>> +    JOB_ROOTFS_OVERLAY_PATH: "${JOB_ARTIFACTS_BASE}/job-rootfs-overlay.tar.gz"
>> +    KERNEL_IMAGE_NAME: Image
>> +    KERNEL_IMAGE_TYPE: "image"
>> +    VISIBILITY_GROUP: "Collabora+fdo"
>> +    HWCI_TEST_SCRIPT: "/install/ci/run_tests.sh"
>> +  dependencies:
>> +    - build:tests-debian-meson-arm64
>> +  needs:
>> +    - build-containers:build-debian
>> +    - build:tests-debian-meson-arm64
>> +  script:
>> +    - ./ci/lava/lava-submit.sh
>> +  artifacts:
>> +    name: "igt_${CI_JOB_NAME}"
>> +    when: always
>> +    paths:
>> +      - results/
>> +  tags:
>> +    - $RUNNER_TAG
>> +  after_script:
>> +    - wget -q "https://${JOB_RESULTS_PATH}" -O- | tar -xz
>> +
>> +test:msm:
>> +  extends:
>> +    - .lava-test:arm64
>> +  stage: test
>> +  variables:
>> +    DEVICE_TYPE: sc7180-trogdor-lazor-limozeen
>> +    DTB: sc7180-trogdor-lazor-limozeen-nots
>> +    BOOT_METHOD: depthcharge
>> +    KERNEL_IMAGE_TYPE: ""
>> +    IGT_FORCE_DRIVER: msm
>> +  tags:
>> +    - mesa-ci-x86-64-lava-rk3399-gru-kevin    # why it doesn't work!? mesa-ci-x86-64-lava-sc7180-trogdor-lazor-limozeen
>> +
> 
> 
> I'm not sure how to dress these questions into words, I'll try my
> best...
> 
> What's the aim of this, from the range of "support testing the kernel"
> to "just test that IGT doesn't break"? In particular, what's the
> kernel used here, how/when is it changed?

The goal here is definitely "just test that IGT doesn't break".

> Can that runner handle the load induced by merging this? For the
> record, IGT forks run the pipelines too, and i915 CI pushes postmerge
> tags _and_ premerge tags to its own ci-tags repository fork so it's
> quote a bit more load than just for every merge to IGT proper.

For now I think we have plenty of capacity as the boards in use are also 
used for Mesa CI, which needs to shard to a minimum of 2 boards, more 
often 5-10.

Also, the number of tests being run are quite small at the moment. If 
the executions start taking longer, then capacity might not be enough.

> If I read that correctly, the results.json is stored in pipeline
> artifacts so there's a way to access the logs if something breaks,
> ok...

Yep, and also the first devcoredump.

> What's the procedure if someone wants to extend this kind of testing
> on their own platform?

I can add docs for this in a future iteration, but most of the work is 
in providing a lab that can receive test jobs from a gitlab runner.

> Is IGT_FORCE_DRIVER necessary? FYI there's also
> IGT_DEVICE=pci:vendor=vendorstring along with other types of
> IGT_DEVICE filters. The semantics are slightly different but
> IGT_FORCE_DRIVER is the older, slightly less capable, device selection
> interface from before IGT_DEVICE.

Back when I looked at this for kernelci.org, that only worked for PCI 
devices, but not platform. Has this changed?

Thanks,

Tomeu