[igt-dev] [PATCH v3 12/12] ci: Add job for testing changes to tests in Qualcomm devices

Fri Apr 22 07:18:18 UTC 2022

On Thu, Apr 21, 2022 at 09:28:48AM +0200, Tomeu Vizoso wrote:
> 
> 
> On 4/4/22 1:00 PM, Petri Latvala wrote:
> > On Fri, Mar 11, 2022 at 03:13:00PM +0100, Tomeu Vizoso wrote:
> > > Will allow for more efortless testing of changes to tests that could
> > > affect Qualcomm devices.
> > > 
> > > With these changes, it should be fairly trivial to add testing on other
> > > KMS and GPU drivers.
> > > 
> > > v2: Update expectations after 9f32d552afd7 ("tests/kms_setmode: Use dynamic subtests")
> > > 
> > > v3: Use igt_runner and store a list of subtests for msm CI
> > > 
> > > Signed-off-by: Tomeu Vizoso <tomeu.vizoso at collabora.com>
> > > ---
> > >   .gitlab-ci.yml            | 49 +++++++++++++++++++
> > >   ci/msm_results.txt        | 98 ++++++++++++++++++++++++++++++++++++++
> > >   ci/run_tests.sh           | 14 ++++++
> > >   tests/msm_ci/msm.testlist | 99 +++++++++++++++++++++++++++++++++++++++
> > >   4 files changed, 260 insertions(+)
> > >   create mode 100644 ci/msm_results.txt
> > >   create mode 100755 ci/run_tests.sh
> > >   create mode 100644 tests/msm_ci/msm.testlist
> > > 
> > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > > index f820661496c4..ec865bef5e28 100644
> > > --- a/.gitlab-ci.yml
> > > +++ b/.gitlab-ci.yml
> > > @@ -325,6 +325,55 @@ default:
> > >         test -e "${CI_JOB_JWT_FILE}" &&
> > >         export CI_JOB_JWT="$(<${CI_JOB_JWT_FILE})" &&
> > >         rm "${CI_JOB_JWT_FILE}"
> > > +
> > > +.lava-test:arm64:
> > > +  image: $CI_REGISTRY/$CI_PROJECT_PATH/build-debian:commit-$CI_COMMIT_SHA
> > > +  variables:
> > > +    BASE_SYSTEM_HOST_PREFIX: "${MINIO_HOST}/mesa-lava"
> > > +    FDO_UPSTREAM_REPO: gfx-ci/rootfs
> > > +    ARCH: arm64
> > > +    # Tag corresponding to artifacts from https://gitlab.freedesktop.org/gfx-ci/rootfs/
> > > +    KERNEL_ROOTFS_TAG: "2022-03-11-jq--2022-03-08-bzip2--290b79e0e78eab67a83766f4e9691be554fc4afd"
> > > +    BASE_SYSTEM_MAINLINE_HOST_PATH: "${BASE_SYSTEM_HOST_PREFIX}/${FDO_UPSTREAM_REPO}/${KERNEL_ROOTFS_TAG}/${ARCH}"
> > > +    JOB_ARTIFACTS_BASE: ${PIPELINE_ARTIFACTS_BASE}/${CI_JOB_ID}
> > > +    JOB_RESULTS_PATH: "${JOB_ARTIFACTS_BASE}/results.tar.gz"
> > > +    MINIO_ARTIFACT_NAME: igt-arm64
> > > +    BUILD_PATH: "${PIPELINE_ARTIFACTS_BASE}/${MINIO_ARTIFACT_NAME}.tar.gz"
> > > +    JOB_ROOTFS_OVERLAY_PATH: "${JOB_ARTIFACTS_BASE}/job-rootfs-overlay.tar.gz"
> > > +    KERNEL_IMAGE_NAME: Image
> > > +    KERNEL_IMAGE_TYPE: "image"
> > > +    VISIBILITY_GROUP: "Collabora+fdo"
> > > +    HWCI_TEST_SCRIPT: "/install/ci/run_tests.sh"
> > > +  dependencies:
> > > +    - build:tests-debian-meson-arm64
> > > +  needs:
> > > +    - build-containers:build-debian
> > > +    - build:tests-debian-meson-arm64
> > > +  script:
> > > +    - ./ci/lava/lava-submit.sh
> > > +  artifacts:
> > > +    name: "igt_${CI_JOB_NAME}"
> > > +    when: always
> > > +    paths:
> > > +      - results/
> > > +  tags:
> > > +    - $RUNNER_TAG
> > > +  after_script:
> > > +    - wget -q "https://${JOB_RESULTS_PATH}" -O- | tar -xz
> > > +
> > > +test:msm:
> > > +  extends:
> > > +    - .lava-test:arm64
> > > +  stage: test
> > > +  variables:
> > > +    DEVICE_TYPE: sc7180-trogdor-lazor-limozeen
> > > +    DTB: sc7180-trogdor-lazor-limozeen-nots
> > > +    BOOT_METHOD: depthcharge
> > > +    KERNEL_IMAGE_TYPE: ""
> > > +    IGT_FORCE_DRIVER: msm
> > > +  tags:
> > > +    - mesa-ci-x86-64-lava-rk3399-gru-kevin    # why it doesn't work!? mesa-ci-x86-64-lava-sc7180-trogdor-lazor-limozeen
> > > +
> > 
> > 
> > I'm not sure how to dress these questions into words, I'll try my
> > best...
> > 
> > What's the aim of this, from the range of "support testing the kernel"
> > to "just test that IGT doesn't break"? In particular, what's the
> > kernel used here, how/when is it changed?
> 
> The goal here is definitely "just test that IGT doesn't break".
> 
> > Can that runner handle the load induced by merging this? For the
> > record, IGT forks run the pipelines too, and i915 CI pushes postmerge
> > tags _and_ premerge tags to its own ci-tags repository fork so it's
> > quote a bit more load than just for every merge to IGT proper.
> 
> For now I think we have plenty of capacity as the boards in use are also
> used for Mesa CI, which needs to shard to a minimum of 2 boards, more often
> 5-10.
> 
> Also, the number of tests being run are quite small at the moment. If the
> executions start taking longer, then capacity might not be enough.
> 
> > If I read that correctly, the results.json is stored in pipeline
> > artifacts so there's a way to access the logs if something breaks,
> > ok...
> 
> Yep, and also the first devcoredump.
> 
> > What's the procedure if someone wants to extend this kind of testing
> > on their own platform?
> 
> I can add docs for this in a future iteration, but most of the work is in
> providing a lab that can receive test jobs from a gitlab runner.
> 
> > Is IGT_FORCE_DRIVER necessary? FYI there's also
> > IGT_DEVICE=pci:vendor=vendorstring along with other types of
> > IGT_DEVICE filters. The semantics are slightly different but
> > IGT_FORCE_DRIVER is the older, slightly less capable, device selection
> > interface from before IGT_DEVICE.
> 
> Back when I looked at this for kernelci.org, that only worked for PCI
> devices, but not platform. Has this changed?

What does "platform" mean here? "Any msm device"?

The device filters can use pci:vendor=hexcode for selecting the first
device matching the vendor part of the PCI ID, or the sysfs selector
like sys:/sys/devices/platform/vgem. You can check what the device
selection can use with tools/lsgpu. The default listing is only --drm
selectors, try --sys / --pci options separately.

(There's named vendors mappint strings to hex in
lib/igt_device_scan.c, looks like so far there's only intel="8086" and
amd="1002")

The major difference with FORCE_DRIVER vs. IGT_DEVICE is the
capability for selecting multiple devices to be used with
drm_open_driver_another(), for testing e.g. prime between msm and
vkms. I don't know how relevant that would be to you. Under the hood
FORCE_DRIVER only forces DRIVER_ANY to mean
DRIVER_THISONE. Practically that does nothing if there's only one
/dev/dri/card* node in the system.

-- 
Petri Latvala