Mesa (main): ci/freedreno: Test with non-redistributable traces
GitLab Mirror
gitlab-mirror at kemper.freedesktop.org
Wed Jul 28 14:16:56 UTC 2021
Module: Mesa
Branch: main
Commit: 92f9141f009f9ffe98bea42ba002a8c4bd7a74bd
URL: http://cgit.freedesktop.org/mesa/mesa/commit/?id=92f9141f009f9ffe98bea42ba002a8c4bd7a74bd
Author: Antonio Caggiano <antonio.caggiano at collabora.com>
Date: Wed Feb 10 19:07:32 2021 +0100
ci/freedreno: Test with non-redistributable traces
Add a job to test with traces that we cannot redistribute, listed in a
separate file. Since those traces might not be accessible by everyone,
this job is created only when the pipeline is triggered by `marge-bot`.
This job is optional because otherwise it could be blocking a merge
request of someone who cannot really debug the issue due to lack of
access to these traces.
The documentation available under `docs/ci` goes into more details
explaining the rationale behind optional traces.
Signed-off-by: Antonio Caggiano <antonio.caggiano at collabora.com>
Signed-off-by: Tomeu Vizoso <tomeu.vizoso at collabora.com>
Reviewed-by: Emma Anholt <emma at anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6441>
---
.gitlab-ci.yml | 2 +-
.gitlab-ci/piglit/run.sh | 6 ++--
docs/ci/index.rst | 35 +++++++++++++++++++++
src/freedreno/ci/gitlab-ci.yml | 18 +++++++++++
src/freedreno/ci/restricted-traces-freedreno.yml | 39 ++++++++++++++++++++++++
5 files changed, 97 insertions(+), 3 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 1c07f5cf214..d7f2763b43d 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1265,7 +1265,7 @@ debian-mingw32-x86_64:
variables:
PIGLIT_PROFILES: replay
PIGLIT_REPLAY_UPLOAD_TO_MINIO: 1
- PIGLIT_REPLAY_EXTRA_ARGS: --keep-image --db-path ${CI_PROJECT_DIR}/replayer-db/
+ PIGLIT_REPLAY_EXTRA_ARGS: --keep-image --db-path ${CI_PROJECT_DIR}/replayer-db/ --minio_host=minio-packet.freedesktop.org --minio_bucket=mesa-tracie-public --role-session-name=${CI_PROJECT_PATH}:${CI_JOB_ID} --jwt=${CI_JOB_JWT}
PIGLIT_JUNIT_RESULTS: 1
.deqp-test:
diff --git a/.gitlab-ci/piglit/run.sh b/.gitlab-ci/piglit/run.sh
index 10ccad88ae3..5a45b4a3867 100755
--- a/.gitlab-ci/piglit/run.sh
+++ b/.gitlab-ci/piglit/run.sh
@@ -212,6 +212,10 @@ fi
FAILURE_MESSAGE=$(printf "%s" "Unexpected change in results:")
+if [ "x$PIGLIT_PROFILES" = "xreplay" ]; then
+ ci-fairy minio login $MINIO_ARGS $CI_JOB_JWT
+fi
+
eval $RUN_CMD
if [ $? -ne 0 ]; then
@@ -237,8 +241,6 @@ mkdir -p .gitlab-ci/piglit
if [ "x$PIGLIT_PROFILES" = "xreplay" ] \
&& [ ${PIGLIT_REPLAY_UPLOAD_TO_MINIO:-0} -eq 1 ]; then
- ci-fairy minio login $MINIO_ARGS $CI_JOB_JWT
-
__PREFIX="trace/$PIGLIT_REPLAY_DEVICE_NAME"
__MINIO_PATH="$PIGLIT_REPLAY_ARTIFACTS_BASE_URL"
__MINIO_TRACES_PREFIX="traces"
diff --git a/docs/ci/index.rst b/docs/ci/index.rst
index d3c5c52388f..1cb90f79b17 100644
--- a/docs/ci/index.rst
+++ b/docs/ci/index.rst
@@ -52,6 +52,41 @@ The three GitLab CI systems currently integrated are:
LAVA
docker
+Application traces replay
+-------------------------
+
+The CI replays application traces with various drivers in two different jobs. The first
+job replays traces listed in ``src/<driver>/ci/traces-<driver>.yml`` files and if any
+of those traces fail the pipeline fails as well. The second job replays traces listed in
+``src/<driver>/ci/restricted-traces-<driver>.yml`` and it is allowed to fail. This second
+job is only created when the pipeline is triggered by `marge-bot` or any other user that
+has been granted access to these traces.
+
+A traces YAML file also includes a ``download-url`` pointing to a MinIO
+instance where to download the traces from. While the first job should always work with
+publicly accessible traces, the second job could point to an url with restricted access.
+
+Restricted traces are those that have been made available to Mesa developers without a
+license to redistribute at will, and thus should not be exposed to the public. Failing to
+access that URL would not prevent the pipeline to pass, therefore forks made by
+contributors without permissions to download non-redistributable traces can be merged
+without friction.
+
+As an aside, only maintainers of such non-redistributable traces are responsible for
+ensuring that replays are successful, since other contributors would not be able to
+download and test them by themselves.
+
+Those Mesa contributors that believe they could have permission to access such
+non-redistributable traces can request permission to Daniel Stone <daniels at collabora.com>.
+
+gitlab.freedesktop.org accounts that are to be granted access to these traces will be
+added to the OPA policy for the MinIO repository as per
+https://gitlab.freedesktop.org/freedesktop/helm-gitlab-config/-/commit/a3cd632743019f68ac8a829267deb262d9670958 .
+
+So the jobs are created in personal repositories, the name of the user's account needs
+to be added to the rules attribute of the Gitlab CI job that accesses the restricted
+accounts.
+
Intel CI
--------
diff --git a/src/freedreno/ci/gitlab-ci.yml b/src/freedreno/ci/gitlab-ci.yml
index 00385b5d299..dd75b5aa2c2 100644
--- a/src/freedreno/ci/gitlab-ci.yml
+++ b/src/freedreno/ci/gitlab-ci.yml
@@ -1,3 +1,6 @@
+variables:
+ PRIVATE_TRACES_COMMITTERS: "/^(robclark|anholt|flto|cwabbott0|Danil|tomeu)$/"
+
.freedreno-test:
extends:
- .baremetal-test
@@ -279,3 +282,18 @@ a630-traces:
# missing.
MESA_GLSL_VERSION_OVERRIDE: "460"
MESA_GL_VERSION_OVERRIDE: "4.6"
+
+a630-traces-restricted:
+ extends:
+ - a630-traces
+ variables:
+ PIGLIT_REPLAY_DESCRIPTION_FILE: "/install/restricted-traces-freedreno.yml"
+ PIGLIT_REPLAY_EXTRA_ARGS: --keep-image --db-path ${CI_PROJECT_DIR}/replayer-db/ --minio_host=minio-packet.freedesktop.org --minio_bucket=mesa-tracie-private --role-session-name=${CI_PROJECT_PATH}:${CI_JOB_ID} --jwt=${CI_JOB_JWT}
+ rules:
+ - if: '($GITLAB_USER_LOGIN =~ $PRIVATE_TRACES_COMMITTERS)'
+ when: on_success
+ - if: '($GITLAB_USER_LOGIN == "marge-bot") &&
+ $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == $CI_COMMIT_REF_NAME'
+ when: on_success
+ - when: never
+ allow_failure: true
diff --git a/src/freedreno/ci/restricted-traces-freedreno.yml b/src/freedreno/ci/restricted-traces-freedreno.yml
new file mode 100644
index 00000000000..313033c88b7
--- /dev/null
+++ b/src/freedreno/ci/restricted-traces-freedreno.yml
@@ -0,0 +1,39 @@
+traces-db:
+ download-url: "http://10.42.0.1:8888/cache/?uri=https://minio-packet.freedesktop.org/mesa-tracie-private/"
+
+traces:
+# Crashes
+# - path: alien-isolation/AlienIsolation.bin.1-trim--k-f2000-v20201203.trace
+# expectations:
+# - device: freedreno-a630
+# checksum: 4ae01533a347796097e3cefff10b029e
+ - path: civilization-v/CivilizationV-trim--s705-761-f762-v20201203.trace
+ expectations:
+ - device: freedreno-a630
+ checksum: ae376491202642d29498285e463538fa
+ - path: golf-with-your-friends/GolfWithYourFriends-trim--f1070-v20201203.trace
+ expectations:
+ - device: freedreno-a630
+ checksum: 7b08782ff6a54a3d088729363080c5b6
+ - path: hollow-knight/HollowKnight-trim--f2020-v20201203.trace
+ expectations:
+ - device: freedreno-a630
+ checksum: e72925568412a8ebdbac68c929972a55
+# Crashes
+# - path: ksp/KSP-trim--f4800-v20201203.trace
+# expectations:
+# - device: freedreno-a630
+# checksum: 59da6ef5c9509616d595d1613fdac42e
+ - path: overcooked2/Overcooked2-trim--f3301-v20201203.trace
+ expectations:
+ - device: freedreno-a630
+ checksum: a45f37ca95f54eb3255f5ca24e2c14cb
+# Crashes
+# - path: plaugue-inc-evolved/PlagueIncEvolved-trim--f1200-v20201203.trace
+# expectations:
+# - device: freedreno-a630
+# checksum: 59da6ef5c9509616d595d1613fdac42e
+ - path: portal2/Portal2-trim--f1020-v20201203.trace
+ expectations:
+ - device: freedreno-a630
+ checksum: 4df1b02310aa08d1e0118240d1f4ae05
More information about the mesa-commit
mailing list