[Mesa-ci-intel] Clarification about 19.1 CI branches

[Clayton Craft]

On Tue, Jul 02, 2019 at 06:19:10PM +0200, Juan A. Suarez Romero wrote:
>Hello.
>
>First of all, I'd like to know if staging/19.1 branch is not working properly in
>the CI.
>
>I've pushed several patches to the branch, but the CI keeps showing only the
>build from the 21th of last month:
>
>https://mesa-ci.01.org/mesa_19_1_staging/builds/

There were some issues over the last few days where the CI results site was not
being updated (our IT did some 'upgrades' that broke things and I wasn't aware
of it until yesterday). Those issues have been resolved, and I have manually
triggered the import of the results for those two builds, they are visible now.

>
>
>The other clarification is about mesa_19_1 and mesa_19_1_daily.

I've summarized the *current* stable release/staging CI jobs below (for my and
anyone else's benefit):

mesa_19.1
        - triggers per-checkin
        - tests stable branch
        - does not test 32-bit and does not test a lot of older/slower platforms
mesa_19.1_daily
        - tests stable branch
        - tests 32-bit and a lot of older/slower platforms
mesa_19.1_staging
        - triggers per-checkin
        - tests staging branch
        - does not test 32-bit and does not test a lot of older/slower platforms
mesa_19.1_staging_weekly
        - triggered once a week on tuesday morning (PST)
        - tests staging branch
        - includes lots of older, slower hardware that would take too long to
          test per-commit
        - intended to help prevent regressions like the G33/i915_dri regression
          that was introduced on 19.0

>
>The thing is that I only push into 19.1 just when I'm going to make the release
>pre-announcement. Which happens once every two weeks. I don't see the need to
>test 19.1 daily if it only will change once in two weeks.

Yea this seems like it could be changed to where mesa_19.1 tests both m32, old
platforms, and atom platforms (in addition to the m64 and other platforms it
currently tests), and the daily stable branch job (mesa_19.1_daily) could be
removed.
>
>
>The only interesting thing from this daily run is that it executes the 32bit
>testsuites. Something that mesa_19_1 does not do. So hence, when I push to 19.1,
>I need to wait for the next day if I want to see what happened with the 32bit
>testsuites.

The intention of the mesa_19.1_staging_weekly CI job is to test 32-bit and
slow/old platforms (previously this was not happening on staging branch..), and
provide results before the release is made. It's a lot of load on the CI to do
this, hence the decision to make this job scheduled weekly instead of
per-checkin. Maybe weekly is too infrequent.

>
>
>I think in the past we didn't have the daily setup, and the mesa_19_1 were
>running both the 64bit and the 32bit testsuites, after a push in 19.1 branch. I
>would like to bring this configuration, because we would saving time running the
>daily setup, and I would get the results for 32bits after pushing the 19.1
>release candidate.

The reason 32-bit testing is deferred to run daily instead of per-checkin for
stable branch testing is because it generates a lot of load on CI: X number of
platforms doing both 32-bit *and* 64-bit runs of Y number of test suites.

>
>
>What do you think?

I think we need a better set of jobs that consider:
        1) coverage on 64-bit and 32-bit for both staging and stable branches
        2) coverage on all platforms for both staging and stable branches
        3) not overloading CI by testing too much too often
        4) getting you (release maintainer) feedback on staging branch/release
        candidates before stable release

With these in mind, I propose the following set of CI jobs, replacing the
current set of CI jobs for staging/release branch testing:

mesa_19.1
        - triggers per-checkin
        - tests stable branch
        - tests 32-bit and tests a lot of older/slower platforms
mesa_19.1_staging
        - triggers per-checkin
        - tests staging branch
        - does *not* test 32-bit and does *not* test a lot of older/slower platforms
mesa_19.1_staging_daily
        - triggered once per day
        - tests staging branch
        - tests 32-bit and tests a lot of older/slower platforms
        - intended to help prevent regressions like the G33/i915_dri regression
          that was introduced on 19.0 from being introduced into the stable
          branch

The stable release branch would get tested with everything CI can throw at it
when new commits are made.

The staging branch would be tested with the same testing that happens on
developer branches today when new commits are made.

The staging branch would also be tested daily with everything CI can throw at it
once per day, regardless of whether or not commits were made to it (the history
built up over time from this can also help identify intermittent failures).
Since this is a big job, running it per-checkin on the staging branch could lead
to the CI becoming overloaded easily if you, for example, pushed two changes to
the branch in a short amount of time.

-Clayton
>
>
>br
>
>	J.A.
>
>
>_______________________________________________
>Mesa-ci-intel mailing list
>Mesa-ci-intel at lists.freedesktop.org
>https://lists.freedesktop.org/mailman/listinfo/mesa-ci-intel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/mesa-ci-intel/attachments/20190702/acee1a47/attachment.sig>