[Intel-gfx] [QUERY] How many CI mails is too many?

[Chris Wilson]

Quoting Joonas Lahtinen (2017-11-28 08:15:13)
> On Mon, 2017-11-27 at 16:54 +0200, Arkadiusz Hiler wrote:
> > Hey all,
> > 
> > For some time already CI sends out 1-2 mails per series per (re)run, i.e. BAT
> > results and "full IGT" results (if BAT has not failed).
> > 
> > Recently we have added 32bit build check, and if that fails it sends out
> > additional mail In-Reply-To the series.
> > 
> > I am working on adding some static checks to the CI (spare and checkpatch at the
> > moment, more may come in the future), which may generate even more commotion on
> > the mailing list.
> > 
> > How much of CI noise is too much and how you would like to have the results
> > grouped?
> > 
> > Couple of options to start the discussion:
> > 
> >  1. Group all static checks (and the 32bit build?) into one mail:
> >     - just one additional mail,
> >     - may be hard to read in case of catastrophic failure,
> >     - we can send it only when something actually fails.
> > 
> >  2. Send out the results as a part of BAT results:
> >     - even less noise than (1),
> >     - BAT results already feel cluttered, this may decrease readability.
> > 
> >  3. Have each check as a separate mail, but send it only if the check fails:
> >     - noisy: may result in many mails, depending how many checks fail,
> >     - easier to read and easier to follow on patchwork.
> 
> The best user experience I could think of;
> 
> 1. If all CI checks succeed, delay and only send one mail with all the
> results. This would indicate it's good to merge, go do it.
> 2. When a CI checks fail, immediately send that out so the developer
> gets to work on the fix.
> 
> Above requires that all the checks complete rather quickly and a trust
> is gained to the system so that the absence of e-mail always means the
> series is doing good, not that the system is clogged in some way :)

Or just 2. The first being the compilation report; saying we
have received your patch and it compiles fine, it will be queued to the
farm currently in slot N (or it doesn't even compile!). The second being
the success or failure of the CI run.

From the user pov, we can't do anything until the CI report so
intermediate emails saying congrats are just fluff. Useful simply to
know the patch hasn't fall out of the system, but not supplying any
actionable information.
-Chris