[Intel-gfx] [QUERY] How many CI mails is too many?

[Joonas Lahtinen]

On Mon, 2017-11-27 at 16:54 +0200, Arkadiusz Hiler wrote:
> Hey all,
> 
> For some time already CI sends out 1-2 mails per series per (re)run, i.e. BAT
> results and "full IGT" results (if BAT has not failed).
> 
> Recently we have added 32bit build check, and if that fails it sends out
> additional mail In-Reply-To the series.
> 
> I am working on adding some static checks to the CI (spare and checkpatch at the
> moment, more may come in the future), which may generate even more commotion on
> the mailing list.
> 
> How much of CI noise is too much and how you would like to have the results
> grouped?
> 
> Couple of options to start the discussion:
> 
>  1. Group all static checks (and the 32bit build?) into one mail:
>     - just one additional mail,
>     - may be hard to read in case of catastrophic failure,
>     - we can send it only when something actually fails.
> 
>  2. Send out the results as a part of BAT results:
>     - even less noise than (1),
>     - BAT results already feel cluttered, this may decrease readability.
> 
>  3. Have each check as a separate mail, but send it only if the check fails:
>     - noisy: may result in many mails, depending how many checks fail,
>     - easier to read and easier to follow on patchwork.

The best user experience I could think of;

1. If all CI checks succeed, delay and only send one mail with all the
results. This would indicate it's good to merge, go do it.
2. When a CI checks fail, immediately send that out so the developer
gets to work on the fix.

Above requires that all the checks complete rather quickly and a trust
is gained to the system so that the absence of e-mail always means the
series is doing good, not that the system is clogged in some way :)

Regards, Joonas

> 
> Any opinions? Any other ideas?
> 
-- 
Joonas Lahtinen
Open Source Technology Center
Intel Corporation