minutes of ESC call ...

[Stephan Bergmann]

On 09/05/2019 16:46, Michael Meeks wrote:
> * Jenkins / CI update (Christian)
>      from:Thu May  2 13:58:26 2019
>      master linux rel  jobs: 129 ok: 129 ko:   0 fail ratio:  0.00 % break:   0 broken duration: 0.00%
>      master linux dbg  jobs:  86 ok:  83 ko:   3 fail ratio:  3.49 % break:   2 broken duration: 4.03%
>      master mac rel    jobs:  75 ok:  70 ko:   5 fail ratio:  6.67 % break:   4 broken duration: 5.67%
>      master mac dbg    jobs:  69 ok:  62 ko:   7 fail ratio: 10.14 % break:   4 broken duration:12.32%
>      master win rel    jobs:  86 ok:  63 ko:  23 fail ratio: 26.74 % break:  17 broken duration:23.09%
>      master win dbg    jobs:  90 ok:  75 ko:  15 fail ratio: 16.67 % break:  12 broken duration:12.80%
>      master win64 dbg  jobs:  88 ok:  67 ko:  21 fail ratio: 23.86 % break:  16 broken duration:25.91%
>      lo-5.3 mac        jobs:   0 ok:   0 ko:   0 fail ratio:  0.00 % break:   0 broken duration: 0.00%
>      lo-5.4 mac        jobs:   0 ok:   0 ko:   0 fail ratio:  0.00 % break:   0 broken duration: 0.00%
>      master gerrit lin jobs: 451 ok: 331 ko:  21 fail ratio:  4.66% time for ok: mean:  21 median:  21
>      master gerrit plg jobs: 462 ok: 297 ko:  92 fail ratio: 19.91% time for ok: mean:  36 median:  33
>      master gerrit win jobs: 498 ok: 292 ko:  75 fail ratio: 15.06% time for ok: mean:  65 median:  64
>      master gerrit mac jobs: 478 ok: 291 ko:  74 fail ratio: 15.48% time for ok: mean:  62 median:  37
>      master gerrit all jobs: 434 ok: 240 ko: 165 fail ratio: 38.02% time for ok: mean: 120 median:  85
>     + https://dev-www.libreoffice.org/tmp/gerrit_190509.html
>     + notable test failures in last seven days:
>        4 JunitTest_forms_unoapi_2
>        5 UITest_conditional_format
>        7 CppunitTest_sc_bugfix_test
>       11 UITest_writer_tests5
>     + dis-connected bots over the weekend
>     + one failing mac killed a lot of builds until disabled
>     + 2 Mac Pros dis-connected, one mis-behaving, one in power-save state ?
>        + right now – 1 Mac short.
>        + 1 is at Norbert’s company – needs an ask for a reboot, or re-route
>     + could we get another Mac Pro for hosting ? (Thorsten)
>        + hard to host – but no mounting for racks (Christian)
>           + newer Mac Minis are perhaps suitable.
>        + Macs shouldn’t be the bottleneck anyway
>        + Windows is the limiting factor

Whenever I recently happened to look at Gerrit/Jenkins builds that 
hadn't finished yet, it was the Mac build that was still outstanding, so 
I think we do have a bottleneck there after all.

Witness:

* My "May 16 22:51" comment at 
<https://gerrit.libreoffice.org/#/c/72433/>: "gerrit_mac is hopelessly 
overloaded right now, with a 30+ long build queue (waiting builds #33071 
through #33095)" (though that's rather 20+ than 30+, of course)

* Armin's "May 16 23:32" comment at 
<https://gerrit.libreoffice.org/#/c/72401/>: "sooo... Build is blocked 
prob due to Mac missing - due to no Macs available...?"