[igt-dev] [PATCH i-g-t] intel-ci: add a pre-merge blacklist to reduce the testing queue
Petri Latvala
petri.latvala at intel.com
Fri Feb 21 11:27:30 UTC 2020
On Fri, Feb 21, 2020 at 12:20:33PM +0200, Petri Latvala wrote:
> On Thu, Feb 20, 2020 at 05:32:09PM +0200, Martin Peres wrote:
> > When arriving at the office on Monday morning, the reported queue
> > size was ~100 hours. This defeats the point of pre-merge testing and
> > vastly exceeds our target of ~6 hours.
> >
> > We have a lot of work needed to reduce testing time, but this patches
> > reduces the reported run time by 15-30% depending on the platforms:
> >
> > - shard-skl: 23.9 -> 18.2 minutes (18.5%)
> > - shard-kbl: 21.2 -> 16.2 minutes (20%)
> > - shard-apl: 25.9 -> 18.5 minutes (24.3%)
> > - shard-glk: 24.7 -> 17.6 minutes (24.8%)
> > - shard-icl: 25.1 -> 16.7 minutes (28.7%)
> > - shard-tgl: 28.2 -> 19.6 minutes (26.4%)
> >
> > The reason why the reported runtime is so low compared to the
> > actual time is due to:
> >
> > - Unaccounted time spent outside of the IGT subtests (exec(), fixtures)
> > - Unaccounted time spent in suspend (monotonic clock, 20s / suspend)
> > - Boot time / extra reboots between shards to workaround kernel failures
> > - Intel GFX CI shard scheduling overhead
> > - More?
> >
> > Tomi and Petri are working on reducing these overheads by detecting the
> > bad conditions and rebooting the machine only at this point rather than
> > between every single shard, and increasing the size of the shard test
> > lists to reduce the per-shard CI overhead.
> >
> > Because of this, the actual savings are way smaller in percentage
> > but still compound over the tens of executions we do per week:
> >
> > - shard-skl: ~58 -> ~52 minutes
> > - shard-kbl: ~50 -> ~45 minutes
> > - shard-apl: ~53 -> ~46 minutes
> > - shard-glk: ~38 -> ~31 minutes
> > - shard-icl: ~47 -> ~39 minutes
> > - shard-tgl: ~60 -> ~51 minutes
> >
> > More work needed, but we'll get there :)
> >
> > Signed-off-by: Martin Peres <martin.peres at linux.intel.com>
> > ---
> > tests/intel-ci/README | 7 +
> > tests/intel-ci/blacklist-pre-merge.txt | 221 +++++++++++++++++++++++++
>
>
> With a drive-by addition of a forgotten blacklist.txt, squash this in:
>
> diff --git a/tests/intel-ci/meson.build b/tests/intel-ci/meson.build
> index b3abb848..62811e92 100644
> --- a/tests/intel-ci/meson.build
> +++ b/tests/intel-ci/meson.build
> @@ -3,6 +3,8 @@ intelci_files = [
> 'fast-feedback.testlist',
> 'meta.testlist',
> 'README',
> + 'blacklist.txt',
> + 'blacklist-pre-merge.txt',
> ]
>
> install_data(sources : intelci_files, install_dir : datadir)
With Martin's IRL ack to squashing this, the patch is now merged.
--
Petri Latvala
More information about the igt-dev
mailing list