[igt-dev] Running [some/most of] IGT as non-root (Was: Re: [PATCH i-g-t] tests/core_unauth_vs_render: new test for the relaxed DRM_AUTH handling)

[Daniel Vetter]

On Wed, Jan 23, 2019 at 5:28 PM Emil Velikov <emil.l.velikov at gmail.com> wrote:
>
> Hi all,
>
> On Wed, 23 Jan 2019 at 11:18, Daniel Vetter <daniel at ffwll.ch> wrote:
>
> > > > > + * By default IGT is executed as root.
> > > >
> > > > It's not just the default, it's a hard requirement. The runner has checks
> > > > for other drm clients and whether you're root and bails out if that's not
> > > > the case. Lots&lots of igts break if you run them as non-root or with
> > > > other drm clients running. Only thing that's allowed is enumerating
> > > > subtests (because we need that on our build machines to generate the
> > > > testlists).
> > > >
> > > > tldr; Please fold in.
> > > >
> > > Sure will fold.
> > >
> > > Modulo an exception or two, IGT tests only require debugfs (write perm
> > > for i915) and user in the video group (to open the node).
> > > That's based on a quick look/test.
> > >
> > > Since few distributions don't run the display server and/or compositor
> > > as root, ideally IGT will get root-free at some point.
> > > Ake IGT testing will be closer to real world use-cases.
> >
> > IGT isnt like piglit, most of the tests check for corner cases where you
> > really can't have anything else touching the display (e.g. anything kms).
> > Being root is the easiest way to check for that (in debugfs, if there's
> > any other drm client the igt runner complains&quits). I guess there's some
> > igts which can be run like piglits, but those are the rare exceptions.
> >
> > Now the entire "run compositor as non-root" business is a good point. We
> > do have some igts that drop CAP_SYS_ADMIN (see igt_drop_root()), but none
> > that make sure this holds for kms/core functionality. It's just a few
> > i915-gem tests. Would be a good gap to fill though. I think for the
> > non-rendernode EACCESS test we'll need that (since root can do whatever).
> >
>
> As explained by Daniel above, some tests require that:
>  - they are the sole client
>  - use generic [kms] functionality, which in theory don't need to be ran as root
>  - test [kms] corner-cases, that need(?) to be ran as root
> ** I'd imagine the above is also applicable for vendor specific tests.
>
> Thus I wonder what people think of the following plan:
>  - add a script that toggles the debugfs permissions - since most
> tests require +rx (and some +w) permissions
>  - add a root vs non-root and non-concurrent vs concurrent notations
> to the tests
> For example piglit has been doing the last one for years.
>  - have the IGT runner honour and enforce the notations
>  - new tests default to non-root and concurrent and existing ones are
> as marked the opposite
>  - audit and fix existing tests to become non-root & concurrent, if possible
>  - document ^^
>
> Currently soliciting input on this rather crazy area, nothing more.
> If it's something to aim towards, please copy it to the IGT TODO list,
> since I'm not sure where it lives these days.
>
> What do you guys think?

I entertained this, and we do have a bunch of tests that probably
could be run in parallel. But if you look at the runtime, they just
don't matter. The only ones you can run in parallel are rendernode
interface tests that don't hog system resources. That's just ioctl
validation tests mostly, and those already run really fast. For the
intel CI we even have ideas to make those run faster without any
changes to the tests, but didn't bother since not worth it. CI cost is
dominated by everything else.

All the tests that burn down real system time can't be parallelized
(anything kms, any of the gem stress tests, anything that hangs the
gpu, anything requiring master). So given the massive amounts of
effort and the minimal payoff, I don't think this is worth it.

piglit (and mesa testing in general) is completely different:
- Lots of time spent wasting cpu cycles compiling shaders and other stuff.
- The actual workloads run on the gpu are tiny - mostly just 1 shader
painting a few pixels.
- No tests for anything global, mesa testing assumes the gpu never
hangs, display is managed by the compositor (which takes care of
switching windows if that's even necessary, for winsys stuff)

So perfect workload that scales by adding more cores. igt aims to fill
all the gaps here (we do run the quick tests in piglit as part of our
kernel pre-merge CI, just almost never fails), anything that can be
run in parallel is probably better done as a piglit/mesa test. That
also means the default is never going to be "parallel" tests, or
defacto everyone just labels their tests as single-threaded.

tldr; before we do anything here we need an estimate of how much this
will help, and then make a call whether that's worth it.

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch