[Mesa-dev] [PATCH] RFC: Workaround for pthread_setaffinity_np() seccomp filtering

Wed Mar 13 13:29:28 UTC 2019

Hi

On Wed, Mar 13, 2019 at 8:53 AM Mathias Fröhlich
<Mathias.Froehlich at gmx.net> wrote:
>
> Marek, Marc-Andre,
>
> On Wednesday, 13 March 2019 00:03:26 CET Marek Olšák wrote:
> > The env var workaround is fine.
> >
> > Thread affinity is used for cache topology related optimizations. I think
> > it's a mistake to treat it only as a resource allocation tool.
>
> For a shorter term solution to the problem.
> One Idea that comes into my mind:
>
> Can we check the currently set thread affinity mask if it still contains the
> cpu we are aiming for and narrow the mask down to our cpu if we can do
> that by narrowing. If we would need to assign our thread to a cpu that
> we are not bound anymore just do nothing.
>

getaffinity() is also blocked by current qemu policy.

It should be possible to allow a narrower setaffinity(), with some complex rule.

> That would obviously require that we can still call into pthread_setaffinity_np
> without being just killed straight away because we touch something that
> somebody else wants to control. And that we even succeed in just narrowing
> down the allowed set of cpus.
> Marc-Andre, would that still work with qemu then?

For now, Daniel proposed "seccomp: don't kill process for resource
control syscalls": the resource control syscalls will return -1/EPERM.

>
> Of course this still leaves a small race condition open if somebody changes the
> affinitiy mask of the current thread in between our call to pthread_getaffinity_np
> and pthread_setaffinity_np from the outside of our linux task. Then we may
> experience a non narrowing set affinity operation anymore because of an other set
> operation that came in between and we may get killed then.
> ... which is an other argument against just killing. But ok ...
> IMO this condition happens sufficiently seldom to accept that.
>
> Could that solve our problem??
>
> best
> Mathias
>
>
> >
> > Marek
> >
> > On Tue, Mar 12, 2019, 1:59 AM Marc-André Lureau <marcandre.lureau at gmail.com>
> > wrote:
> >
> > > Hi
> > >
> > > On Fri, Mar 1, 2019 at 12:13 PM Mathias Fröhlich
> > > <Mathias.Froehlich at gmx.net> wrote:
> > > >
> > > > On Friday, 1 March 2019 12:15:08 CET Eero Tamminen wrote:
> > > > > Hi,
> > > > >
> > > > > On 1.3.2019 11.12, Michel Dänzer wrote:
> > > > > > On 2019-02-28 8:41 p.m., Marek Olšák wrote:
> > > > > >>> On Thu, Feb 28, 2019 at 1:37 PM Eero Tamminen <
> > > eero.t.tamminen at intel.com>
> > > > > >>>> Why distro versions of Qemu filter sched_setaffinity() syscall?
> > > > > >>>
> > > > > >>> (https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1815889)
> > > > > >>>
> > > > > >>> Daniel Berrange (berrange) wrote on 2019-02-27: #19
> > > > > >>>
> > > > > >>> "IMHO that mesa change is not valid. It is settings its affinity to
> > > > > >>> run on all threads which is definitely *NOT* something we want to
> > > be
> > > > > >>> allowed. Management applications want to control which CPUs QEMU
> > > runs
> > > > > >>> on, and as such Mesa should honour the CPU placement that the QEMU
> > > > > >>> process has.
> > > > > >>>
> > > > > >>> This is a great example of why QEMU wants to use seccomp to block
> > > > > >>> affinity changes to prevent something silently trying to use more
> > > CPUs
> > > > > >>> than are assigned to this QEMU."
> > > > > >>>
> > > > > >>
> > > > > >> Mesa uses thread affinity to optimize memory access performance on
> > > some
> > > > > >> CPUs (see util_pin_thread_to_L3). Other places in Mesa need to
> > > restore the
> > > > > >> original thread affinity for some child threads. Additionally, if
> > > games
> > > > > >> limit the thread affinity, Mesa needs to restore the full thread
> > > affinity
> > > > > >> for some of its child threads.
> > > > > >
> > > > > > The last part sounds like Mesa clearly overstepping its authority.
> > > > > >
> > > > > >
> > > > > >> In essence, the thread affinity should only be considered a hint
> > > for the
> > > > > >> kernel for optimal performance. There is no reason to kill the
> > > process if
> > > > > >> it's disallowed. Just ignore the call or modify the thread mask to
> > > make it
> > > > > >> legal.
> > > > > >
> > > > > > The fundamental issue here is that Mesa is using the thread affinity
> > > API
> > > > > > for something else than it's intended for. If there was an API for
> > > what
> > > > > > Mesa wants (encouraging certain sets of threads to run on
> > > topologically
> > > > > > close cores), there should be no need to block that.
> > > > >
> > > > > Why such process needs to be killed instead the request being masked
> > > > > suitably, is there some program that breaks subtly if affinity request
> > > > > is masked (and that being worse than the program being killed)?
> > > >
> > > > But that is still a situation that could be nicely handled with a
> > > > EPERM error return. Way better than just kill a process.
> > > > That 'badly affected' program still can call abort then.
> > > > But nicely working programs don't get just killed then!!
> > >
> > >
> > > Returning an error seems less secure that prohibiting it completely.
> > > And it may lead to subtle bugs in rarely tested code paths.
> > >
> > > It's legitimate that QEMU and management layers want to prevent
> > > arbitrary code from changing resource allocation etc.
> > >
> > > There are no easy way I can think of for mesa (and other libraries) to
> > > probe the seccomp filters and associated action.
> > >
> > > So we need a way to tell mesa not to call setaffinity() (and other
> > > syscalls). MESA_NO_THREAD_AFFINITY or MESA_NO_SYSCALLS=setaffinity,...
> > > seem like a relatively easy way to go.
> > >
> > > thanks
> > >
> > >
> > > --
> > > Marc-André Lureau
> > > _______________________________________________
> > > mesa-dev mailing list
> > > mesa-dev at lists.freedesktop.org
> > > https://lists.freedesktop.org/mailman/listinfo/mesa-dev
> >
>
>
>
>

-- 
Marc-André Lureau