[Mesa-dev] [PATCH] RFC: Workaround for pthread_setaffinity_np() seccomp filtering

[Mathias Fröhlich]

Marek, Marc-Andre,

On Wednesday, 13 March 2019 00:03:26 CET Marek Olšák wrote:
> The env var workaround is fine.
> 
> Thread affinity is used for cache topology related optimizations. I think
> it's a mistake to treat it only as a resource allocation tool.

For a shorter term solution to the problem.
One Idea that comes into my mind:

Can we check the currently set thread affinity mask if it still contains the
cpu we are aiming for and narrow the mask down to our cpu if we can do
that by narrowing. If we would need to assign our thread to a cpu that
we are not bound anymore just do nothing.

That would obviously require that we can still call into pthread_setaffinity_np
without being just killed straight away because we touch something that
somebody else wants to control. And that we even succeed in just narrowing
down the allowed set of cpus.
Marc-Andre, would that still work with qemu then?

Of course this still leaves a small race condition open if somebody changes the
affinitiy mask of the current thread in between our call to pthread_getaffinity_np
and pthread_setaffinity_np from the outside of our linux task. Then we may
experience a non narrowing set affinity operation anymore because of an other set
operation that came in between and we may get killed then.
... which is an other argument against just killing. But ok ...
IMO this condition happens sufficiently seldom to accept that.

Could that solve our problem??

best
Mathias


> 
> Marek
> 
> On Tue, Mar 12, 2019, 1:59 AM Marc-André Lureau <marcandre.lureau at gmail.com>
> wrote:
> 
> > Hi
> >
> > On Fri, Mar 1, 2019 at 12:13 PM Mathias Fröhlich
> > <Mathias.Froehlich at gmx.net> wrote:
> > >
> > > On Friday, 1 March 2019 12:15:08 CET Eero Tamminen wrote:
> > > > Hi,
> > > >
> > > > On 1.3.2019 11.12, Michel Dänzer wrote:
> > > > > On 2019-02-28 8:41 p.m., Marek Olšák wrote:
> > > > >>> On Thu, Feb 28, 2019 at 1:37 PM Eero Tamminen <
> > eero.t.tamminen at intel.com>
> > > > >>>> Why distro versions of Qemu filter sched_setaffinity() syscall?
> > > > >>>
> > > > >>> (https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1815889)
> > > > >>>
> > > > >>> Daniel Berrange (berrange) wrote on 2019-02-27: #19
> > > > >>>
> > > > >>> "IMHO that mesa change is not valid. It is settings its affinity to
> > > > >>> run on all threads which is definitely *NOT* something we want to
> > be
> > > > >>> allowed. Management applications want to control which CPUs QEMU
> > runs
> > > > >>> on, and as such Mesa should honour the CPU placement that the QEMU
> > > > >>> process has.
> > > > >>>
> > > > >>> This is a great example of why QEMU wants to use seccomp to block
> > > > >>> affinity changes to prevent something silently trying to use more
> > CPUs
> > > > >>> than are assigned to this QEMU."
> > > > >>>
> > > > >>
> > > > >> Mesa uses thread affinity to optimize memory access performance on
> > some
> > > > >> CPUs (see util_pin_thread_to_L3). Other places in Mesa need to
> > restore the
> > > > >> original thread affinity for some child threads. Additionally, if
> > games
> > > > >> limit the thread affinity, Mesa needs to restore the full thread
> > affinity
> > > > >> for some of its child threads.
> > > > >
> > > > > The last part sounds like Mesa clearly overstepping its authority.
> > > > >
> > > > >
> > > > >> In essence, the thread affinity should only be considered a hint
> > for the
> > > > >> kernel for optimal performance. There is no reason to kill the
> > process if
> > > > >> it's disallowed. Just ignore the call or modify the thread mask to
> > make it
> > > > >> legal.
> > > > >
> > > > > The fundamental issue here is that Mesa is using the thread affinity
> > API
> > > > > for something else than it's intended for. If there was an API for
> > what
> > > > > Mesa wants (encouraging certain sets of threads to run on
> > topologically
> > > > > close cores), there should be no need to block that.
> > > >
> > > > Why such process needs to be killed instead the request being masked
> > > > suitably, is there some program that breaks subtly if affinity request
> > > > is masked (and that being worse than the program being killed)?
> > >
> > > But that is still a situation that could be nicely handled with a
> > > EPERM error return. Way better than just kill a process.
> > > That 'badly affected' program still can call abort then.
> > > But nicely working programs don't get just killed then!!
> >
> >
> > Returning an error seems less secure that prohibiting it completely.
> > And it may lead to subtle bugs in rarely tested code paths.
> >
> > It's legitimate that QEMU and management layers want to prevent
> > arbitrary code from changing resource allocation etc.
> >
> > There are no easy way I can think of for mesa (and other libraries) to
> > probe the seccomp filters and associated action.
> >
> > So we need a way to tell mesa not to call setaffinity() (and other
> > syscalls). MESA_NO_THREAD_AFFINITY or MESA_NO_SYSCALLS=setaffinity,...
> > seem like a relatively easy way to go.
> >
> > thanks
> >
> >
> > --
> > Marc-André Lureau
> > _______________________________________________
> > mesa-dev mailing list
> > mesa-dev at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/mesa-dev
>