[Mesa-dev] [PATCH] RFC: Workaround for pthread_setaffinity_np() seccomp filtering

[Mathias Fröhlich]

Hi,

On Tuesday, 12 March 2019 09:59:17 CET Marc-André Lureau wrote:
> Hi
> 
> On Fri, Mar 1, 2019 at 12:13 PM Mathias Fröhlich
> <Mathias.Froehlich at gmx.net> wrote:
> >
> > On Friday, 1 March 2019 12:15:08 CET Eero Tamminen wrote:
> > > Hi,
> > >
> > > On 1.3.2019 11.12, Michel Dänzer wrote:
> > > > On 2019-02-28 8:41 p.m., Marek Olšák wrote:
> > > >>> On Thu, Feb 28, 2019 at 1:37 PM Eero Tamminen <eero.t.tamminen at intel.com>
> > > >>>> Why distro versions of Qemu filter sched_setaffinity() syscall?
> > > >>>
> > > >>> (https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1815889)
> > > >>>
> > > >>> Daniel Berrange (berrange) wrote on 2019-02-27: #19
> > > >>>
> > > >>> "IMHO that mesa change is not valid. It is settings its affinity to
> > > >>> run on all threads which is definitely *NOT* something we want to be
> > > >>> allowed. Management applications want to control which CPUs QEMU runs
> > > >>> on, and as such Mesa should honour the CPU placement that the QEMU
> > > >>> process has.
> > > >>>
> > > >>> This is a great example of why QEMU wants to use seccomp to block
> > > >>> affinity changes to prevent something silently trying to use more CPUs
> > > >>> than are assigned to this QEMU."
> > > >>>
> > > >>
> > > >> Mesa uses thread affinity to optimize memory access performance on some
> > > >> CPUs (see util_pin_thread_to_L3). Other places in Mesa need to restore the
> > > >> original thread affinity for some child threads. Additionally, if games
> > > >> limit the thread affinity, Mesa needs to restore the full thread affinity
> > > >> for some of its child threads.
> > > >
> > > > The last part sounds like Mesa clearly overstepping its authority.
> > > >
> > > >
> > > >> In essence, the thread affinity should only be considered a hint for the
> > > >> kernel for optimal performance. There is no reason to kill the process if
> > > >> it's disallowed. Just ignore the call or modify the thread mask to make it
> > > >> legal.
> > > >
> > > > The fundamental issue here is that Mesa is using the thread affinity API
> > > > for something else than it's intended for. If there was an API for what
> > > > Mesa wants (encouraging certain sets of threads to run on topologically
> > > > close cores), there should be no need to block that.
> > >
> > > Why such process needs to be killed instead the request being masked
> > > suitably, is there some program that breaks subtly if affinity request
> > > is masked (and that being worse than the program being killed)?
> >
> > But that is still a situation that could be nicely handled with a
> > EPERM error return. Way better than just kill a process.
> > That 'badly affected' program still can call abort then.
> > But nicely working programs don't get just killed then!!
> 
> 
> Returning an error seems less secure that prohibiting it completely.
> And it may lead to subtle bugs in rarely tested code paths.
> 
> It's legitimate that QEMU and management layers want to prevent
> arbitrary code from changing resource allocation etc.

I *never* saw this api as resource allocation.

Such a call finally dates back into the IRIX threads (the api *before* pthreads
on those very old OpenGL dinosaurs from Silicon Graphics) where this was pretty
much used in contexts like exactly what Marek wanted to.
To make use hardware topology that you can access specific hardware faster from
specific cpu's. Or aiming for cache locality between threads that exchange lots of
data between each other. Think of high performance computing applications for
cache locality or VR application middle end libraries for hardware OpenGL access.
That api was replaced on that ancient operating system by pthreads that contained
the equivalent api call. And later on the linux pthread implementation gained the
equivalent pthread_setaffinity_np call when SMP linux systems got used more often.
Means if you just kill an application that tries to optimize for valid uses using an API
that used to work for that purpose for years you will just break existing API's.

Beside breaking exiting behavior, just think of what you are doing from an applications
view. I think as an application writer now I believe that I want to change this property. Now I
know that if I touch that specific property I may just be dead. So, then I want to know if I can
touch this property without being dead. But there is no such way to find that out.
Well, then this means basically the api is finally unusable because I can't tolerate that
I am just dead when trying to change something for good!!!
What you want as an application writer is to change that value and if that does not work
handle that accordingly. Where 'handle that' can be ranging from, but is not limited to:
Either silently internally handle that in the application, may be use an other algorithm that fits that case. 
Present the user some user interface message that you need to bind threads to cpus and the
operating system does not allow that and that re configuring the operating system by the
administrator may help.

I think there is lots of ways to control resource allocation beside taking an API call that is used
in a different way up to now and then just killing processes that call that API function.

I am sorry, but I think you shall not just kill processes then.

best
Mathias

> 
> There are no easy way I can think of for mesa (and other libraries) to
> probe the seccomp filters and associated action.
> 
> So we need a way to tell mesa not to call setaffinity() (and other
> syscalls). MESA_NO_THREAD_AFFINITY or MESA_NO_SYSCALLS=setaffinity,...
> seem like a relatively easy way to go.
> 
> thanks
> 
> 
>