[Mesa-dev] [PATCH] gallium/util: don't let children of fork & exec inherit our thread affinity

[Marek Olšák]

On Tue, Oct 30, 2018 at 6:32 PM Gustaw Smolarczyk <wielkiegie at gmail.com>
wrote:

> wt., 30 paź 2018, 23:01 Marek Olšák <maraeo at gmail.com>:
>
>> On Mon, Oct 29, 2018 at 12:43 PM Michel Dänzer <michel at daenzer.net>
>> wrote:
>>
>>> On 2018-10-28 11:27 a.m., Gustaw Smolarczyk wrote:
>>> > pon., 17 wrz 2018 o 18:24 Michel Dänzer <michel at daenzer.net>
>>> napisał(a):
>>> >>
>>> >> On 2018-09-15 3:04 a.m., Marek Olšák wrote:
>>> >>> On Fri, Sep 14, 2018 at 4:53 AM, Michel Dänzer <michel at daenzer.net>
>>> wrote:
>>> >>>>
>>> >>>> Last but not least, this doesn't solve the issue of apps such as
>>> >>>> blender, which spawn their own worker threads after initializing
>>> OpenGL
>>> >>>> (possibly not themselves directly, but via the toolkit or another
>>> >>>> library; e.g. GTK+4 uses OpenGL by default), inheriting the thread
>>> affinity.
>>> >>>>
>>> >>>>
>>> >>>> Due to these issues, setting the thread affinity needs to be
>>> disabled by
>>> >>>> default, and only white-listed for applications where it's known
>>> safe
>>> >>>> and beneficial. This sucks, but I'm afraid that's the reality until
>>> >>>> there's better API available which allows solving these issues.
>>> >>>
>>> >>> We don't have the bandwidth to maintain whitelists. This will either
>>> >>> have to be always on or always off.
>>> >>>
>>> >>> On the positive side, only Ryzens with multiple CCXs get all the
>>> >>> benefits and disadvantages.
>>> >>
>>> >> In other words, only people who spent relatively large amounts of
>>> money
>>> >> for relatively high-end CPUs will be affected (I'm sure they'll be
>>> glad
>>> >> to know that "common people" aren't affected. ;). Affected
>>> applications
>>> >> will see their performance decreased by a factor of 2-8 (the number of
>>> >> CCXs in the CPU).
>>> >>
>>> >> OTOH, only a relatively small number of games will get a significant
>>> >> benefit from the thread affinity, and the benefit will be smaller
>>> than a
>>> >> factor of 2. This cannot justify risking a performance drop of up to a
>>> >> factor of 8, no matter how small the risk.
>>> >>
>>> >> Therefore, the appropriate mechanism is a whitelist.
>>> >
>>> > Hi,
>>> >
>>> > What was the conclusion of this discussion? I don't see any
>>> > whitelist/blacklist for this feature.
>>> >
>>> > I have just tested blender and it still renders on only a single CCX
>>> > on mesa from git master. Also, there is a bug report that suggests
>>> > this regressed performance in at least one game [1].
>>>
>>> I hooked up that bug report to the 18.3 blocker bug.
>>>
>>>
>>> > If you think enabling it by default is the way to go, we should also
>>> > implement a blacklist so that it can be turned off in such cases.
>>>
>>> I stand by my opinion that a white-list is appropriate, not a
>>> black-list. It's pretty much the same as mesa_glthread.
>>>
>>
>> So you are saying that gallium multithreading show be slower than
>> singlethreading by default.
>>
>> Marek
>>
>
> Hi Marek,
>
> The Ryzen optimization helps a lot of applications (mostly games) and
> improves their performance, mostly because of the reduced cost of
> communication between application's GL API thread and driver's pipe/winsys
> threads.
>
> However, not all of the applications respond in the same way. The thread
> affinity management is hacky, by which I mean that this mechanism was not
> meant to mess with application threads from within library's threads. As an
> example, blender's threads, which use OpenGL "by accident", are forced to
> use the same CCX as the main gallium/winsys thread, even if they are many
> and want to work on as many CCXs as are possible. The thread that starts
> using GL spawns many more threads that don't use GL at all, and the current
> atfork mechanism doesn't help.
>
> The current mechanism of tweaking thread affinities doesn't work
> universally with all Linux applications. We need a mechanism of tweaking
> this behavior, either through a whitelist or through a blacklist. As any
> application using OpenGL can be affected, I would opt towards disabling
> this by default and providing a whitelist for applications we know it would
> help.
>

Multithreading is slower than singlethreading. You are pretty much saying
that Mesa shouldn't use multithreading by default. Just think about that.
NVIDIA would destroy us at all price points. I'm not that insane to disable
the thread pinning by default.

The thread affinity API is very bad for this, but it's the only thing we
have. Linux lacks a proper thread management API for the Zen architecture
and the Linux scheduler does the worst thing for Ryzen (it puts threads on
different CCXs), so the scheduler always works against us. It makes Ryzen
as slow as possible.

Only Blender is affected negatively and there is a patch for it. Blender is
open source, so it can just reset the thread affinity for new threads by
itself, or set a thread affinity that works best with Ryzen. Mesa can
contain the workaround out of courtesy.

Marek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20181030/656a1511/attachment-0001.html>