[Mesa-dev] [RFC] pipe-loader: abstract GALLIUM_STATIC_TARGETS behind pipe_loader API

[Rob Clark]

On Wed, Sep 30, 2015 at 7:37 PM, Marek Olšák <maraeo at gmail.com> wrote:
> On Wed, Sep 30, 2015 at 10:43 PM, Rob Clark <robdclark at gmail.com> wrote:
>> From: Rob Clark <robclark at freedesktop.org>
>>
>> Not actually working yet, ie. doesn't even compile yet, but an idea.
>>
>> Initial motivation was for drm_gralloc/pipe, which is essentially a sort
>> of mini state-tracker, that needs to be able to share pipe_screen with
>> libGL linked into the same process (to ensure we didn't end up with
>> duplicate pipe_resource's, etc).  I think same situation happens with
>> vdpau+gl interop.
>>
>> Currently drm_gralloc/pipe (and other state trackers??) statically link
>> the winsys code, which completely defeats the purpose magic in the
>> winsys code to detect when the same device is opened multiple times and
>> return the same pipe_screen (since you end up w/ multiple copies of the
>> hashtable in the same process).  See for example:
>>
>>   5bb41d9094b3c9bdf0669fd55418981ed83347e3
>>   fee0686c21c631d96d6042741267a3c218c23ffc
>>
>> Rough idea is that we should have something like libgallium.so which
>> contains the pipe-loader API, and then optionally (depending on
>> GALLIUM_STATIC_TARGETS) all of the gallium pipe drivers.  The various
>> different state trackers would link against (or dlopen()) libgallium.so
>> and use the pipe-loader API to get themselves a pipe_screen (and config
>> params, etc).  And then you end up with:
>>
>>                 +---+
>>                 | l |
>>     clover  --> | i |
>>                 | b |
>>     mesa-st --> | g |
>>                 | a |--> pipe driver
>>     vdapau  --> | l |
>>                 | l |
>>     gralloc --> | i |
>>                 | u |
>>     xa      --> | m |
>>                 |   |
>>                 +---+
>> or:
>>                 +---+-------------+
>>                 | l |             |
>>     clover  --> | i |             |
>>                 | b |             |
>>     mesa-st --> | g |             |
>>                 | a | pipe driver |
>>     vdapau  --> | l |             |
>>                 | l |             |
>>     gralloc --> | i |             |
>>                 | u |             |
>>     xa      --> | m |             |
>>                 |   |             |
>>                 +---+-------------+
>>
>> depending on GALLIUM_STATIC_TARGETS.  Either way, all the state trackers
>> in the same process share a single copy of the hashtable in the winsys
>> code which allows them to share the same pipe_screen.
>>
>> I think that ends up being an extra level of library indirection vs
>> current state w/ pipe drivers, ie. with mesa dri loader stuff directly
>> loading gallium_dri.so which contains all the drivers plus mesa state
>> tracker.  If this was concerning to someone, what I'd suggest would be
>> to, for all the state trackers that already have some sort of loader
>> sitting between them and the user, just pull them directly into the
>> mega-mega libgallium.so, ie. something like:
>>
>>       +---------+---+-------------+
>>       |         |   |             |
>>       | clover  | l |             |
>>       |         | i |             |
>>       | mesa-st | b |             |
>>       |         | g | pipe driver |
>>       | vdapau  | a |             |
>>       |         | l |             |
>>       +---------| l |             |
>>                 | i |             |
>>     gralloc --> | u |             |
>>                 | m |             |
>>     xa      --> |   |             |
>>                 |   |             |
>>                 +---+-------------+
>>
>> Anyways, I'm far from an expert in the build architecture of things so
>> I might have some facts wrong or be a bit confused.  And I'll probably
>> regret bringing the subject up.  But somehow or another it seems like
>> it would be good to (a) clean up all the GALLIUM_STATIC_TARGETS
>> ifdeffery spread throughout all the different state trackers, and (b)
>> have the pipe driver[*] only exist once per process rather than once per
>> state-tracker.  Especially for android where each process using GL will
>> have both gralloc and mesa-st.. and perhaps even clover/omx/etc.
>
> Radeon and amdgpu already have one pipe_screen per process per fd. It
> works like this:
>
> - The GL, VDPAU, OMX etc. libs each ship its own copy of the whole
> driver, so we have multiple screens and multiple winsyses.
>
> - All libs export "radeon_drm_winsys_create", which is this function:
> struct radeon_winsys *radeon_drm_winsys_create(
>    int fd, radeon_screen_create_t screen_create);
>
> - The first lib that is loaded registers "radeon_drm_winsys_create",
> all other libs loaded later can't override it. Therefore, if any lib
> calls radeon_drm_winsys_create, the version from the first loaded lib
> is called.
>
> - The first caller passes the screen_create function to
> radeon_drm_winsys_create, the screen is created only once for each
> input fd in that function along with the winsys. When the winsys is
> returned, radeon_winsys::screen is the returned screen.
>
> - Any other call to radeon_drm_winsys_create will return the same
> winsys+screen combo for that fd. (it also needs to recognize dup()
> fds, etc.)
>
> - Of course, all libs must use the same version of the driver.
>
> - The winsys is reference-counted. pipe_screen::destroy doesn't
> destroy itself until the winsys counter drops to 0.

Yes, I've seen how radeon and nouveau implement that, and I've done
something similar.  But it relies on some linker script stuff to
ensure there is only one xyz_drm_winsys_create(), which I couldn't get
to work properly on android (which has it's own dynamic loader).  Not
sure to what extent that was an issue w/ bionic dl loader vs me
screwing something up in android build system or something.  I do know
at some point recently the bionic dl loader broke weak symbols, and a
few years ago didn't even support LD_PRELOAD, so I'm not inclined to
trust it too much with fancy stuff.

Having a single gallium lib that is dynamically linked into all it's
users seems like a simple solution that is guaranteed to work with the
most brain-dead of dynamic loaders.

(But that said, I think the first time the mega-loader stuff came up,
I was too busy trying to draw triangles and didn't pay much attention
to the discussions or reasons things are like they are today)

> Alternative solution other than the gallium lib:
> - use a winsys lib or move the winsys into libdrm

separate winsys lib is an interesting possibility..

> Indeed, libdrm_amdgpu only creates one instance of amdgpu_device per
> process per fd.

I had considered that.. but we have additional resource tracking in
the pipe driver, since tilers try to be really lazy about submitting
work to the kernel/hw..

BR,
-R

> Marek