[Mesa-dev] [RFC] pipe-loader: abstract GALLIUM_STATIC_TARGETS behind pipe_loader API

[Marek Olšák]

On Thu, Oct 1, 2015 at 3:53 PM, Rob Clark <robdclark at gmail.com> wrote:
> On Wed, Sep 30, 2015 at 7:37 PM, Marek Olšák <maraeo at gmail.com> wrote:
>> On Wed, Sep 30, 2015 at 10:43 PM, Rob Clark <robdclark at gmail.com> wrote:
>>> From: Rob Clark <robclark at freedesktop.org>
>>>
>>> Not actually working yet, ie. doesn't even compile yet, but an idea.
>>>
>>> Initial motivation was for drm_gralloc/pipe, which is essentially a sort
>>> of mini state-tracker, that needs to be able to share pipe_screen with
>>> libGL linked into the same process (to ensure we didn't end up with
>>> duplicate pipe_resource's, etc).  I think same situation happens with
>>> vdpau+gl interop.
>>>
>>> Currently drm_gralloc/pipe (and other state trackers??) statically link
>>> the winsys code, which completely defeats the purpose magic in the
>>> winsys code to detect when the same device is opened multiple times and
>>> return the same pipe_screen (since you end up w/ multiple copies of the
>>> hashtable in the same process).  See for example:
>>>
>>>   5bb41d9094b3c9bdf0669fd55418981ed83347e3
>>>   fee0686c21c631d96d6042741267a3c218c23ffc
>>>
>>> Rough idea is that we should have something like libgallium.so which
>>> contains the pipe-loader API, and then optionally (depending on
>>> GALLIUM_STATIC_TARGETS) all of the gallium pipe drivers.  The various
>>> different state trackers would link against (or dlopen()) libgallium.so
>>> and use the pipe-loader API to get themselves a pipe_screen (and config
>>> params, etc).  And then you end up with:
>>>
>>>                 +---+
>>>                 | l |
>>>     clover  --> | i |
>>>                 | b |
>>>     mesa-st --> | g |
>>>                 | a |--> pipe driver
>>>     vdapau  --> | l |
>>>                 | l |
>>>     gralloc --> | i |
>>>                 | u |
>>>     xa      --> | m |
>>>                 |   |
>>>                 +---+
>>> or:
>>>                 +---+-------------+
>>>                 | l |             |
>>>     clover  --> | i |             |
>>>                 | b |             |
>>>     mesa-st --> | g |             |
>>>                 | a | pipe driver |
>>>     vdapau  --> | l |             |
>>>                 | l |             |
>>>     gralloc --> | i |             |
>>>                 | u |             |
>>>     xa      --> | m |             |
>>>                 |   |             |
>>>                 +---+-------------+
>>>
>>> depending on GALLIUM_STATIC_TARGETS.  Either way, all the state trackers
>>> in the same process share a single copy of the hashtable in the winsys
>>> code which allows them to share the same pipe_screen.
>>>
>>> I think that ends up being an extra level of library indirection vs
>>> current state w/ pipe drivers, ie. with mesa dri loader stuff directly
>>> loading gallium_dri.so which contains all the drivers plus mesa state
>>> tracker.  If this was concerning to someone, what I'd suggest would be
>>> to, for all the state trackers that already have some sort of loader
>>> sitting between them and the user, just pull them directly into the
>>> mega-mega libgallium.so, ie. something like:
>>>
>>>       +---------+---+-------------+
>>>       |         |   |             |
>>>       | clover  | l |             |
>>>       |         | i |             |
>>>       | mesa-st | b |             |
>>>       |         | g | pipe driver |
>>>       | vdapau  | a |             |
>>>       |         | l |             |
>>>       +---------| l |             |
>>>                 | i |             |
>>>     gralloc --> | u |             |
>>>                 | m |             |
>>>     xa      --> |   |             |
>>>                 |   |             |
>>>                 +---+-------------+
>>>
>>> Anyways, I'm far from an expert in the build architecture of things so
>>> I might have some facts wrong or be a bit confused.  And I'll probably
>>> regret bringing the subject up.  But somehow or another it seems like
>>> it would be good to (a) clean up all the GALLIUM_STATIC_TARGETS
>>> ifdeffery spread throughout all the different state trackers, and (b)
>>> have the pipe driver[*] only exist once per process rather than once per
>>> state-tracker.  Especially for android where each process using GL will
>>> have both gralloc and mesa-st.. and perhaps even clover/omx/etc.
>>
>> Radeon and amdgpu already have one pipe_screen per process per fd. It
>> works like this:
>>
>> - The GL, VDPAU, OMX etc. libs each ship its own copy of the whole
>> driver, so we have multiple screens and multiple winsyses.
>>
>> - All libs export "radeon_drm_winsys_create", which is this function:
>> struct radeon_winsys *radeon_drm_winsys_create(
>>    int fd, radeon_screen_create_t screen_create);
>>
>> - The first lib that is loaded registers "radeon_drm_winsys_create",
>> all other libs loaded later can't override it. Therefore, if any lib
>> calls radeon_drm_winsys_create, the version from the first loaded lib
>> is called.
>>
>> - The first caller passes the screen_create function to
>> radeon_drm_winsys_create, the screen is created only once for each
>> input fd in that function along with the winsys. When the winsys is
>> returned, radeon_winsys::screen is the returned screen.
>>
>> - Any other call to radeon_drm_winsys_create will return the same
>> winsys+screen combo for that fd. (it also needs to recognize dup()
>> fds, etc.)
>>
>> - Of course, all libs must use the same version of the driver.
>>
>> - The winsys is reference-counted. pipe_screen::destroy doesn't
>> destroy itself until the winsys counter drops to 0.
>
> Yes, I've seen how radeon and nouveau implement that, and I've done
> something similar.  But it relies on some linker script stuff to
> ensure there is only one xyz_drm_winsys_create(), which I couldn't get
> to work properly on android (which has it's own dynamic loader).  Not
> sure to what extent that was an issue w/ bionic dl loader vs me
> screwing something up in android build system or something.  I do know
> at some point recently the bionic dl loader broke weak symbols, and a
> few years ago didn't even support LD_PRELOAD, so I'm not inclined to
> trust it too much with fancy stuff.
>
> Having a single gallium lib that is dynamically linked into all it's
> users seems like a simple solution that is guaranteed to work with the
> most brain-dead of dynamic loaders.
>
> (But that said, I think the first time the mega-loader stuff came up,
> I was too busy trying to draw triangles and didn't pay much attention
> to the discussions or reasons things are like they are today)
>
>> Alternative solution other than the gallium lib:
>> - use a winsys lib or move the winsys into libdrm
>
> separate winsys lib is an interesting possibility..

I think it would be better to have a minimum "fd -> pipe_screen cache"
lib that has lib_get_pipe_screen(fd, screen_create). It should work
the same as radeon - return en existing screen for the fd or create a
new one. Another function would be bool lib_drop_screen_ref(fd) that
would decrement the reference counter for the screen associated with
that fd. This is much better than pulling a half of gallium into a
separate lib.

Marek