[Mesa-dev] Static/shared pipe-drivers (was megadriver/pipe-loader-to-all)

[Emil Velikov]

On 18 June 2014 08:21, Chia-I Wu <olvaffe at gmail.com> wrote:
> Hi Emil,
>
> On Fri, Jun 13, 2014 at 3:56 AM, Emil Velikov <emil.l.velikov at gmail.com> wrote:
>> Hi all,
>>
>> These patches add support for building (grouping) the various targets per
>> API, meaning that only one library will be created  for e.g. vdpau
>> (libvdpau_gallium) with individual ones (libvdpau_r600) being a hardlink
>> to it.
>>
>> This allows us to have substantial space savings as the API(state-tracker)
>> is available only once. Additionally it adds support for shared
>> pipe-drivers via a _unstable_ interface, which saves us the duplication
>> across X APIs.
>>
>> The former method has been used by the egl-static while the latter by
>> opencl and gbm targets since they were introduced.
>>
>> By default we build with "static pipe-drivers".
>>
>> Some numbers + extra info [1]
>>
>> [Static]
>> dri:            (r600|radeonsi|nouveau)_dri.so       -> 6.5 MiB
>> vdpau:          libvdpau_(r600|radeonsi|nouveau).so  -> 3.5 MiB
>>
>> Total: 10MiB
>>
>> [Shared]
>> Libraries:
>> dri:            (r600|radeonsi|nouveau)_dri.so       -> 3.9 MiB
>> vdpau:          libvdpau_(r600|radeonsi|nouveau).so  -> 633 KiB
>> gallium-pipe:   pipe_(r600|radeonsi|nouveau).so      -> 5.3 MiB
>>
>> Total: 9.8MiB
>>
>> [Current]
>> dri:            (r600|radeonsi|nouveau)_dri.so       -> 5.0+4.5+5.3 = 14.8 MiB
>> vdpau:          libvdpau_(r600|radeonsi|nouveau).so  -> 1.9+1.2+2.3 = 5.4 MiB
>>
>> Total: 20.2MiB
>>
>>
>> The previous series can be found here [2]
>> Changes since then
>>  - Convert targets individually.
>>  - OMX targets now work, and the final library is now libomx-mesa.so
>>  - Dropped the DRI targets for now
>>  - A handfull of typos thinkos and bugs fixed.
>>
>>
>> My plan is to have these pushed in ~4 stages, with two stages per week.
>> This way I will be able to crack on with the remaining bits and have all
>> of it tested well before we branch the next release.
>>
>> Series is availabe at
>> https://github.com/evelikov/Mesa/tree/static-or-shared-pipe-drivers
>>
>> As always comments and suggestions are greatly appreciated.
> Thanks for working on this.  This is a tough issue to tackle.  I have
> a few questions/comments, which I am fine to see them resolved either
> before or after landing your series.
>
> I see this work as to define an internal API to manage pipe drivers.
Interesting, I do not see this as an attempt to define an API, but to
cleanup all the mayhem that our targets currently are:
 * Cleanup the build system - drop symlinks, including the same source
files from different locations.
 * Make targets less error prone by using static pipe-drivers by
default. Shared ones we lack versioning and ... are a big can of
worms.
 * Minimize all the target.c duplication across each target. Makefiles
are in similar boat.
 * Allow people to use the unstable pipe-drivers if they are really
short on size and know what they are doing.

> The lack of such API previously led us to "targets", where each target
> knows how to load a specific driver.  With your changes, state
> trackers that need to work with pipe drivers have a way to do so.  As
> a result, files such as
>
>   dri/target.c,
>   xa/target.c,
>   xvmc/target.c,
>   vdpau/target.c, and
>   omx/target.c
>
> become quite dummy and redundant.  Do you see a way to get rid of
> "targets" entirely?
>
Indeed if/when an API comes around these targets may become redundant.

> In the same view, and noticing that all users of the API have this snippet
>
> #if GALLIUM_STATIC_TARGETS
>    scrn->base.pscreen = dd_create_screen(fd);
> #else
>    if (pipe_loader_drm_probe_fd(&scrn->base.dev, fd, true))
>       scrn->base.pscreen = pipe_loader_create_screen(scrn->base.dev,
> PIPE_SEARCH_DIR);
> #endif // GALLIUM_STATIC_TARGETS
>
> I think it makes sense hide this difference behind the API. Another
I'm not sure that is feasible, yet. Biggest obstacle is the
pipe-loaders API, and mainly the software winsys' with their
variation.

> thing I noted is that the non-static path allows the user to auth the
> fd while the static path doesn't.  It is not clear to me how come the
> static path works.
>
Some of the targets do not need an auth as the loader (libGL,
libEGL...) already does that. I have not extensively tested the
pipe-loader paths but they seems to work with simple tasks - glxgears,
mplayer(vpdau). It could be that we might need to drop the auth in
some cases - to be sorted out once confirmed to be an issue.

> On the other hand, the implementation of the API extends itself a bit
> when DRI_TARGET is defined.  That is ugly from the API's point of
> view.  Could that be abstracted somehow so that it can be used
> elsewhere or at least looks nicer?
>
> Finally, the API is limited to C API (i.e., inline_{drm,sw}_helper.h).
> At the build system level, we also like a way to easily manage a pipe
> driver.  For example, we can see this snippet in many of the
> Makefile.am
>
> if HAVE_GALLIUM_RADEONSI
> STATIC_TARGET_CPPFLAGS += -DGALLIUM_RADEONSI
> STATIC_TARGET_LIB_DEPS += \
>         $(top_builddir)/src/gallium/drivers/radeonsi/libradeonsi.la \
>         $(RADEON_LIBS)
> endif
>
> If pipe drivers can be accompanies by some Makefile rules specifying
> how it should be built, in addition to how it can be loaded, we can
> get rid of those duplicated rules.
>
Thanks for giving the radeon as an example, it is the most complex one
currently - three drivers, one common "driver" across r600 and
radeonsi, and a single winsys that is used by all.
I have tried merging all these into a single location, although
presently we define HAVE_ST_VDPAU and HAVE_GALLIUM_R300, yet we do not
want a libvdpau_r300. I believe that this is the biggest issue
presently with the "split the rules into manageable and non-repetitive
manner".

Perhaps we can isolate the requirements/dependencies in a template per
driver (gallium/driver/Makefile.template) and explicitly check and
include the correct template when required.

Thanks for the input Chia-I. Let me know if any of the above makes sense :)

-Emil

>
>>
>> Cheers,
>> -Emil
>>
>> [1] http://lists.freedesktop.org/archives/mesa-dev/2014-May/059806.html
>> [2] http://lists.freedesktop.org/archives/mesa-dev/2014-May/059628.html
>>
>>
>> _______________________________________________
>> mesa-dev mailing list
>> mesa-dev at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/mesa-dev
>
>
>
> --
> olv at LunarG.com