[RFC wayland] scanner: Add support for dynamic symbol loading

Fri Feb 26 12:24:05 UTC 2016

On Thu, 18 Feb 2016 18:51:33 +0800
Jonas Ådahl <jadahl at gmail.com> wrote:

> In certain configurations the wl_interface symbols up various
> interfaces from wayland core (such as wl_surface_interface,
> wl_pointer_interface etc) are not available at build time, for example
> when the application will dlopen() the required symbols at run time.
> 
> To make it possible to build files generated by wayland-scanner this
> way, it needs to be possible to generate files that don't reference any
> external symbols, while enabling the application to provide these
> symbols at a later state.
> 
> To provide this, two new commands are added to wayland-scanner:
> 'dynamic-code' and 'dynamic-header'.
> 
> The 'dynamic-code' command generates a C source code file similar to the
> one generated by the code command, but instead of filling out the
> 'types' wl_interface array with external symbols, it fills it with only
> NULL pointers, while adding a function for providing all the necessary
> wl_interface pointers. The application must call this function before
> using any Wayland IPC functionality.
> 
> The 'dynamic-header' command generates a C header file with the
> declaration of the function the application must call to initialize the
> wl_interface types array.
> 
> Signed-off-by: Jonas Ådahl <jadahl at gmail.com>
> ---
> 
> Hi,
> 
> While adding support for using wayland-protocol extensions in libSDL I came
> across this issue with the generated -protocol.c file. The issue is that SDL
> will not link to libwayland-client on build time but instead on-demand dlopen
> all of the required symbols at run time. This makes all -protocol.c files
> generated by wayland-scanner fail to build since they reference wl_interface
> symbols for various core protocol interfaces.
> 
> To make it possible to support building protocol extension files and then load
> them at run time I had to add this feature to wayland-scanner to make it work.
> 
> I imagine another potential solution would be to split up libwayland-client.so
> and let libSDL run wayland-scanner on the installed wayland.xml and keep its
> own copies of all the wl_interface symbols, but it  at a glance adding this
> feature to wayland-scanner seems like the better option.
> 
> Let me know what you all think.

Hi Jonas,

TL;DR, it's a bit of an incoherent rant; I think we should make a
dlopenable API a whole new set of Wayland C language bindings, and your
RFC isn't going far enough.

I think the previous times this came up, we have shrugged it off with
"you should create your own library you dlopen() instead of
libwayland-client". It's easy to say without knowing anything of the
target software, and does have some perks like not always loading code
you might not use and reducing the API you have to dlsym(), but I
suppose it's also somewhat arrogant.

Doesn't libSDL already have an abstraction towards window systems? How
difficult would it be to write a SDL-internal library to be dlopen()'d
instead of libwayland-client?

Are there other reasons SDL does not use internal .so to dlopen()?
Perhaps the desire to support several different major versions on a
platform?

It's not like you are switching between API implementations here like
with GL, but switching a whole backend and backends have nothing in
common in the system library APIs they dlopen(), so the whole
dlsym()'ing is different in each backend.

On a completely different matter, we don't have any wayland-scanner
tests, which makes me uneasy to land any changes to wayland-scanner.
OTOH, we have listed adding those tests as a part of potential GSoC
tasks.

>  src/scanner.c | 268 ++++++++++++++++++++++++++++++++++++++++++++++++++--------
>  1 file changed, 231 insertions(+), 37 deletions(-)

I took a look at what this generates.

I understand that to use this, you have to do the following during a
build:

1. generate the .c file using "dynamic-code" instead of "code"
2. generate the usual headers using "client-header"
3. generate an extra header with "dynamic-header"

Is that correct?

This seems to take care of the data symbols partially, but I think
there other things missing.

The wl_interface data symbols that are defined in the generated .c file
could be plugged in by the generator directly. The other interface
symbols must be provided like you have done, so you are dlsym()'ing
e.g. wl_surface_interface.

Another problem is the generated headers, "client-header", which
assumes it can statically call libwayland-client entry points, but you
need to dlsym() them. You could probably hack your way around them [1],
but it seems ugly without a generator mode to use function pointers
instead to avoid name clashes.

Even more so, as libwayland-client provides a pre-generated
wayland-client-protocol.h, so you'd best avoid it and generate yourself.

Is it really easier or somehow more desireable to do all that than
write your own internal library to dlopen()?

I also consider wayland.xml being exposed in libwayland-client to be
the odd one out. Just like language bindings, for dlopen() I'd expect
you to generate your own bindings from the scratch, since
libwayland-client ABI which you dlsym() is not what
wayland-client-protocol.h describes.

Btw. if libwayland-client.so is dlopened with RTLD_LOCAL, and Mesa's
libEGL is dlopened with RTLD_LOCAL, does it mean we get two instances
of libwayland-client.so in the process? If SDL dlopened just one lib,
that links to both libEGL.so and libwayland-client.so, would it save
anything?

SDL_waylanddyn.c uses SDL_LoadObject() which explicitly sets
RTLD_LOCAL. That file is an unholy mess, and I believe we could do
better, if not using a private library then having wayland-scanner go
all the way supporting dlsym() rather than just the parts you cannot
hack around.

Thanks,
pq

[1] http://hg.libsdl.org/SDL/file/0897c3accd52/src/video/wayland/SDL_waylanddyn.h
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 811 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20160226/5da5a28d/attachment.sig>