[PATCH wayland 0/4] Untangle the symbol export duplication

[Emil Velikov]

Hi Pekka,

Thanks again for keeping up.

On 16 September 2016 at 10:46, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> Hi,
>
> there seems to be two different issues discussed here.
>
You cought me, yes there are.

> 1) libwayland-client and libwayland-server export the same interface
> symbols each, and cannot stop exporting them. You proposed to
> consolidate them into a third library.
>
> I don't think that is necessary.
>
> 2) Push everyone to build and install a .so and generated headers for
> every protocol extension XML file they offer for public use. You claim
> it would have benefits over the static build approach.
>
Almost...

s/Push/Suggest/ with an alternative on how to keep things static, yet
not exported. If anything your suggestion is the "pushy" one ;-)

> The static approach can break only when the interface symbols are
> exported and therefore possible to have conflicting symbols from
> different binaries all loaded into the same process. I think that is
> solved well enough by making wayland-scanner to stop forcing the export
> of the interface symbols.
>
Going a level up, abstracting ourselves of the actual wayland
specifics, there are a few simple (imho) points:
 - exposing the same symbols from different libraries is bad
 - hiding the exports will break applications and people will notice
that _after_ things are already broken.
 - any arguments against shared libraries is applicable to every
project in the wild.
 - users will have the option to static/shared link the code with info
how _not_ to export them (in case of the former).


>>  - Both client and server, take const pointer to the symbol in order
>> to bind an instance (client) and create the global (?) resource
>> (server).
>> Which in itself requires a provider for the said symbol(s). I believe
>> the latter is what you meant with "don't work", correct ?
>
> That is the "you cannot write all the necessary code" part, yes.
>
> The other part, which wayland-drm.xml does not excercise, is in the
> generated client headers as I explained above.
>
I fear you're getting on a tangent here - simple question was asked
and answered :-)

"header's won't work" == "headers _may_ pull dependency of _interface
symbol(s). latter of which should be resolved at link time"

>> >
>> > X_interface is only needed when creating objects of that kind.
>> >
>> Which (m)any new servers would like to do.
>
> I'm not sure what you're trying to say.
>
Simple. You mentioned "X_interface is only needed when creating
objects of that kind."
and I replied that "(m)any new servers would like to do." such a thing.

Those projects need to have the symbols resolved - be that statically
or otherwise is another topic.

>
> An interface can only ever be defined in one place, and no other place
> can change that or inherit from it (create a new interface of the same
> name but with added messages) because it will conflict eventually.
>
[...]
>
> Interface development by version bumps cannot branch, it must always be
> kept linear, which means that we really want to have just one XML file
> defining the interface.
>
I admit, the word "inherit" is wrong here. Dully noted.


> Such libfoo would not include any text, only variables with data that
> is constant and completely defined by the XML file it was created from.
> All the functions in the headers will be inline and call into libwayland
> directly.
>
Yes the actual "code" will be inlined into the user. Yes the library
will only add unique symbol(s) to the global namespace.

> All such libraries would be ignored by all other language bindings.
> Language bindings have their own scanners generating wrappers suitable
> for their language directly. The symbols we are talking about are just
> a curiosity of the C language bindings.
>
NB: other languages can reuse C binding, no ? Obviously they can use
the native ones as well.

> I really do not see much benefit from that. I do see a hassle with
> installing libraries and versioning everything properly.
Let's try to keep things technical please. "hassle" is ...

> Wayland-protocols would need to install 15 libraries already and that
> number will just grow and never go down.
Considering new protocols are added how could one expect _anything_ to
get less ?
If the number (15) of libraries is an issue one can fold things into a
single library. It will be a tiny bit of a cheat, but it's perfectly
reasonable.

> Add to that all the work
> needed in projects using these to start using the libraries.
>
Seems like my earlier point did not get through - old and new users
should need _zero_ changes.
If they opt for static + hide - one line, shared - ~5 or so.

> A project cannot make use of an interface v5 if it only implements
> support for v4. Using the library you suggest or not does not change
> that, support for a new interface version always requires new
> hand-written code for the new semantics. Therefore it is good enough to
> build the interface v4 definitions into the program directly, and avoid
> the runtime dependency hassle. The built-in v4 will work just fine,
> even if the other side of IPC supported only up to v3 or all the way up
> to v8.
>
The above does not consider/attribute for buggy applications (as
mentioned below), as mentioned before.

>
> There should never be multiple different providers, that is the core
> underlying issue.
>
Correct and we cannot go back in time to fix it.

> We can fix that for everything except libwayland by not exporting the
> symbols.
>
As before: one _cannot_ "unexport" the symbols since this will break
applications. And you really don't want to intentionally break
applications, when there is a way to avoid that ?

>> Combine that with the lack
>> of MIN(...) in (some) clients, equals a recipe for disaster.
>> Admittedly this (complete lack of MIN) is an application bug, yet it's
>> something that cannot (should not) be fixed/handled in wayland,
>> correct ?
>
> Are you talking about ignoring available_in_symbols version, or not
> doing version checks at all?
>
Not doing version checks _at all_.

Yes it is an application bug, and the proposed solution lets one
mitigate things. That obviously does _not_ mean that applications
should not be fixed.

>> > After all, people already use the headers they know are good, so getting
>> > a mismatching symbol for *_interface at runtime is quite a surprise.
>> >
>> Personally I haven't had bad experiences but as we get more wayland
>> programs and people using those, it is unlikely that we'll be so
>> fortunate.
>
> Yes, so let's stop the forced exporting where we can.
>
There's more to it - you're suggesting to hide them by default (the
non libwayland ones of course). I'm saying that one should keep them
around (in alternative form) by default and allow people to hide them.

>> Hiding the symbols is likely to lead to local workarounds, in the form
>> of "let's check-in this generated files and fix them because
>> wayland-scanner produces strange/broken files". Sadly we've all seen a
>> fair few similar cases.
>
> That will happen in any case. Let upstream at least try to do the right
> thing where it still can, shall we?
>
With the proposed solution there will be little-to-no way this can happen ;-)


>> Similar to libwayland-util.so, any old and new projects will continue
>> to work and the symbol duplication will be resolved.
>
> You only resolve the duplication between libwayland-server and
> libwayland-client, which has never been a problem heard about in
> upstream. The symbols exported by them are identical, so it does not
> matter which one provides them.
>
The "don't provide duplicate symbols" rule/suggestion/etc. is there
for a reason.

Just because in this particular place (project) and time those have
identical implementation there is no guarantee that things will stay
the same.
If they may never change - great ! If they do one will spend a lot
more time retroactively fixing things.

If anything, wayland/wayland-protocols would have been updated ~same
time as it takes to convince you/others that one wants to make things
robust.


> I agree that problems can arise if you link several different versions
> of libwayland-client and/or -server into the same program at runtime. Am
> I the only one who thinks that is a distribution error?
>
> Introducing libwayland-interface.so would not fix that case either,
> when you mix one lib from before and one from after
> libwayland-interface.so was introduced.
>
I never suggested that there is, was or will be a distribution
problem. Although yes if there is one, things will be
mitigated/resolved with my proposal ;-)


> Does libva not carry a copy wayland-drm.xml and generate its own code
> from it during build?
>
> Or is it really depending on the... hmm, libEGL.so? - exporting
> wl_drm_interface?
>
IIRC it carries the XML only for the client header, yet it depends on
libEGL.so. And yes, they _expect_ to have a cannonical provider for
the _interface symbol(s). It's only fair to assume that they're not
the only ones (but perhaps for other _interface symbols).

>> tldr: I'm suggesting creating a libwayland-interface.so shared library
>> and updating wayland-scanner to suggest/educate devs to either a)
>> create their own shared ones as needed or b) how to hide the symbols
>> if they opt to static link the protocol code.
>
> They cannot hide the symbols until we fix wayland-scanner to not
> force-export them.
>
>> Using shared library:
>> pros: code reuse, single provider for the _interface symbol.
>> cons: extra bit in dependency tracking/tree
>>
>> Using static linking:
>> pros: simpler dependency tree/tracking
>> cons: 'bloated' binaries, duplicated symbols which may or may not be
>> conflicting (in one keeps the WL_EXPORT), broken applications (if one
>> drops WL_EXPORT)
>
> IMO, let's go with static only. There really isn't enough to share in
> a .so to warrant all the work needed for it all over.
If it makes sense, but you won't be able to help just opt for "I can
see it, sadly I don't have time for it atm.". Objecting/blocking based
on your present availability is a bit strage.

> An .so would not
> make the checks on the available_in_symbols version any less necessary,
I never mentioned, suggested or hinted anything like that. It will be
used to complement the existing checks and thus mitigate the
problem(s).

> you could just trade it off to checking .so version at runtime.
>


The reluctance on the topic strikes me, bth:
 - Nobody is expecting you/others to hack on this. Fwiw we're talking
about ~20 loc for wayland.
 - The proposal preserves your suggestion, yet it is not the default option.
 - New projects will continue to work as before with zero changes and
they can opt for either new 'route'.
 - And last but not least, it preserves ABI compat and mitigates
existing and future problems.

Regards,
Emil