[Mesa-dev] [Junk released by User action] Re: Statically linking libstdc++ and libgcc

[Francisco Jerez]

"Pierre-Loup A. Griffais" <pgriffais at valvesoftware.com> writes:

> On 03/12/2015 08:20 PM, Michel Dänzer wrote:
>> On 13.03.2015 03:07, Pierre-Loup A. Griffais wrote:
>>> On 03/11/2015 09:40 AM, Ian Romanick wrote:
>>>> On 03/11/2015 09:31 AM, Tobias Klausmann wrote:
>>>>> The problem in not forcing this to link statically is, that if a
>>>>> distribution decides to not use this static option, the problem persists
>>>>> on that distribution. On top every lib pulled in by steam from the
>>>>> system would need to be link statically, like mesa. Instead of fixing
>>>>> every lib steam pulls in (how many are there?), fix the steam runtime to
>>>>
>>>> Yeah, static linking is a terrible, partial solution.
>>>>
>>>>> a) not ship libstdc++.so and libgcc_s.so and declare older version of
>>>>> these libs as not supported (thats what people do when they face the
>>>>> incompatibility problem with steams versions of these libs: just delete
>>>>> them from the runtime and everything is fine)
>>>>
>>>> Let's be 100% clear.  This is NOT an option for Steam.  They ship
>>>> thousands of closed-source applications.  These applications were built
>>>> and tested against specific versions of specific libraries.  Removing
>>>> support for old run-times is equivalent to removing support for those
>>>> applications.  They can't tell their customers, "You upgraded your
>>>> distro, so that game you paid money for no longer works.  Tough break,
>>>> kid."
>>>
>>> Quite true; developers build and test their games against this known set
>>> of libraries through the Steam Runtime toolchain. Expecting them to
>>> manage all of their dependencies (through bundling or static linking) is
>>> unreasonable; all other platforms they're used to dealing with solve
>>> this problem one way or another. Admittedly the current implementation
>>> of the Steam runtime is only a first step towards providing a solution,
>>> and some things could be done better, but this issue will stand regardless.
>>>
>>> The C++ runtime is a very complex beast and swapping it under an
>>> application that makes heavy use of it is prone to failure. We'll keep
>>> running games against the C++ runtime of the SDK version they were built
>>> against, forever.
>>>
>>> The LLVM-enabled variants of Mesa are the only implementations where
>>> this is an issue; this is actually causing affected users in the wild to
>>> switch to the Catalyst driver, since it (along with all others) goes out
>>> of its way to avoid this specific problem for this exact reason.
>>>
>>> Their only other option is to slam the whole Steam runtime off, which
>>> breaks a lot of games outright because they depend on some library that
>>> recent distros don't ship anymore, or moved to the new major version.
>>
>> In my experience, and from other reports I've seen, simply deleting
>> libstdc++ / libgcc from the Steam runtime works just fine. This shows
>> that a) the problem is that Steam overrides the system versions of these
>> libraries with older versions, b) doing that isn't necessary, as the
>> games work just fine with the newer system versions of these libraries,
>> since they preserve backwards compatibility.
>>
>> So, a simple solution would be for Steam not to override these
>> libraries[0] when the system already has the same or newer versions of
>> them available. That doesn't require deviating from long established
>> best practices for building and distributing software in distros.
>
> Who would be responsible for supporting any fallout caused by running on 
> top of an unknown C++ runtime?
>
It turns out that the GCC C++ runtime provides a forwards-compatible ABI
(please read [1]).

If a certain application requires such a level of paranoia (and I think
it's fair to call it paranoia until someone provides any evidence that
the documentation of libstdc++ is wrong) that not even an ABI-compatible
minor version update is acceptable, why is the application dynamically
linking against libstdc++ in the first place?  Arguably the slight level
of risk was deemed acceptable in order to get some benefit from dynamic
linking? (reduced binary size, memory footprint, the possibility of
getting security, performance and bug-fix library updates without
recompiling).  Doesn't Steam's frozen C++ runtime effectively neutralize
those benefits?

> The contract right now is that we provide a runtime environment for 
> games to run in. If things break, we're responsible. We feel confident 
> providing that level of support because we're running against a set 
> known bits that were thoroughly tested.
>
> The traditional distro process is awesome at what it does, but not great 
> for third-party software. The idea of providing stable runtime 
> environments for these isn't anything new and I think an increasing 
> number of people in the community are recognizing the need for it and 
> rolling out similar solutions.
>
> The graphics stack is a particularly tricky edge-case in that end-users 
> fully expect to be able to upgrade it in-place to improve the existing 
> applications that they already purchased, unlike the rest of the runtime 
> these applications were built against.

Here you raise a general argument against having dynamic dependencies to
OS libraries (which could very well apply to having Mesa as external
dependency from the OS -- Or can anyone guarantee that future, unknown
versions of Mesa won't change their behaviour in subtle ways that will
prevent released Steam titles from running?), that brings you to
distribute fixed releases of most shared libraries, then you realize
that by doing that you have to give up one of the inherent benefits of
dynamic linking (in-place updates), and decide to make an exception for
this specific library *only*, without considering its dependencies.  The
problem here is not the fact that dependencies exist, but simply that
you're only willing to make an exception for an inconsistently defined
set of them.

> It also has strong ties to the rest of the host system, where some
> drivers have versioned dependencies to kernel bits, etc.

That's not completely true, kernel interfaces are generally intended to
be stable, if you bundled a current build of Mesa and libdrm it would
likely continue to work for a long time.  Nothing prevents you from
giving Mesa the same treatment you give to the other shared libraries in
the Steam runtime.  Actually this might be the sanest solution -- The
libstdc++ case is in fact a relatively "easy" one because the GCC
project is actually committed to keeping a stable ABI, but in the future
Mesa might benefit from linking to other shared libraries with less
strict versioning policies, if it's not doing it already
*cough*libudev*cough*.

>
> For all these reasons making sure the graphics driver libraries are as 
> self-contained as possible will result in a better experience for 
> end-users, be it Steam customers or GNOME application sandbox runtime 
> users or whomever else.
>

You disregard the benefits of dynamic linking.  IMHO this would result
in a worse end-user experience for everyone except Steam users, and the
latter only because Steam fails to give the application a consistent
environment of shared libraries where all components have their
dependencies satisfied.

> In a perfect world this wouldn't be needed at all; the GL libraries are 
> pretty much our only required window onto the host system, and something 
> like dlmopen() with a new linkmap would be an elegant solution to 
> sidestep that problem. In practice this loader path is barely 
> maintained, crashes more often than not, and is not supported at all by 
> GDB and other debugging tools.
>
> I expect we'll get there eventually, but in the meantime I'm not sure 
> that the proposed workaround is horrible enough to justify the ado it's 
> causing.
>
> Thanks,
>   - Pierre-Loup
>
>>
>> [0] Note that "these libraries" refers to the specific SONAMEs, e.g.
>> libstdc++.so.6 and libgcc_s.so.1. If the system only provides different
>> SONAMEs for these libraries, it should be fine for Steam to use its own
>> versions of these SONAMEs.
>>
>>

[1] https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 212 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/mesa-dev/attachments/20150315/1414fc7e/attachment-0001.sig>