[Xcb] [PROPOSAL] how to solve ABI/API issues when fixing protocol definition bugs

[Jeremy Huddleston Sequoia]

> On Jan 10, 2016, at 06:59, Christian Linhart <chris at DemoRecorder.com> wrote:
> 
> Hi Mark and all,
> 
> Mark, thank you for your feedback.
> That's very helpful to find a good solution.
> 
> So, we have the following situation:
> * The protocol definitions in the last official version are pretty buggy.
>    (at least in some extensions such as Xinput an XKB)
> 
> * For some of these bugs, the fix inevitably changes the ABI and/or API.
> 
> * We cannot use symbol-versioning to solve the ABI-compatibility problem
>   because it is not supported by all platforms. (for good reasons)
> 
> * API compatibility is important, so we
>   - either have to accumulate a lot of cruft in the API
>      (using version-suffixes for newer versions of functions.
>      The functions without suffix are the old, buggy ones.)

This can be done via macros in a way that can hide the complexity from clients.

>   - or bump the major version, and include the major version
>     in the path or the names of the include-files,
>     so the source of both major versions can coexist on the same system.

Yuck.

> This is a quite unfortunate situation with no easy and elegant solutions.
> 
> Two solutions come to my mind:
> 1. Frequent major releases:
>    * Do a major release once a year or so.
>       Collect the ABI/API breaker changes in a separate branch,
>       so that we can do minor releases in between major releases.
>    * The include-path contains the major release number.
>        (e.g. "/usr/include/xcb2/xinput.h"  vs "/usr/include/xcb/xinput.h"

Doing things like this can place an extra burden on those that care about maintaining binary compatibility.  For example, in XQuartz, we're shipping libpng 1.2, 1.4, 1.5, and 1.6 because we've shipped them in the past and someone might be linking against them.  We only ship the headers for the most recent version, but we have to keep shipping the libraries from those older branches as well.  Luckily, upstream keeps releasing security updates for the older branches, but once that stops, we'll be in a position of needing to determine if we want to take care of backporting security fixes, shipping with known vulnerabilities, or breaking clients.

> 2. Do explicit symbol versioning, i.e., add suffixes to names and types and functions
>    every time there is an ABI/API-breaking change.
>    For example:
>        xcb_input_input_state_t__v1_12
>    And keep the old versions of these symbols around.
>    (That would place quite a burden on application developers because
>    they have to know for each function and type, which version to use.
>    They might accidentally use an old, buggy version.)

This doesn't need to put a burden on developers.  You can have the headers determine which implementation is used based on deployment target (or just force the use of the most recent version if you don't care about older deployment targets).

> When I compare both of these solutions, I favor solution 1 (frequent major releases)
> because:
> * there is an endless supply of major release numbers
> * we avoid collecting cruft in the API, that would make the API difficult to use.
> 
> Though, solution 1 also has some disadvantages:
> * We'll have to merge stuff between branches.
>   Though we could avoid that by making only major releases ;-)
> 
> * system installations will need quite a number of major releases installed
>  to keep some applications working that are not migrated to new major versions.
>  Though there could be some social pressure to port such applications,
>  so the problem might not be that severe.

These disadvantages are why I dislike solution 1.  Solution 2 seems much simpler and less error-prone when done right.  I think you got caught up on clients needing to manually select which implementation they want, but that's not necessarily the case.

> 
> What do you think?
> 
> Any other ideas?
> 
> Cheers,
> 
> Chris
> 
> On 2016-01-10 13:45, Mark Kettenis wrote:
>>> From: Christian Linhart <chris at DemoRecorder.com>
>>> Date: Sun, 10 Jan 2016 12:35:26 +0100
>>> 
>>> 2. ABI compatibility
>>> 
>>> ABI compatibility is more important than API compatibility because
>>> it affects end users that install new versions of XCB libraries on an
>>> existing system. If ABI compatibility is violated, then applications
>>> will stop working or will work in erroneous ways.
>> I'd argue that in an open source world, API compatibility is just as
>> important as ABI compatibility.  The ability to recompile applications
>> on newer versions of an OS is important on systems where there are
>> only limited guarantees on ABI compatibility between OS releases.
>> 
>>> To ensure ABI compatibility I propose to use symbol versioning.
>> Not going to fly, because:
>> 
>>> * Do all of our platforms support symbol versioning?
>>>   I know that Linux does, but what about Cygwin, etc?
>> Not all platforms support symbol versioning the way Linux does.  And
>> some don't support it at all.  Symbol versioning as you know it is
>> strongly tied to the ELF executable binary format.  So platforms that
>> don't use ELF, such as OS X and Windows/Cygwin, probably don't support
>> it at all.
>> 
>> Solaris, which AFAIK is the OS that invented symbol versioning, does
>> not allow you to export multiple versions of a symbol.  That is, you
>> *cannot* have both name at XCB_1_12 and name at XCB_1_13.  That feature is a
>> GNU extension.  And it requires support from the dynamic linker.  So
>> even systems that use the GNU toolchain don't necessarily support it.
>> That is the case on OpenBSD for example.  FreeBSD and NetBSD might be
>> affected too.  In fact I think some of the alternative libc's on Linux
>> (musl, bionic) don't support it either.
>> 
>> We (OpenBSD) are not planning to implement Linux-style symbol
>> versioning.  We consider the "binary compatibility forever" approach a
>> serious security hazard because it allows people to keep old buggy
>> code around and leads to accumulation of cruft in your shared
>> libraries.  Instead we try to provide an easy way for our users to
>> upgrade their systems to a new version of the OS with freshly compiled
>> binaries.  Hence my different view on API compatibility ;).
>> 
>> The only portable way to handle ABI compatibility is to bump the major
>> number of the shared library whenever you break the ABI.
>> 
>> Cheers,
>> 
>> Mark
>> _______________________________________________
>> Xcb mailing list
>> Xcb at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/xcb
> 
> _______________________________________________
> Xcb mailing list
> Xcb at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/xcb