[Xcb] [PROPOSAL] how to solve ABI/API issues when fixing protocol definition bugs

[Christian Linhart]

Hi Mark and all,

Mark, thank you for your feedback.
That's very helpful to find a good solution.

So, we have the following situation:
* The protocol definitions in the last official version are pretty buggy.
    (at least in some extensions such as Xinput an XKB)

* For some of these bugs, the fix inevitably changes the ABI and/or API.

* We cannot use symbol-versioning to solve the ABI-compatibility problem
   because it is not supported by all platforms. (for good reasons)

* API compatibility is important, so we
   - either have to accumulate a lot of cruft in the API
      (using version-suffixes for newer versions of functions.
      The functions without suffix are the old, buggy ones.)
   - or bump the major version, and include the major version
     in the path or the names of the include-files,
     so the source of both major versions can coexist on the same system.

This is a quite unfortunate situation with no easy and elegant solutions.

Two solutions come to my mind:
1. Frequent major releases:
    * Do a major release once a year or so.
       Collect the ABI/API breaker changes in a separate branch,
       so that we can do minor releases in between major releases.
    * The include-path contains the major release number.
        (e.g. "/usr/include/xcb2/xinput.h"  vs "/usr/include/xcb/xinput.h"

2. Do explicit symbol versioning, i.e., add suffixes to names and types and functions
    every time there is an ABI/API-breaking change.
    For example:
        xcb_input_input_state_t__v1_12
    And keep the old versions of these symbols around.
    (That would place quite a burden on application developers because
    they have to know for each function and type, which version to use.
    They might accidentally use an old, buggy version.)

When I compare both of these solutions, I favor solution 1 (frequent major releases)
because:
* there is an endless supply of major release numbers
* we avoid collecting cruft in the API, that would make the API difficult to use.

Though, solution 1 also has some disadvantages:
* We'll have to merge stuff between branches.
   Though we could avoid that by making only major releases ;-)

* system installations will need quite a number of major releases installed
  to keep some applications working that are not migrated to new major versions.
  Though there could be some social pressure to port such applications,
  so the problem might not be that severe.

What do you think?

Any other ideas?

Cheers,

Chris

On 2016-01-10 13:45, Mark Kettenis wrote:
>> From: Christian Linhart <chris at DemoRecorder.com>
>> Date: Sun, 10 Jan 2016 12:35:26 +0100
>>
>> 2. ABI compatibility
>>
>> ABI compatibility is more important than API compatibility because
>> it affects end users that install new versions of XCB libraries on an
>> existing system. If ABI compatibility is violated, then applications
>> will stop working or will work in erroneous ways.
> I'd argue that in an open source world, API compatibility is just as
> important as ABI compatibility.  The ability to recompile applications
> on newer versions of an OS is important on systems where there are
> only limited guarantees on ABI compatibility between OS releases.
>
>> To ensure ABI compatibility I propose to use symbol versioning.
> Not going to fly, because:
>
>> * Do all of our platforms support symbol versioning?
>>    I know that Linux does, but what about Cygwin, etc?
> Not all platforms support symbol versioning the way Linux does.  And
> some don't support it at all.  Symbol versioning as you know it is
> strongly tied to the ELF executable binary format.  So platforms that
> don't use ELF, such as OS X and Windows/Cygwin, probably don't support
> it at all.
>
> Solaris, which AFAIK is the OS that invented symbol versioning, does
> not allow you to export multiple versions of a symbol.  That is, you
> *cannot* have both name at XCB_1_12 and name at XCB_1_13.  That feature is a
> GNU extension.  And it requires support from the dynamic linker.  So
> even systems that use the GNU toolchain don't necessarily support it.
> That is the case on OpenBSD for example.  FreeBSD and NetBSD might be
> affected too.  In fact I think some of the alternative libc's on Linux
> (musl, bionic) don't support it either.
>
> We (OpenBSD) are not planning to implement Linux-style symbol
> versioning.  We consider the "binary compatibility forever" approach a
> serious security hazard because it allows people to keep old buggy
> code around and leads to accumulation of cruft in your shared
> libraries.  Instead we try to provide an easy way for our users to
> upgrade their systems to a new version of the OS with freshly compiled
> binaries.  Hence my different view on API compatibility ;).
>
> The only portable way to handle ABI compatibility is to bump the major
> number of the shared library whenever you break the ABI.
>
> Cheers,
>
> Mark
> _______________________________________________
> Xcb mailing list
> Xcb at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/xcb