[Xcb] [PROPOSAL] how to solve ABI/API issues when fixing protocol definition bugs

[Christian Linhart]

Hi all,

A while ago we did some brainstorming about ABI/API issues when fixing
 protocol definition bugs that change API and/or ABI.

Based on that, and some additional thinking I have a proposal.
If we reach consensus about how to handle this,
I'll make an example implementation based on that,

The example implementation will solve (some of) the ABI/API issues of xinput
and possibly other extensions.
(making the next release compatible with the last official release.)

The proposal is as follows:

============
1. Description of the problem.

When some bugs in the protocol definitions ( xcb xml files ) are fixed,
then the generated code may change in a way that breaks ABI/API compatibility.

In this proposal, I first discuss the ABI issues and a possible solution.
Then I'll address API compatibility.

2. ABI compatibility

ABI compatibility is more important than API compatibility because
it affects end users that install new versions of XCB libraries on an
existing system. If ABI compatibility is violated, then applications
will stop working or will work in erroneous ways.

To ensure ABI compatibility I propose to use symbol versioning.

2.1. Some things to consider
* Since we use plain C we only need to care about function symbols
   Type names do not appear at ABI level in plain C as far as I know.
   Of course, types are relevant indirectly, because they define the
   behavior of functions.
   That is, if type changes, then all functions dealing with that type
   need to get compatibility functions which use the old type.
   Old types will have another name, with the version mangled into their name.
   Since type-names do not appear at ABI level, this is OK.

   For some type changes, functions need not be redefined, e.g.,
    - in some cases, when stuff is added at the end of a struct
    - or when a function only accesses members of a struct that
      appear before the first change,
      so the offset of these members stay the same.

* Do all of our platforms support symbol versioning?
   I know that Linux does, but what about Cygwin, etc?

* I think that we need to alter the generated code for this.
   After all, the existing functions do not have a version.
   So the generated code needs to give functions a different name,
   e.g., "name__ver_1_12", that can then be tied to the new library-version
   using something like
        __asm__(".symver name__ver_1_12 name at XCB_1_12")

   This can be controlled by an extra tag in the xml-definition where
   the version is set.

2.2. How to generate this code

In general I propose a hybrid scheme of generated and handwritten code.

In detail this can be done as follows:
* the handwritten compatibility stuff can be in files that have postfix of "_compat",
   i.e., xinput_compat.c

* names of types and functions of old versions will have a postfix
   like "__ver1_11".
   Please note the double underscore which should avoid name-clashes
   with main symbols.
   (such function names will be mapped to versioned symbols with .symver)

* generated code will work as follows:
    Multiple protocol entities with the same name can appear in the xml
    as long as they have different version tags.
    One of these entities may have no version tag.
    This means that this is oldest version of it, having no version at ABI level.

For bugfixes in the protocol definition, we can probably work with generated code.
(Unless we want to map the corrected protocol at X11-connection level
to the buggy protocol definition at ABI level.
That would need manually written code, but would probably be
too much work and/or be too risky with respect to introducing bugs
in that adaption code. In some important cases we may choose to
manually write such adaption code.)

For bugfixes in the generator, we may have to resort to manual code.
(Though we should generally avoid changes in the generator that break ABI or API.
There may be exceptions to this from time to time.)

3. API compatibility

API compatibilty is not strictly needed for a good end-user experience,
but it makes life easier for developers and distribution-package maintainers.
So, in general we should ensure API compatibility, but we need not
support strict API compatibility.

In detail this can be done as follows:
* the following will not affect API compatibility.
    - adding new functions or types
    - adding members to structs

* but the following will affect API compatibility:
    - removing members from structs
    - changing parameter lists of functions
    - removing functions
   It is not possible to provide full API-compatibility for these cases,
   as far as I know, but we can do it almost as good:

   We can optionally support old API-versions when corresponding
   macros are defined.
   These macros can be called  XCB_API_1_11, or something like that.

   Then, we'll #define names to the names used in the ABI compatibility code
   when an API-compat-macro is defined.

============

What do you think about this proposal?

If you have any questions or comments please post them.

Cheers,

Chris