[Xcb] [PROPOSAL] how to solve ABI/API issues when fixing protocol definition bugs

[Jeremy Huddleston Sequoia]

On OS X, we solve this class of problems using symbol aliasing.  For example, we changed our stat struct about 10 years ago to allow for 64bit inodes, and we obviously didn't want to break binary or source compatibility.  The way we accomplished this was by symbol aliasing.  The older version of the stat syscall remained unchanged, and the newer version was named stat$INODE64.  Our headers are marked up like this (a bit simplified):

---
// Backwards compatible mode for targeting older versions of OS X
#if DEPLOYMENT_TARGET_DOESNT_SUPPORT_INODE64
struct stat {
   ....
};
#define __DARWIN_SUF_64_BIT_INO_T /* nothing */
#else
struct stat {
   ....
};
#define __DARWIN_SUF_64_BIT_INO_T "$INODE64"
#endif

#define __DARWIN_INODE64(sym) __asm("_" __STRING(sym) __DARWIN_SUF_64_BIT_INO_T)

int stat(const char *, struct stat *) __DARWIN_INODE64(stat);
---

This allows for binary compatibility (old symbol provides the same functionality as it always did), source compatibility (same function prototype), and migration of newly compiled code to the new functionality.

More comments inline below.

> On Jan 10, 2016, at 04:45, Mark Kettenis <mark.kettenis at xs4all.nl> wrote:
> 
>> From: Christian Linhart <chris at DemoRecorder.com>
>> Date: Sun, 10 Jan 2016 12:35:26 +0100
>> 
>> 2. ABI compatibility
>> 
>> ABI compatibility is more important than API compatibility because
>> it affects end users that install new versions of XCB libraries on an
>> existing system. If ABI compatibility is violated, then applications
>> will stop working or will work in erroneous ways.
> 
> I'd argue that in an open source world, API compatibility is just as
> important as ABI compatibility.  The ability to recompile applications
> on newer versions of an OS is important on systems where there are
> only limited guarantees on ABI compatibility between OS releases.

Yes.  Absolutely.  Let's not break API or ABI if at all possible.

>> To ensure ABI compatibility I propose to use symbol versioning.
> 
> Not going to fly, because:
> 
>> * Do all of our platforms support symbol versioning?
>>   I know that Linux does, but what about Cygwin, etc?
> 
> Not all platforms support symbol versioning the way Linux does.  And
> some don't support it at all.  Symbol versioning as you know it is
> strongly tied to the ELF executable binary format.  So platforms that
> don't use ELF, such as OS X and Windows/Cygwin, probably don't support
> it at all.

As mentioned above, we have ways to deal with this in OS X that are slightly different, and I think our approach is actually portable.

> Solaris, which AFAIK is the OS that invented symbol versioning, does
> not allow you to export multiple versions of a symbol.  That is, you
> *cannot* have both name at XCB_1_12 and name at XCB_1_13.  That feature is a
> GNU extension.  And it requires support from the dynamic linker.  So
> even systems that use the GNU toolchain don't necessarily support it.
> That is the case on OpenBSD for example.  FreeBSD and NetBSD might be
> affected too.  In fact I think some of the alternative libc's on Linux
> (musl, bionic) don't support it either.
> 
> We (OpenBSD) are not planning to implement Linux-style symbol
> versioning.  We consider the "binary compatibility forever" approach a
> serious security hazard because it allows people to keep old buggy
> code around and leads to accumulation of cruft in your shared
> libraries. Instead we try to provide an easy way for our users to
> upgrade their systems to a new version of the OS with freshly compiled
> binaries.  Hence my different view on API compatibility ;).

Well, not all vendors have that level of flexibility.  On OS X, we still need to make sure that super old versions of Adobe Photoshop which were compiled before these changes still continues to work, and vendors want to make sure that their products work on older versions of OS X that might not have certain changes.  Thus, it is important for us that we have a way to phase in such changes while maintaining binary compatibility.

> The only portable way to handle ABI compatibility is to bump the major
> number of the shared library whenever you break the ABI.

True, but I like our intermediate approach.  We do not break the ABI, and instead transition clients to replacement symbols for the new functionality.

FWIW, we've done this a few times now (64bit inodes, UNIX-compliance, etc) and had success with this approach.

> 
> Cheers,
> 
> Mark
> _______________________________________________
> Xcb mailing list
> Xcb at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/xcb