[systemd-devel] providing backwards compatibility

[Lennart Poettering]

On Fri, 17.01.14 04:06, Zbigniew Jędrzejewski-Szmek (zbyszek at in.waw.pl) wrote:

> So there's "binary" compatibility, and "source" compatibility.
> It would be nice if we could do those changes without
> 
> a) requiring recompilation ("binary")
> b) requiring editing build scripts in dependent packages ("source").
> 
> For b, keeping around a .pc file which points to the *new* library
> name should be enough:
> 
> - Libs: -L${libdir} -lsystemd-login
> + Libs: -L${libdir} -lsystemd
> 
> On our side it's a tiny effort, and avoid a major PITA for distributions.
> 
> For a, a compatibility symlink (e.g. libsystemd-login.so.0 -> libsystemd.so.0)
> can be provided. I'm attaching a POC patch which moves the symbol exports
> so that programs compiled against libsystemd-login will continue working.
> The symlink is not created, this part must be done by hand.
> 
> I think that if we ever decide to futher merge libraries, those
> would be the steps to take.

So here's my opinion on all of this:

First let me say that I am pretty strongly of the belief that API+ABI
breaks are something that distros have to deal with, it's one of the
primary reasons distros exist, so that they can deal with things like
this across the whole system. For example, recently glibc reorganized
"-lrt", and if they can do that, we certainly can do this too.

That said, while I believe that distros must be prepared to deal with
this, I think we can certainly help to make this easy for them, and make
the transition gradual rather than abrupt. However, for us upstream a
couple of things matter:

a) I don't want legacy/dead code in our upstream repo that we don't use
   and hence test regularly. Thus I don't think it is an option to
   simply keep two copies of everything in our repo, once for the old
   and once for the new library.

b) The symbols after they have moved must carry a new, correct symbol
   version, it's not an option to just move the library they are defined
   in.

c) Manual symblinks between libraries are not an option I think, this
   will break too easily, and will likely confuse tools like ldconfig...

d) Compatibility for the old library names and versions must be
   isolated in the source tree, and it must be clear that we can remove this
   easily within a year or so.

So, here's what I'd propose to maintain compatibility in both API and
ABI for a year or so:

1) Merge the libraries now. Give everything new symbol versions.

2) Expose the sd-bus/sd-event/sd-memfd/ symbols only if --enable-kdbus
   is specified. People who enable this will hence get a different API
   for this library than others! This is OK I think, by specifying
   --enable-kdbus users basically declare that they know that there are
   no API/ABI guarantees. 

3) The new library will ship one .pc file only.

4) In a new subdirectory src/compat-libs/ (or something like that) we
   will provide compat stubs that will build libraries (for ABI compat)
   and pc files (for API compat) that redirect to
   the new versions. This stuff should be optional though, and only be
   enabled if --enable-compat-libs is specified, and the README should
   name a date when this will be removed.

Now, the way I'd propose the stub libs of item #4 to be implemented is
via a combination of ".symver" asm magic, and the "ifunc" function
attribute. With a script we'd first generate for every old symbol code
like the following:

    void new_sd_id128_from_string(void);
    __asm__(".symver new_sd_id128_from_string,sd_id128_from_string at LIBSYSTEMD_209");

This would define a new symbol new_sd_id128_from_string which refers to
the specific version of the function from thew new library. Note that we
don't care for the right function signature, we only care for the naked
symbol name, that's all.

Then, we'd use ifunc to define a local function with the right name,
that just maps to the new implementation:

    static void (*resolve_sd_id128_from_string (void)) (void) {
            return new_sd_id128_from_string;
    }
    void sd_id128_from_string(void) __attribute__((ifunc("resolve_sd_id128_from_string")));

This newly defined function would then be assigned the right old version
with the ".sym" file, as before.

(Note that ideally we'd use the "weakref" gcc extension for this, but
that can only be used to define static aliases, and we need to export
this here, so this doesn't work. The ifunc trick is a work-around for
this limitation that should behave as close to this as possible, and
expose optimized performance, since ifunc is only evaluated once.)

Given that we don't care for the functions ignatures here it should be
trivial to write a script that generates this for all functions from the
old .sym files.

The symbols exposed should probably also carry the "deprecated" gcc
attribute so that people linking against them get a clear explanation
what to do and how to update to the new libs. Unfortunately .pc files
provide no support for encoding whether a lib is obsoelte, but we
should at least update the description string of the old .pc files to
declare them obsolete.

Yay for compiler magic!

Lennart

-- 
Lennart Poettering, Red Hat