[Bug 797185] New: Fix API export/import decorators in general and "inconsistent DLL linkage" with MSVC on Windows

[GStreamer (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=797185

            Bug ID: 797185
           Summary: Fix API export/import decorators in general and
                    "inconsistent DLL linkage" with MSVC on Windows
    Classification: Platform
           Product: GStreamer
           Version: git master
                OS: All
            Status: NEW
          Severity: normal
          Priority: Normal
         Component: gstreamer (core)
          Assignee: gstreamer-bugs at lists.freedesktop.org
          Reporter: t.i.m at zen.co.uk
        QA Contact: gstreamer-bugs at lists.freedesktop.org
     GNOME version: ---

Quick write-up about the "recent" (just before 1.14) and upcoming changes about
how public symbols are exported in the various shared GStreamer libraries.
Mostly so there's something to reference from the various commits and a place
to track regressions, if any.

# Building shared libraries

On *nix systems or with gcc/clang-based toolchains (like mingw), pretty much
all symbols will be exported by default unless they were explicitly marked as
static or internal/hidden. In GStreamer we have in the past just exported all
symbols that match a certain regular expression ('starts with gst_|GST_' more
or less). This regexp exporting is not standardised across different
toolchains, so a bit iffy to make work consistently across the board.

With the MSVC toolchain however, by default no symbols will be exported. There
symbols have to be exported explicitly. This could be done via a version/map
file of some sort (like the .def files we used to have in
win32/common/libgstxyz.def), or it can be done explicitly by marking symbols to
be exported with __declspec(dllexport). The .def file approach is not so
practical when the list of exported symbols is not always the same but may vary
depending on the configuration or what dependencies are available exactly.

On *nix systems one does not have to import external symbols explicitly, it is
enough to have a declaration in a header file. On Windows/MSVC this is mostly
fine as well, at least for functions, but variables are treated specially and
they must be imported explicitly via __declspec(dllimport).

In GStreamer we have recently added 'decorators' to all public exported API.
Initially this was just GST_EXPORT for all public API in all libraries.
However, that caused problems and lots of 'inconsistent dll linkage' warnings
and other compiler and linker warnings on the Windows MSVC builds.

The reason is as follows: Consider the case where we compile libgstbase-1.0.
This shared library will *export* a variety of symbols such as gst_base_sink_*
for example. At the same time it will implicitly or explicitly *import* (use) a
variety of public symbols from libgstreamer-1.0. Now when building
libgstbase-1.0 we will just #include headers such as gst.h an gstbasesink.h,
and there all public functions would be marked with the same export decorator
GST_EXPORT. If we define -DGST_EXPORTS then the GST_EXPORT will be turned into
__declspec(dllexport) on Windows/MSVC. But as we can't have different values
for GST_EXPORT depending on which headers we include, all public API will be
marked for exporting, even the one from libgstreamer-1.0 which we want to
*import*. This causes the warnings.

Having discovered this issue we therefore changed the API decorators into
per-library GST_*_API decorators just before the 1.14.0 release, so we now have
GST_API and GST_BASE_API etc. They still all mapped to GST_EXPORT however, so
we didn't solve the problem yet back then, but we laid the foundation to be
able to define them to separate things (e.g. GST_API = import, GS_BASE_API =
export) later.

The last step will be achieved by passing -DBUILDING_GST and
-DBUILDING_GST_BASE etc. when building each library. That define can then be
used in the header files by the API decorators to evaluate to either the export
or the import macro.

Example:

#ifdef BUILDING_GST_BASE
#define GST_BASE_API export
#else
#define GST_BASE_API import
#endif

The exact export define will depend on a number of things, such as the
toolchain used and the operating system. It can be one of (for now):

  - __declspec(dllexport) extern
  - extern __attribute__ ((visibility ("default")))
  - extern

We will decide which one it is in configure and/or meson.build, and add an
GST_API_EXPORT define to config.h that maps to the right thing (we do not want
to pass this via the command line, that's pretty much impossible to get right
with autotools + libtool due to all the layers and quoting we'd have to
handle).

The import define is easier, we can just hardcode that to one of

  - __declspec(dllimport) extern
  - extern

depending on whether a Windows/MSVC toolchain is used or not. We will do that
in gst/gstconfig.h as GST_API_IMPORT.

Then we end up with:

#ifdef BUILDING_GST
#define GST_API GST_API_EXPORT  /* from config.h */
#else
#define GST_API GST_API_IMPORT  /* from gst/gstconfig.h */
#endif

#ifdef BUILDING_GST_BASE
#define GST_BASE_API GST_API_EXPORT  /* from config.h */
#else
#define GST_BASE_API GST_API_IMPORT  /* from gst/gstconfig.h */
#endif

So when building libgstbase-1.0 BUILDING_GST_BASE will be defined and
BUILDING_GST will not be defined, and GST_API will map to GST_API_IMPORT and
GST_BASE_API to GST_API_EXPORT.

Since the export define comes from config.h we need to make sure that all .c
files include config.h as first thing, so the GST_API_EXPORT decorator is
defined before they include any header files. Failure to do so will result in
easy to spot compiler errors.

We could have stuck with the .def file approach, but that was never really very
good. People would always forget to add new API, and it doesn't work for libs
like libgstgl-1.0 where what is included depends on the build options or
operating system (which is already the case for the other libs too really just
ignored for now).

With the decorator system we can handle all of this correctly and consistently.

We have also switched to disabling export of all symbols by default on *nix
with gcc/clang toolchains by passing -fvisibility=hidden where supported. This
way only symbols that are decorated with __attribute__ ((visibility
("default"))) get exported.

This way we can make sure we get consistent exports on all systems/toolchains
and will quickly notice if an export is missing somewhere.

The only thing that's a bit messy is libgstcheck which includes an internally
built copy of libcheck whose symbols we leak in libgstcheck for historical
reasons. There we export based on a static list of functions for now with
autotools (with meson it's not a problem).

# Building static libraries

To link to GStreamer statically on Windows, one must define
GST_STATIC_COMPILATION or the prototypes will cause the compiler to search for
the symbol inside a DLL. (N.B. we pretty much only support
everything-gst-static or everything-gst-dynamic but do not support
mix'n'match.)

The fact that we have to define GST_STATIC_COMPILATION on Windows also means we
can't re-use objects between a shared library and a static library on
Windows/MSVC, but have to build everything twice, but that's a job for the
build system to handle (cf. https://github.com/mesonbuild/meson/issues/4133).

We don't need to worry about the export define when building a static library,
if it maps to dllexport it will just be ignored in the static library case.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.