"Failed to load plugin" / "undefined symbol" using ThreadSanitizer (TSan)

[Juan Navarro]

Hi,

I'm trying to run the Kurento Media Server application with the 
ThreadSanitizer (aka TSan) data race detector, which comes included in 
GCC and Clang compilers. However a runtime error happens while updating 
the plugin cache, related to name resolution on shared libraries. I 
think it is an issue that probably would happen to any GStreamer 
application, so it's (maybe?) not Kurento-specific.

This is a couple examples of the errors, which happen right after doing 
"rm -rf ~/.cache/gstreamer*" and starting the application:

     (gst-plugin-scanner:17057): GStreamer-WARNING **: Failed to load 
plugin './kms-core/src/server/libkmscoreimpl.so': 
./kms-jsonrpc/src/libjsonrpc.so.1: undefined symbol: __tsan_vptr_read

     (gst-plugin-scanner:17057): GStreamer-WARNING **: Failed to load 
plugin './kms-core/src/server/libkmscoremodule.so': 
./kms-core/src/gst-plugins/commons/rtpsync/libkmsrtpsync.so.6: undefined 
symbol: __tsan_init

"kmscore" refers to a set of libraries which should get loaded as 
GStreamer modules.

I have studied the situation and all Google had to offer, and my 
conclusion is that the tsan symbols (__tsan_*) are undefined in the 
shared libraries generated by the build process, but they are actually 
present in the final executable. As far as I know, Clang links the 
static tsan library, and uses the linker option `--whole-archive`, in 
order to ensure that the executable does have all symbols from the 
library, even seemingly unused ones. This happens with the intention 
that later loaded (or dlopen()-ed) shared libraries will be able to find 
any tsan symbols that they might need.

Examples:
     $ nm -g ./kms-jsonrpc/src/libjsonrpc.so.1 | grep tsan
         U __tsan_vptr_read
         (... and more symbols)
     $ nm -g 
./kms-core/src/gst-plugins/commons/rtpsync/libkmsrtpsync.so.6 | grep tsan
         U __tsan_init
         (... and more symbols)
     $ nm -g ./kurento-media-server/server/kurento-media-server | grep tsan
         000000000014beb0 T __tsan_vptr_read
         000000000013b750 T __tsan_init
         (... and more symbols)

However, the GStreamer plugin cache is updated by an independent 
program, "gst-plugin-scanner", which is obviously not compiled with the 
tsan symbols. GStreamer modules/plugins are loaded by default by the 
function gst_plugin_load_file() (in gstplugin.c), which in turn uses 
GLib's g_module_open() (in gmodule.c). The later one can be called with 
the flag "G_MODULE_BIND_LAZY", and that would make it work if it was the 
main application loading the plugins. However, this flag won't make any 
difference for the program "gst-plugin-scanner", which will just load 
the plugins and deem them as invalid after not being able to resolve all 
those "__tsan_*" symbols.

Are my assumptions correct? Does anyone here have experience with this 
issue or with the AddressSanitizer faimily of tools, and knows any way 
to solve it? As I mentioned earlier, I guess that any application making 
use of GStreamer plugins must have been suffering the same issue; if 
that's the case then it becomes more interesting for the GStreamer 
project as a whole, instead of being just my specific use-case.

Version info:
* Kurento Media Server 6.6.1
* GStreamer 1.8.1 (provided by Kurento repos)
* Clang 3.8 (also tested latest snapshot, 5.0)
* GCC 5.4.1 (also tested latest snapshot, 7.1.0)

Thanks and regards,
Juan