[Mesa-dev] LLVM gallivm issue in Mesa 12.1.0

[Jose Fonseca]

On 19/10/16 20:38, Marek Olšák wrote:
> On Wed, Oct 19, 2016 at 9:17 PM, Jose Fonseca <jfonseca at vmware.com> wrote:
>> On 19/10/16 18:35, Matt Turner wrote:
>>>
>>> On Wed, Oct 19, 2016 at 9:54 AM, Marek Olšák <maraeo at gmail.com> wrote:
>>>>
>>>> On Wed, Oct 19, 2016 at 6:06 PM, Emil Velikov <emil.l.velikov at gmail.com>
>>>> wrote:
>>>>>
>>>>> On 19 October 2016 at 15:55, Marek Olšák <maraeo at gmail.com> wrote:
>>>>>>
>>>>>> On Wed, Oct 19, 2016 at 12:47 PM, Emil Velikov
>>>>>> <emil.l.velikov at gmail.com> wrote:
>>>>>>>
>>>>>>> On 19 October 2016 at 11:35, Grigori Goronzy <greg at chown.ath.cx>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On 2016-10-04 12:32, Emil Velikov wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 2 October 2016 at 14:17, Axel Davy <axel.davy at ens.fr> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I'd prefer myself Oct 14, because we have a lot of patches for
>>>>>>>>>> nine, and
>>>>>>>>>> they deserve more cleaning and testing, but if it's Oct 7, we'll
>>>>>>>>>> try be
>>>>>>>>>> on
>>>>>>>>>> time.
>>>>>>>>>>
>>>>>>>>> 14th it is. As mentioned before: _don't_ wait for the last week to
>>>>>>>>> get
>>>>>>>>> things merged. Once you're reasonably happy just send the new work
>>>>>>>>> review and commit it.
>>>>>>>>> Same applies for bugfixes :-)
>>>>>>>>>
>>>>>>>>
>>>>>>>> What happened to these plans? It is the October 19th already. Nine
>>>>>>>> fixes
>>>>>>>> have trickled into Mesa and radv was merged also. What's the holdup?
>>>>>>>>
>>>>>>> I've spent a (bit too many) days on trying to get things working with
>>>>>>> LLVM 3.9.
>>>>>>> So on my end, I'll consider it broken and let the LLVM wizards take
>>>>>>> care of it.
>>>>>>
>>>>>>
>>>>>> Is the LLVM 3.9 issue related to radeonsi?
>>>>>>
>>>>> Plain gallium, as per below.
>>>>>
>>>>> ../../../../src/gallium/auxiliary/.libs/libgallium.a(lp_bld_misc.o):
>>>>> In function
>>>>> `llvm::RTDyldMemoryManager::getSymbolAddress(std::__cxx11::basic_string<char,
>>>>> std::char_tra
>>>>> its<char>, std::allocator<char> > const&)':
>>>>> /usr/local/include/llvm/ExecutionEngine/RTDyldMemoryManager.h:87:
>>>>> undefined reference to
>>>>>
>>>>> `llvm::RTDyldMemoryManager::getSymbolAddressInProcess(std::__cxx11::basic_string<ch
>>>>> ar, std::char_traits<char>, std::allocator<char> > const&)'
>>>>> /usr/local/include/llvm/ExecutionEngine/RTDyldMemoryManager.h:87:
>>>>> undefined reference to
>>>>>
>>>>> `llvm::RTDyldMemoryManager::getSymbolAddressInProcess(std::__cxx11::basic_string<ch
>>>>> ar, std::char_traits<char>, std::allocator<char> > const&)'
>>>>>
>>>>> An identical symbol with different signature is provided by the static
>>>>> lib and header:
>>>>>
>>>>> $ objdump -CtT libLLVMRuntimeDyld.a | grep
>>>>> "llvm::RTDyldMemoryManager::getSymbolAddressInProcess"
>>>>> ... 0000000000000149
>>>>> llvm::RTDyldMemoryManager::getSymbolAddressInProcess(std::string
>>>>> const&)
>>>>>
>>>>> $ grep getSymbolAddressInProcess .../include/
>>>>> RTDyldMemoryManager.h:  static uint64_t
>>>>> getSymbolAddressInProcess(const std::string &Name);
>>>>>
>>>>> And in the LLVM 3.8 case (which works like a charm):
>>>>>
>>>>> $ objdump -CtT libLLVMRuntimeDyld.a | grep
>>>>> "llvm::RTDyldMemoryManager::getSymbolAddressInProcess"
>>>>> ...0000000000000181
>>>>>
>>>>> llvm::RTDyldMemoryManager::getSymbolAddressInProcess(std::__cxx11::basic_string<char,
>>>>> std::char_traits<char>, std::allocator<char> > const&)
>>>>>
>>>>> $ grep getSymbolAddressInProcess .../include/
>>>>> RTDyldMemoryManager.h:  static uint64_t
>>>>> getSymbolAddressInProcess(const std::string &Name);
>>>>>
>>>>> It smells like partial/incomplete build with the C++11 ABI, but trying
>>>>> to untangle the lot is quite time consuming.
>>>>
>>>>
>>>> I admit I have no idea what all that means.
>>>
>>>
>>> This looks like C++ ABI mismatch. See
>>> https://developerblog.redhat.com/2015/02/05/gcc5-and-the-c11-abi/ for
>>> details.
>>>
>>> Recompile LLVM and Mesa with the same g++.
>>> _______________________________________________
>>> mesa-dev mailing list
>>> mesa-dev at lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
>>>
>>
>> Perhaps the problem is that LLVM 3.9 is built with -std=c++11:
>> $ llvm-config-3.9 --cxxflags
>> -I/usr/lib/llvm-3.9/include -std=c++0x -gsplit-dwarf -Wl,-fuse-ld=gold -fPIC
>> -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings
>> -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long
>> -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment
>> -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -O2 -g
>> -DNDEBUG  -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS
>> -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
>>
>> I know we've been cherrypicking which bits of llvm-config's output we want.
>> Perhaps we need to make sure we pick --std=c++11.
>>
>> But this also implies that *all* Mesa needs to build reliably with
>> --std=c++11.  I recall there were problems some time ago when OpenSWR wanted
>> to use c++11.  So it might not be trivial.
>>
>>
>> I also looked up into replace our lp_build_create_jit_compiler_for_module
>> with some standard LLVM-C binding entry point.  But
>> LLVMCreateMCJITCompilerForModule doesn't seem to do nowhere near the stuff
>> we do...
>
> Can we add C wrappers of the necessary lp function into LLVM?

It's certainly desirable to have LLVM C wrappers to allow o do the same. 
  But would require careful thought and design.

A one-size-fits-all LLVMCreateMCJITJustLikeMesaWantsIt that matches the 
current lp_build_create_jit_compiler_for_module implementation won't 
work: upstream would never accept it, and even we had to change the 
internals of lp_build_create_jit_compiler_for_module many times (and not 
just to fix LLVM breakage.)

So we'd need several C bindings to do all the things we do.  It's not a 
straightfoward project.

Jose