[Mesa-dev] LLVM gallivm issue in Mesa 12.1.0

[Jose Fonseca]

On 19/10/16 20:59, Jose Fonseca wrote:
> On 19/10/16 20:38, Marek Olšák wrote:
>> On Wed, Oct 19, 2016 at 9:17 PM, Jose Fonseca <jfonseca at vmware.com>
>> wrote:
>>> On 19/10/16 18:35, Matt Turner wrote:
>>>>
>>>> On Wed, Oct 19, 2016 at 9:54 AM, Marek Olšák <maraeo at gmail.com> wrote:
>>>>>
>>>>> On Wed, Oct 19, 2016 at 6:06 PM, Emil Velikov
>>>>> <emil.l.velikov at gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> On 19 October 2016 at 15:55, Marek Olšák <maraeo at gmail.com> wrote:
>>>>>>>
>>>>>>> On Wed, Oct 19, 2016 at 12:47 PM, Emil Velikov
>>>>>>> <emil.l.velikov at gmail.com> wrote:
>>>>>>>>
>>>>>>>> On 19 October 2016 at 11:35, Grigori Goronzy <greg at chown.ath.cx>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> On 2016-10-04 12:32, Emil Velikov wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 2 October 2016 at 14:17, Axel Davy <axel.davy at ens.fr> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I'd prefer myself Oct 14, because we have a lot of patches for
>>>>>>>>>>> nine, and
>>>>>>>>>>> they deserve more cleaning and testing, but if it's Oct 7, we'll
>>>>>>>>>>> try be
>>>>>>>>>>> on
>>>>>>>>>>> time.
>>>>>>>>>>>
>>>>>>>>>> 14th it is. As mentioned before: _don't_ wait for the last
>>>>>>>>>> week to
>>>>>>>>>> get
>>>>>>>>>> things merged. Once you're reasonably happy just send the new
>>>>>>>>>> work
>>>>>>>>>> review and commit it.
>>>>>>>>>> Same applies for bugfixes :-)
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What happened to these plans? It is the October 19th already. Nine
>>>>>>>>> fixes
>>>>>>>>> have trickled into Mesa and radv was merged also. What's the
>>>>>>>>> holdup?
>>>>>>>>>
>>>>>>>> I've spent a (bit too many) days on trying to get things working
>>>>>>>> with
>>>>>>>> LLVM 3.9.
>>>>>>>> So on my end, I'll consider it broken and let the LLVM wizards take
>>>>>>>> care of it.
>>>>>>>
>>>>>>>
>>>>>>> Is the LLVM 3.9 issue related to radeonsi?
>>>>>>>
>>>>>> Plain gallium, as per below.
>>>>>>
>>>>>> ../../../../src/gallium/auxiliary/.libs/libgallium.a(lp_bld_misc.o):
>>>>>> In function
>>>>>> `llvm::RTDyldMemoryManager::getSymbolAddress(std::__cxx11::basic_string<char,
>>>>>>
>>>>>> std::char_tra
>>>>>> its<char>, std::allocator<char> > const&)':
>>>>>> /usr/local/include/llvm/ExecutionEngine/RTDyldMemoryManager.h:87:
>>>>>> undefined reference to
>>>>>>
>>>>>> `llvm::RTDyldMemoryManager::getSymbolAddressInProcess(std::__cxx11::basic_string<ch
>>>>>>
>>>>>> ar, std::char_traits<char>, std::allocator<char> > const&)'
>>>>>> /usr/local/include/llvm/ExecutionEngine/RTDyldMemoryManager.h:87:
>>>>>> undefined reference to
>>>>>>
>>>>>> `llvm::RTDyldMemoryManager::getSymbolAddressInProcess(std::__cxx11::basic_string<ch
>>>>>>
>>>>>> ar, std::char_traits<char>, std::allocator<char> > const&)'
>>>>>>
>>>>>> An identical symbol with different signature is provided by the
>>>>>> static
>>>>>> lib and header:
>>>>>>
>>>>>> $ objdump -CtT libLLVMRuntimeDyld.a | grep
>>>>>> "llvm::RTDyldMemoryManager::getSymbolAddressInProcess"
>>>>>> ... 0000000000000149
>>>>>> llvm::RTDyldMemoryManager::getSymbolAddressInProcess(std::string
>>>>>> const&)
>>>>>>
>>>>>> $ grep getSymbolAddressInProcess .../include/
>>>>>> RTDyldMemoryManager.h:  static uint64_t
>>>>>> getSymbolAddressInProcess(const std::string &Name);
>>>>>>
>>>>>> And in the LLVM 3.8 case (which works like a charm):
>>>>>>
>>>>>> $ objdump -CtT libLLVMRuntimeDyld.a | grep
>>>>>> "llvm::RTDyldMemoryManager::getSymbolAddressInProcess"
>>>>>> ...0000000000000181
>>>>>>
>>>>>> llvm::RTDyldMemoryManager::getSymbolAddressInProcess(std::__cxx11::basic_string<char,
>>>>>>
>>>>>> std::char_traits<char>, std::allocator<char> > const&)
>>>>>>
>>>>>> $ grep getSymbolAddressInProcess .../include/
>>>>>> RTDyldMemoryManager.h:  static uint64_t
>>>>>> getSymbolAddressInProcess(const std::string &Name);
>>>>>>
>>>>>> It smells like partial/incomplete build with the C++11 ABI, but
>>>>>> trying
>>>>>> to untangle the lot is quite time consuming.
>>>>>
>>>>>
>>>>> I admit I have no idea what all that means.
>>>>
>>>>
>>>> This looks like C++ ABI mismatch. See
>>>> https://developerblog.redhat.com/2015/02/05/gcc5-and-the-c11-abi/ for
>>>> details.
>>>>
>>>> Recompile LLVM and Mesa with the same g++.
>>>> _______________________________________________
>>>> mesa-dev mailing list
>>>> mesa-dev at lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
>>>>
>>>
>>> Perhaps the problem is that LLVM 3.9 is built with -std=c++11:
>>> $ llvm-config-3.9 --cxxflags
>>> -I/usr/lib/llvm-3.9/include -std=c++0x -gsplit-dwarf
>>> -Wl,-fuse-ld=gold -fPIC
>>> -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter
>>> -Wwrite-strings
>>> -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long
>>> -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment
>>> -Werror=date-time -std=c++11 -ffunction-sections -fdata-sections -O2 -g
>>> -DNDEBUG  -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS
>>> -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
>>>
>>> I know we've been cherrypicking which bits of llvm-config's output we
>>> want.
>>> Perhaps we need to make sure we pick --std=c++11.
>>>
>>> But this also implies that *all* Mesa needs to build reliably with
>>> --std=c++11.  I recall there were problems some time ago when OpenSWR
>>> wanted
>>> to use c++11.  So it might not be trivial.
>>>
>>>
>>> I also looked up into replace our
>>> lp_build_create_jit_compiler_for_module
>>> with some standard LLVM-C binding entry point.  But
>>> LLVMCreateMCJITCompilerForModule doesn't seem to do nowhere near the
>>> stuff
>>> we do...
>>
>> Can we add C wrappers of the necessary lp function into LLVM?
>
> It's certainly desirable to have LLVM C wrappers to allow o do the same.
>  But would require careful thought and design.
>
> A one-size-fits-all LLVMCreateMCJITJustLikeMesaWantsIt that matches the
> current lp_build_create_jit_compiler_for_module implementation won't
> work: upstream would never accept it, and even we had to change the
> internals of lp_build_create_jit_compiler_for_module many times (and not
> just to fix LLVM breakage.)
>
> So we'd need several C bindings to do all the things we do.  It's not a
> straightfoward project.

In short, what I mean is, that's the direction I'd like to go in the 
long term, but I'm afraid neither me nor Roland will have the time to 
pursue this in the near term.

Jose