[Mesa-dev] [PATCH 05/12] gallivm: Use LLVM global context.

[Jose Fonseca]

----- Original Message -----
> On 05/14/2014 03:45 AM, Jose Fonseca wrote:
> > ----- Original Message -----
> >>
> >> Hi Jose,
> >>
> >> On Tuesday, May 13, 2014 08:16:34 Jose Fonseca wrote:
> >>> I tweaked the comment (was below) but indeed did a poor job.  I think
> >>> I'll
> >>> just put the original back:
> >>>
> >>>     "We must never free LLVM contexts, because LLVM has several global
> >>>     caches
> >>>     which pointing/derived from objects owned by the context, causing
> >>>     false
> >>>     memory leaks and false cache hits when these objects are destroyed."
> >>
> >> I have not actually tested your current patch series,
> >> but I am wondering if you expect this to be thread safe then
> >> in the sense that multiple compilations from disjoint OpenGL
> >> contexts running in the same application in different but
> >> concurrent threads are run?
> >>
> >> Up to now this is a problem that strikes me with an application here.
> >> For example every openscenegraph viewer based application driving
> >> multiple contexts from a single program suffers from this problem
> >> as they are just crashing on startup when multiple contexts
> >> in their threads compile their shaders. Flightgear is one such example.
> >> Also I know a guy who is working in medical visualization who is
> >> frequently asking me about llvmpipe getting thread safe in this sense.
> >>
> >> Given the documentation about LLVM contexts I would expect this to
> >> crash?
> >>
> >> I have provided a patch some time ago that uses a struct gallivm_state
> >> private allocated LLVM context. That one runs fine with the applications
> >> in question and also it survived a piglit run without regressions.
> >> Did you see this review request?
> >>
> >> So, I really would like to have this problem solved.
> >>
> >> thanks
> >>
> >> Mathias
> >
> > Hi Mathias,
> >
> > No, you're right, this won't be thread safe.  The fact that LLVM internally
> > also uses its global context even when we use private contexts means that
> > fixing this probably require changes to LLVM itself though.
> >
> > Thread safety hasn't been an immediately for us, but I agree it is
> > important for several apps out there, and we benefit from all the testing
> > llvmpipe gets.  So it's worthwhile addressing it.
> 
> How about simply putting a mutex around the code which invokes LLVM?
> It's not ideal by any means, but I think it might work.

It's not easy because some of the LLVM objects used while building the IR are long lived, which means we'll need need to hold mutex, return to caller, and rely on the caller to callback into gallivm to release the mutex.  Which can be a recipe for a deadlock if something happens and the caller doesn't finish the compilation some how.

But now you mention, I just rememered that we'll also need to protect Frank's global JIT manager in a mutex and/or create a separate JIT manager for each thread.

Jose