[Mesa-dev] [RFC PATCH 00/16] A new IR for Mesa

[Tom Stellard]

On Tue, Sep 02, 2014 at 12:55:44PM -0700, Eric Anholt wrote:
> Tom Stellard <tom at stellard.net> writes:
> 
> > On Wed, Aug 27, 2014 at 03:07:36PM -0700, Ian Romanick wrote:
> >> On 08/27/2014 02:55 PM, Marek Olšák wrote:
> >> > Our plan is to always require the latest released version of LLVM
> >> > because of new features in our LLVM backend that the radeonsi driver
> >> > depends on to advertise all GL features. Some new features listed for
> >> > the radeonsi driver in Mesa release notes are only enabled if you have
> >> > latest LLVM from git/svn.
> >> 
> >> I think this underscores the fundamental problem have having such a
> >> critical, core piece of project infrastructure being completely out of
> >> the control of the project.  For me, trying to ship a product on which
> >> people rely, this is an absolute non-starter.
> >> 
> >
> > There are a lot of misconceptions in this thread, and I was away last week,
> > so instead of replying to 20 separate emails, I'm replying here.  
> >
> > This discussion is about using LLVM IR as a common IR for Mesa.  The challenges
> > we face in coordinating LLVM and Mesa releases for radeonsi are not relevant.
> > The uses cases of LLVM for radeonsi and LLVM for a common IR are vastly different.
> >
> > A better comparison would be with llvmpipe, which still works with LLVM 3.1,
> > but even llvmpipe uses more LLVM features than would be necessary for a common IR
> > in Mesa.
> >
> > For such usage, Mesa would be required to use only two components of the LLVM libraries.
> > The IR builder functions and the optimization passes.  Both of these can be accessed
> > with the C API which will never change.
> >
> > The only reasons for requiring a specific LLVM versions would be:
> >
> > 1. Bug fixes for the IR build functions
> >
> >   This is very unlikely as the functions just manage data structures and
> >   have been stable for a while.
> >
> > 2. Bug fixes for optimization passes
> >
> >   In this case you can still use older versions, you would just need to disable
> >   the specific optimization pass.
> >
> > 3. Extending the C API to gain access to more optimization passes
> > 4. Better optimizations
> >
> >   Again, you could still use older versions, you would just get
> >   less optimized code.
> >
> > Remember that all the optimizations provided by LLVM are for *common*
> > code and they are essentially free.  Backend developers would still
> > need to write their own register allocators, schedulers, and target
> > specific optimization passes.  And they would still be able to write any
> > other optimization passes they wanted either operating on LLVM IR or their
> > own target specific IR.
> >
> > We could reduce the LLVM dependency even further by letting backends
> > pick which optimization passes to run, so that core Mesa would only
> > depend on the IR builder code, which if statically linked would require only
> > 2 libraries which together are about 7 MB.
> >
> > Most of the LLVM - Mesa dependency problems mentioned in this thread are not
> > applicable to the LLVM dependencies Mesa would have if using LLVM IR
> > as a common IR.  In my opinion, whatever dependency issues there might
> > be are not big enough to ban the use of LLVM in core Mesa, and I don't
> > want to discourage people from trying to use LLVM IR as a common IR
> > if they are interested. 
> 
> Thanks for being really explicit here about how most of this thread has
> been just noise about something orthogonal to what this thread was about
> (the data structures between Mesa core and drivers).
> 
> I'm not really excited about LLVM, even if it's just the C API on both
> sides -- 7 MB seems like a ridiculous amount of code for a compiler, and
> it seems like LLVM users have been struggling with performance for using
> it as an online compiler.  This is even more of a concern for me as I've
> moved to hardware with less CPU to spare.  But I'd love to be proven
> wrong, since writing code sucks.
>

In the common IR scenario, compiler performance (meaning how long it
takes to compile code) can be manged effectively by customizing the list
of optimization passes which are run.  llvmpipe for example, only runs a
small subset of the optimizations to limit compile time.

For radeonsi a majority of the compile time is spent during instruction
selection and latter passes, which would not be used in the common
IR scenario.  This is likely true for all projects which use the LLVM
libraries to emit machine code.  The IR passes don't have much impact on
compile times and the LLVM developers work hard to keep compile times low.

-Tom

> If someone wants to send patches for LLVM with actual data (both
> non-cherrypicked GPU performance data, and compiler overhead data), it
> would be interesting to compare to the current state of things.  I'm not
> optimistic, though.
> 
> I look forward to the Intel guys continuing with their NIR work (which I
> expect them to do, since they've been good about doing core compiler
> work in the past).