Advice on modifying Lavapipe to isolate JIT compilation in separate process

Jose Fonseca jfonseca at
Thu Apr 27 13:23:06 UTC 2023

Perhaps I'm getting confused with the terminology, but I don't think moving compilation to a separate process helps here.  IIUC, compilation (as in LLVM IR -> x86 code) can happen anywhere, the problem is loading the JITed code (ie, make writeable memory executable.)

As mentioned, there are many places this can be done.

If Venus does not suite your needs, the easiest way to achieve this would be to:
- back all buffer/texture memory with shared memory, visible to the 2nd process
- modify gallivm to tell LLVM either spit out .so files, or to use shared memory
- modify gallivm_jit_function to return a wrapper that marshals the call into the 2nd process:
  - either a generic C wrapper which can introspect the LLVM IR Function arguments
  - or hand written C wrappers for every function prototype returned by gallivm_jit_function

- there are also a few places where JIT code refers to host process memory addresses explicitly (e.g, util_format_xxx helper functions) which need to be handled separately (e.g, by passing these addresses in a structure which can be rewritten to match the 2nd process)


From: mesa-dev <mesa-dev-bounces at> on behalf of Dave Airlie <airlied at>
Sent: Thursday, April 27, 2023 08:39
To: Josh Gargus <jjosh at>
Cc: mesa-dev at <mesa-dev at>
Subject: Re: Advice on modifying Lavapipe to isolate JIT compilation in separate process

!! External Email

On Thu, 27 Apr 2023 at 15:18, Josh Gargus <jjosh at> wrote:
> Thanks for your advice!  I hadn't looked at Venus, but that seems like a very promising place to start.
> The other approach feels more approachable now too; it feels like there are less "unknown unknowns", although there are plenty of known unknowns to investigate (address independence was one that was already bugging be before I wrote to this list).

I think it shouldn't be too horrible to work out, another option might
be to abuse the cache somehow, but I think that still needs writable +
executable which probably doesn't help, but stuff should be address
independent as I do write x86 asm programs to the cache and read them
back out, only relocating around the global symbols.

> It seems like Venus is the more straightforward approach, so I'm inclined to just go with it.  However, it seems like there would be a performance hit compared to only doing JIT compilation in a separate process.  Do you have a rough sense of the performance hit of serializing everything over Venus?  The answer will depend on the workload, I know.

Yeah I think you'll definitely see a large perf hit than just moving
compilation out to a separate process, I'm not sure of the raw venus
overhead numbers here, someone else might have more information


!! External Email: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the mesa-dev mailing list