[Mesa-dev] Rust drivers in Mesa

[Jason Ekstrand]

On Fri, Oct 2, 2020 at 11:34 AM Eric Anholt <eric at anholt.net> wrote:
>
> On Thu, Oct 1, 2020 at 6:36 PM Alyssa Rosenzweig
> <alyssa.rosenzweig at collabora.com> wrote:
> >
> > Hi all,
> >
> > Recently I've been thinking about the potential for the Rust programming
> > language in Mesa. Rust bills itself a safe system programming language
> > with comparable performance to C [0], which is a naturally fit for
> > graphics driver development.
> >
> > Mesa today is written primarily in C, a notoriously low-level language,
> > with some components in C++. To handle the impedance mismatch, we've
> > built up a number of abstractions in-tree, including multiple ad hoc
> > code generators (GenXML, NIR algebraic passes, Bifrost disassembler). A
> > higher level language can help avoid the web of metaprogramming and
> > effect code that is simpler and easier to reason about. Similarly, a
> > better type system can aid static analysis.
> >
> > Beyond abstraction, Rust's differentiating feature is the borrow checker
> > to guarantee memory safety. Historically, safety has not been a primary
> > concern of graphics drivers, since drivers are implemented as regular
> > userspace code running in the process of the application calling them.
> > Unfortunately, now that OpenGL is being exposed to untrusted code via
> > WebGL, the driver does become an attack vector.
> >
> > For the time being, Mesa attempts to minimize memory bugs with defensive
> > programming, safe in-tree abstractions (including ralloc), and static
> > analysis via Coverity. Nevertheless, these are all heuristic solutions.
> > Static analysis is imperfect and in our case, proprietary software.
> > Ideally, the bugs we've been fixing via Coverity could be caught at
> > compile-time with a free and open source toolchain.
> >
> > As Rust would allow exactly this, I see the primary benefit of Rust in
> > verifying correctness and robustness, rather than security concerns per
> > se.  Indeed, safety guarantees do translate well beyond WebGL.
> >
> > Practically, how would Rust fit in with our existing C codebase?
> > Obviously I'm not suggesting a rewrite of Mesa's more than 15 million
> > lines of C. Instead, I see value in introducing Rust in targeted parts
> > of the tree. In particular, I envision backend compilers written in part
> > in Rust. While creating an idiomatic Rust wrapper for NIR or Gallium
> > would be prohibitively costly for now, a backend compiler could be
> > written in Rust with IR builders exported for use of the NIR -> backend
> > IR translator written in C.
> >
> > This would have minimal impact on the tree. Users that are not building
> > such a driver would be unaffected. For those who _are_ building Rust
> > code, the Rust compiler would be added as a build-time dependency and
> > the (statically linked) Rust standard library would be added as a
> > runtime dependency. There is concern about the Rust compiler requiring
> > LLVM as a dependency, but again this is build-time, and no worse than
> > Mesa already requiring LLVM as a runtime dependency for llvmpipe and
> > clover. As for the standard library, it is possible to eliminate the
> > dependency as embedded Rust does, perhaps calling out to the C standard
> > library via the FFI, but this is likely quixotic. I do regret the binary
> > size increase, however.
> >
> > Implications for the build system vary. Rust prefers to be built by its
> > own package manager, Cargo, which is tricky to integrate with other
> > build systems. Actually, Meson has native support for Rust, invoking the
> > compiler directly and skipping Cargo, as if it were C code. This support
> > is not widely adopted as it prevents linking with external libraries
> > ("crates", in Rust parlance), with discussions between Rust and Meson
> > developers ending in a stand-still [1]. For Mesa, this might be just
> > fine. Our out-of-tree run-time dependencies are minimal for the C code,
> > and Rust's standard library largely avoids the need for us to maintain a
> > Rust version of util/ in-tree. If this proves impractical in the
> > long-term, it is possible to integrate Cargo with Meson on our end [2].
> >
> > One outstanding concern is build-time, which has been a notorious
> > growing pain for Rust due to both language design and LLVM itself [3],
> > although there is active work to improve both fronts [4][5]. I build
> > Mesa on my Arm laptop, so I suppose I'd be hurt more than many of us.
> > There's also awkward bootstrapping questions, but there is work here too
> > [6].
> >
> > If this is of interest, please discuss. It's clear to me Rust is not
> > going away any time soon, and I see value in Mesa embracing the new
> > technology. I'd like to hear other Mesa developers' thoughts.
>
> For me, every day I write C code, I wish I was writing rust.  I've
> written hobby rust (https://crates.io/crates/gpu-trace-perf) and also
> dabbled in a huge project (https://servo.org/), and I've gone through
> a bit of the struggles with the borrow checker and come out the other
> side being really convinced that the language is worth it.  Getting to
> write rust for $dayjob is probably the only thing that could drag me
> away from the Mesa project, which I love.
>
> I think we'll miss out on a ton of the benefits of rust by not using
> cargo, but I think for the c/rust interop story today, right now we
> should stick with meson invoking rust, and vendor in any third party
> crates we might want.  (Writing command line tools without structopt?
> no way).
>
> I agree with others that carving off leaf nodes is the way to start
> introducing rust as a dep to Mesa.  If freedreno/turnip folks were
> interested, I'd probably start with doing the backend instruction
> encode/decode/disasm/parsing area -- where Result<> is *really* nice
> to compared to trying to do that pattern in C, and where integrated
> unit tests can really shine.

I've not spent a huge amount of time writing Rust code but about 6
months ago, I decided to write a little compiler in Rust as a toy
project:

https://github.com/jekstrand/ev3opt

While not all that capable, it has a disassembler, an assembler, and a
half-dozen optimization passes (both algebraic styel and
control-flow).  The project is completely useless thanks to
limitations on that particular platform (turns out program overhead
just doesn't matter) but it was a good learning experience.  Here are
the main take-aways I got:

 1. Rust's Result<> mechanism for error handling is awesome.

 2. Rust's enums look awesome but are only mostly awesome:
    a. Pattern matching on them can lead to some pretty deep
indentation which is a bit annoying.
    b. There's no good way to have multiple cases handled by the same
code like you can with a C switch; you have to either repeat it or
break it out into a generic helper.

 3. I miss deterministic destructors (rust and C++ have them; C does not)

 4. The rust standard library is rich and has most of what you want.
Most of util/ would be gone if we had the rust standard library.  (I
don't mean random crates; I only mean the standard ones.)

 5. Move semantics and immutable-by-default are the right choice.
Writing Rust code, I really feel like mut means something which I
can't say for const in C or C++.  The fact that Rust strongly
encourages embedding rather than pointers helps with this.  Also, Rust
code is basically SSA and, as a compiler dev, that makes me really
happy.

 6. The borrow-checker isn't bad at all if you structure your code to
not fight with it.
   a. This means no linked lists.  It's not that they're impossible to
work with; they're just more painful.  That said, ACO has had good
success just using an array and you can do a lot with replacing
instructions with NOP and having a NOP-remove pass.
   b. Pulling two items out of a list and looking at them is painful.
Unfortunately, this is a surprisingly common operation in compiler
passes.  The usual way to get around it is to structure your code such
that you first look up all the information you want with non-mut
things and then you take a mut reference to the thing you want to
change and change it.  This is certainly a safer pattern but it is
cumbersome in a compiler.

 7. At the end of the day, I didn't feel like Rust saved me from many
bugs.  Bugs that would have been SEGEV on an invalid pointer became
OOB array accesses.  Logic bugs were no less common.

The last point makes me really sad but is also just a sobering
reality.  Rust is not a silver bullet.  Yes, it does prevent certain
classes of errors by having things like type safety.  However, in my
experience, shoving the wrong pointer type in the void * in your hash
table is one of those things that gets caught almost immediately if
you have decent test coverage.  It's not the type of subtle bug that's
going to bite you later when some user hits a corner case.  Those tend
to be logic bugs and Rust won't do anything to help you there.

That said, I really do like Rust as a language.  I'm not sure I'd go
quite as far as Anholt but I feel like it has everything I like about
C++ without most of what I dislike.  I'm not sure how we would do
things like ANV's GenXML multi-compile magic but I'm sure we could
figure out a way.  I'd love to see someone build something in Rust and
figure out some best practices.

--Jason