gfxstream + Mesa: the deal of the century?

[Gurchetan Singh]

tl;dr

- Interested in porting the gfxstream virtualization solution to
  Mesa and wondering how everyone feels about it

----------
| Longer |
----------

Dear all,

In an earnings call with investors, Santa Claus shared a grim
outlook for the North Pole. The Q4 2023 holiday season's yield of
milk and cookies  -- the currency of the North Pole economy -- was
once again drastically below expectations.

As everyone knows, Santa Claus delivers presents to the people of
the world, and in return is rewarded with milk and cookies.  The
milk and cookies are then used to feed the elves, who then create
next year's presents.  This arrangement has proved remarkably
resilient since time immemorial: surviving war, feminine, plagues
and even Episodes 7 through 9 of the Star Wars saga.

However, the past few years have seen demand for a rather unusual
present skyrocket.  Letter after letter has arrived, saying: "Dear
Santa, can we get gfxstream Vulkan ported to Mesa .. an upstream
gfxsteam?!?"

Jolly old Saint Nick, despite his considerable powers, is unable
to deliver this most demanded present.  As a result, cookie and
milk reserves have fallen to dangerous levels.

Santa Claus has been offered a leveraged buyout of the North Pole by
Wall Street firm to fix the shortfall.  The firm has a radical plan
to cut costs and increase revenues:

   - instead of nutritious milk and cookies, the Elves will be fed
     doritos, sugary drinks and adderall
   - no more "naughty or nice" lists: presents will now be based
     on a monthly subscription fee.
   - selling ads on the sled
   - a dog food factory to 'process' the reindeer.

There is still hope, my friends.  In consultation with the Easter
Bunny, we have come with a plan that just might work.  With this
RFC, we have a working port of gfxstream in Mesa.  If all goes well,
the Easter Bunny should be able to deliver gfxstream + Mesa in
easter eggs, and generate an unprecedented amount of carrots in
return.

The carrots will be converted to milk and cookies, and the North
Pole's economy will once again be thriving.

But for this plan to work, we must port gfxstream into Mesa..

--------------------
| Common Questions |
--------------------

Q: What is gfxstream?

gfxstream is an API-virtualization solution

https://android.googlesource.com/platform/hardware/google/gfxstream/

The history and lore of the project can be found here [a].

Q: Where will this be used and productized?

Pretty much everywhere gfxstream is used now, namely:

- Android Emulator [b]
- Cuttlefish [c]
- Google Play Games [d]
- Fuchsia guests [e]
- AAOS reference guests [f]
- a few other more experimental projects

These projects do reach O(million) users.  Most projects aim for
strong backwards compatibility: i.e, Android images from 2019 still
work on the 2023 host emulator release, and efforts are made to make
sure ABI changes don't break developers.

However, in the non-emulator embedded space, gfxstream-ists do plan to
ship the host driver in the guest.

Q: Is gfxstream upstream in the Linux kernel sense?

gfxstream is officially part of the virtio-gpu spec and supported
on open-source VMMs like QEMU and crosvm.  Recent virtio-gpu additions
like blob resources [g] and context types [h] were first tested/enabled
on gfxstream.

Q: Where is the current MR?

Current MR:

https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246

The MR is very dirty at the moment, but do expect to Mesa-ify and
clean it up further.

It's around 216K LoC, but 70K LoC are pre-compiled shaders which we
can do something about.  We also have patches on top of the Vulkan-Docs
repo (around 30K), which can be upstreamed elsewhere too.  It's also one
big mega-commit rather than 1000s of history preserving ones.

But it should be enough to pass deqp-vk, and see the generalized
structure.

We endeavor to meet all coding standards of the Mesa project.  But
first we want see if there are any strong opinions on the general
idea before investing in further Mesa-ification.

Q: What are the reasons for porting gfxstream to Mesa?

- Non-Android developers have expressed desire to decouple gfxstream
from AOSP.  AOSP is gigantic to download and developers also have to
sign a CLA.  Aside from Mesa, the only real option is Github,
which we can all agree is a downgrade from Mesa.

- The Mesa WSI layer uses gfxstream guest Linux support.  Currently
we just rip out parts of Mesa to make it work, but it's better to be
in-tree.  The other option was the ARM Mesa WSI layer, but Mesa
WSI seemed better maintained.

Q: What are the benefits to Mesa?

As gfxstream becomes a Mesa driver, Mesa will be used more than ever
in virtualized environments.

We also see the benefits for improved maintenance of AOSP Mesa, which
is now on a 3-year old commit.  Having a project that is on the official
support path of many Android projects will likely ensure Mesa Vulkan
builds consistently in AOSP (plan is to use ANGLE for GLES).  We
have some Android.bp files downstream, for example.

I suspect we will be bringing 3-6 additional gfxstream developers into
the Mesa project, and some WSI impls for non-Linux OSes.

Over time, some hardware/software stacks could be added eventually to
the Mesa CI that otherwise would not be there.

Q: What are the costs to Mesa?

Just another directory and more code.  If our attempts to transition
gfxstream consumers to gfxstream/Mesa fail, we can just delete the
directory.

Q: So you're porting both the guest and host portions to Mesa?

Yes, we've tried separate repositories before (with gfxstream-protocols,
vulkan-cereal, and goldfish).  It's comparatively much simpler to make
changes in one repo and also helps with testing as well.

Q: Doesn't Mesa already support Vulkan API virtualization via Venus?
In fact, isn't it funded by ChromeOS?  And isn't gfxstream mostly
an Android effort?  Are you seriously telling me Google .. OF ALL
COMPANIES ... would develop two solutions for the same problem
... *faints*?

Historically speaking, gfxstream Vulkan actually was shipping before
work on venus began. ChromeOS was approached before Venus work began
about gfxstream in 2019, but had different development timelines and
vision in mind.

A cynic might conjecture this as another instance of Google's infamous
"15-chat app" problem, where different parts of the organization don't
really coordinate well.

An optimist might say running virtualizing OS #1 on OS #2 is a tricky
problem, and there are already multiple solutions in Mesa (svga,
virgl).  Given the different design centers + target platforms, any
convergence would necessarily have to follow slime mold dynamics:

https://komoroske.com/slime-mold/

gfxstream in Mesa can thus be seen as the natural progression of the
mold.  Over time, the mold might possibly fully converge in the same
code base.

Whether you take the cynical or optimistic view, having gfxstream in
Mesa is a step in the right direction in both instances.

Q: How long do you expect gfxstream to be around?

We can trace gfxstream back to 2010, and it's still around now.  No one
can predict the future, but I'd say 5 years at the minimum.

Q: You're only porting gfxstream Vulkan, and not gfxstream GLES?

Yes, the reason being Gallium is quite opinionated and gfxstream GLES
is more of a DRI "classic" driver.  It doesn't quite fit.  We do have
backwards compatibility plan using prebuilts, but it can be kept
downstream. gfxstream vulkan does use the Mesa common code.

[a] https://github.com/qemu/qemu/commit/1dcc6adbc168366ece9b7fa3b68a96b475ed4ba8
[b] https://developer.android.com/studio/run/emulator
[c] https://source.android.com/docs/setup/create/cuttlefish-ref-gpu
[d] https://play.google.com/googleplaygames
[e] https://fuchsia.dev/fuchsia-src/development/build/emulator#supported-hardware
[f] https://source.android.com/docs/automotive/virtualization/architecture#graphics
[h] https://lists.freedesktop.org/archives/dri-devel/2020-August/275972.html
[g] https://www.phoronix.com/news/VirtIO-Linux-5.16-Ctx-Type