[Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem

[Nicolas Dufresne]

Le mardi 17 mars 2020 à 11:27 -0500, Jason Ekstrand a écrit :
> On Tue, Mar 17, 2020 at 10:33 AM Nicolas Dufresne <nicolas at ndufresne.ca> wrote:
> > Le lundi 16 mars 2020 à 23:15 +0200, Laurent Pinchart a écrit :
> > > Hi Jason,
> > > 
> > > On Mon, Mar 16, 2020 at 10:06:07AM -0500, Jason Ekstrand wrote:
> > > > On Mon, Mar 16, 2020 at 5:20 AM Laurent Pinchart wrote:
> > > > > On Wed, Mar 11, 2020 at 04:18:55PM -0400, Nicolas Dufresne wrote:
> > > > > > (I know I'm going to be spammed by so many mailing list ...)
> > > > > > 
> > > > > > Le mercredi 11 mars 2020 à 14:21 -0500, Jason Ekstrand a écrit :
> > > > > > > On Wed, Mar 11, 2020 at 12:31 PM Jason Ekstrand <jason at jlekstrand.net> wrote:
> > > > > > > > All,
> > > > > > > > 
> > > > > > > > Sorry for casting such a broad net with this one. I'm sure most people
> > > > > > > > who reply will get at least one mailing list rejection.  However, this
> > > > > > > > is an issue that affects a LOT of components and that's why it's
> > > > > > > > thorny to begin with.  Please pardon the length of this e-mail as
> > > > > > > > well; I promise there's a concrete point/proposal at the end.
> > > > > > > > 
> > > > > > > > 
> > > > > > > > Explicit synchronization is the future of graphics and media.  At
> > > > > > > > least, that seems to be the consensus among all the graphics people
> > > > > > > > I've talked to.  I had a chat with one of the lead Android graphics
> > > > > > > > engineers recently who told me that doing explicit sync from the start
> > > > > > > > was one of the best engineering decisions Android ever made.  It's
> > > > > > > > also the direction being taken by more modern APIs such as Vulkan.
> > > > > > > > 
> > > > > > > > 
> > > > > > > > ## What are implicit and explicit synchronization?
> > > > > > > > 
> > > > > > > > For those that aren't familiar with this space, GPUs, media encoders,
> > > > > > > > etc. are massively parallel and synchronization of some form is
> > > > > > > > required to ensure that everything happens in the right order and
> > > > > > > > avoid data races.  Implicit synchronization is when bits of work (3D,
> > > > > > > > compute, video encode, etc.) are implicitly based on the absolute
> > > > > > > > CPU-time order in which API calls occur.  Explicit synchronization is
> > > > > > > > when the client (whatever that means in any given context) provides
> > > > > > > > the dependency graph explicitly via some sort of synchronization
> > > > > > > > primitives.  If you're still confused, consider the following
> > > > > > > > examples:
> > > > > > > > 
> > > > > > > > With OpenGL and EGL, almost everything is implicit sync.  Say you have
> > > > > > > > two OpenGL contexts sharing an image where one writes to it and the
> > > > > > > > other textures from it.  The way the OpenGL spec works, the client has
> > > > > > > > to make the API calls to render to the image before (in CPU time) it
> > > > > > > > makes the API calls which texture from the image.  As long as it does
> > > > > > > > this (and maybe inserts a glFlush?), the driver will ensure that the
> > > > > > > > rendering completes before the texturing happens and you get correct
> > > > > > > > contents.
> > > > > > > > 
> > > > > > > > Implicit synchronization can also happen across processes.  Wayland,
> > > > > > > > for instance, is currently built on implicit sync where the client
> > > > > > > > does their rendering and then does a hand-off (via wl_surface::commit)
> > > > > > > > to tell the compositor it's done at which point the compositor can now
> > > > > > > > texture from the surface.  The hand-off ensures that the client's
> > > > > > > > OpenGL API calls happen before the server's OpenGL API calls.
> > > > > > > > 
> > > > > > > > A good example of explicit synchronization is the Vulkan API.  There,
> > > > > > > > a client (or multiple clients) can simultaneously build command
> > > > > > > > buffers in different threads where one of those command buffers
> > > > > > > > renders to an image and the other textures from it and then submit
> > > > > > > > both of them at the same time with instructions to the driver for
> > > > > > > > which order to execute them in.  The execution order is described via
> > > > > > > > the VkSemaphore primitive.  With the new VK_KHR_timeline_semaphore
> > > > > > > > extension, you can even submit the work which does the texturing
> > > > > > > > BEFORE the work which does the rendering and the driver will sort it
> > > > > > > > out.
> > > > > > > > 
> > > > > > > > The #1 problem with implicit synchronization (which explicit solves)
> > > > > > > > is that it leads to a lot of over-synchronization both in client space
> > > > > > > > and in driver/device space.  The client has to synchronize a lot more
> > > > > > > > because it has to ensure that the API calls happen in a particular
> > > > > > > > order.  The driver/device have to synchronize a lot more because they
> > > > > > > > never know what is going to end up being a synchronization point as an
> > > > > > > > API call on another thread/process may occur at any time.  As we move
> > > > > > > > to more and more multi-threaded programming this synchronization (on
> > > > > > > > the client-side especially) becomes more and more painful.
> > > > > > > > 
> > > > > > > > 
> > > > > > > > ## Current status in Linux
> > > > > > > > 
> > > > > > > > Implicit synchronization in Linux works via a the kernel's internal
> > > > > > > > dma_buf and dma_fence data structures.  A dma_fence is a tiny object
> > > > > > > > which represents the "done" status for some bit of work.  Typically,
> > > > > > > > dma_fences are created as a by-product of someone submitting some bit
> > > > > > > > of work (say, 3D rendering) to the kernel.  The dma_buf object has a
> > > > > > > > set of dma_fences on it representing shared (read) and exclusive
> > > > > > > > (write) access to the object.  When work is submitted which, for
> > > > > > > > instance renders to the dma_buf, it's queued waiting on all the fences
> > > > > > > > on the dma_buf and and a dma_fence is created representing the end of
> > > > > > > > said rendering work and it's installed as the dma_buf's exclusive
> > > > > > > > fence.  This way, the kernel can manage all its internal queues (3D
> > > > > > > > rendering, display, video encode, etc.) and know which things to
> > > > > > > > submit in what order.
> > > > > > > > 
> > > > > > > > For the last few years, we've had sync_file in the kernel and it's
> > > > > > > > plumbed into some drivers.  A sync_file is just a wrapper around a
> > > > > > > > single dma_fence.  A sync_file is typically created as a by-product of
> > > > > > > > submitting work (3D, compute, etc.) to the kernel and is signaled when
> > > > > > > > that work completes.  When a sync_file is created, it is guaranteed by
> > > > > > > > the kernel that it will become signaled in finite time and, once it's
> > > > > > > > signaled, it remains signaled for the rest of time.  A sync_file is
> > > > > > > > represented in UAPIs as a file descriptor and can be used with normal
> > > > > > > > file APIs such as dup().  It can be passed into another UAPI which
> > > > > > > > does some bit of queue'd work and the submitted work will wait for the
> > > > > > > > sync_file to be triggered before executing.  A sync_file also supports
> > > > > > > > poll() if  you want to wait on it manually.
> > > > > > > > 
> > > > > > > > Unfortunately, sync_file is not broadly used and not all kernel GPU
> > > > > > > > drivers support it.  Here's a very quick overview of my understanding
> > > > > > > > of the status of various components (I don't know the status of
> > > > > > > > anything in the media world):
> > > > > > > > 
> > > > > > > >  - Vulkan: Explicit synchronization all the way but we have to go
> > > > > > > > implicit as soon as we interact with a window-system.  Vulkan has APIs
> > > > > > > > to import/export sync_files to/from it's VkSemaphore and VkFence
> > > > > > > > synchronization primitives.
> > > > > > > >  - OpenGL: Implicit all the way.  There are some EGL extensions to
> > > > > > > > enable some forms of explicit sync via sync_file but OpenGL itself is
> > > > > > > > still implicit.
> > > > > > > >  - Wayland: Currently depends on implicit sync in the kernel (accessed
> > > > > > > > via EGL/OpenGL).  There is an unstable extension to allow passing
> > > > > > > > sync_files around but it's questionable how useful it is right now
> > > > > > > > (more on that later).
> > > > > > > >  - X11: With present, it has these "explicit" fence objects but
> > > > > > > > they're always a shmfence which lets the X server and client do a
> > > > > > > > userspace CPU-side hand-off without going over the socket (and
> > > > > > > > round-tripping through the kernel).  However, the only thing that
> > > > > > > > fence does is order the OpenGL API calls in the client and server and
> > > > > > > > the real synchronization is still implicit.
> > > > > > > >  - linux/i915/gem: Fully supports using sync_file or syncobj for explicit
> > > > > > > > sync.
> > > > > > > >  - linux/amdgpu: Supports sync_file and syncobj but it still
> > > > > > > > implicitly syncs sometimes due to it's internal memory residency
> > > > > > > > handling which can lead to over-synchronization.
> > > > > > > >  - KMS: Implicit sync all the way.  There are no KMS APIs which take
> > > > > > > > explicit sync primitives.
> > > > > > > 
> > > > > > > Correction:  Apparently, I missed some things.  If you use atomic, KMS
> > > > > > > does have explicit in- and out-fences.  Non-atomic users (e.g. X11)
> > > > > > > are still in trouble but most Wayland compositors use atomic these
> > > > > > > days
> > > > > > > 
> > > > > > > >  - v4l: ???
> > > > > > > >  - gstreamer: ???
> > > > > > > >  - Media APIs such as vaapi etc.:  ???
> > > > > > 
> > > > > > GStreamer is consumer for V4L2, VAAPI and other stuff. Using asynchronous buffer
> > > > > > synchronisation is something we do already with GL (even if limited). We place
> > > > > > GLSync object in the pipeline and attach that on related GstBuffer. We wait on
> > > > > > these GLSync as late as possible (or superseed the sync if we queue more work
> > > > > > into the same GL context). That requires a special mode of operation of course.
> > > > > > We don't usually like making lazy blocking call implicit, as it tends to cause
> > > > > > random issues. If we need to wait, we think it's better to wait int he module
> > > > > > that is responsible, so in general, we try to negotiate and fallback locally
> > > > > > (it's plugin base, so this can be really messy otherwise).
> > > > > > 
> > > > > > So basically this problem needs to be solved in V4L2, VAAPI and other lower
> > > > > > level APIs first. We need API that provides us these fence (in or out), and then
> > > > > > we can consider using them. For V4L2, there was an attempt, but it was a bit of
> > > > > > a miss-fit. Your proposal could work, need to be tested I guess, but it does not
> > > > > > solve some of other issues that was discussed. Notably for camera capture, were
> > > > > > the HW timestamp is capture about at the same time the frame is ready. But the
> > > > > > timestamp is not part of the paylaod, so you need an entire API asynchronously
> > > > > > deliver that metadata. It's the biggest pain point I've found, such an API would
> > > > > > be quite invasive or if made really generic, might just never be adopted widely
> > > > > > enough.
> > > > > 
> > > > > Another issue is that V4L2 doesn't offer any guarantee on job ordering.
> > > > > When you queue multiple buffers for camera capture for instance, you
> > > > > don't know until capture complete in which buffer the frame has been
> > > > > captured.
> > > > 
> > > > Is this a Kernel UAPI issue?  Surely the kernel driver knows at the
> > > > start of frame capture which buffer it's getting written into.  I
> > > > would think that the kernel APIs could be adjusted (if we find good
> > > > reason to do so!) such that they return earlier and return a (buffer,
> > > > fence) pair.  Am I missing something fundamental about video here?
> > > 
> > > For cameras I believe we could do that, yes. I was pointing out the
> > > issues caused by the current API. For video decoders I'll let Nicolas
> > > answer the question, he's way more knowledgeable that I am on that
> > > topic.
> > 
> > Right now, there is simply no uAPI for supporting asynchronous errors
> > reporting when fences are invovled. That is true for both camera's and
> > CODEC. It's likely what all the attempt was missing, I don't know
> > enough myself to suggest something.
> > 
> > Now, why Stateless video decoders are special is another subject. In
> > CODECs, the decoding and the presentation order may differ. For
> > Stateless kind of CODEC, a bitstream is passed to the HW. We don't know
> > if this bitstream is fully valid, since the it is being parsed and
> > validated by the firmware. It's also firmware job to decide which
> > buffer should be presented first.
> > 
> > In most firmware interface, that information is communicated back all
> > at once when the frame is ready to be presented (which may be quite
> > some time after it was decoded). So indeed, a fence model is not really
> > easy to add, unless the firmware was designed with that model in mind.
> 
> Just to be clear, I think we should do whatever makes sense here and
> not try to slam sync_file in when it doesn't make sense just because
> we have it.  The more I read on this thread, the less out-fences from
> video decode sound like they make sense unless we have a really solid
> plan for async error reporting.  It's possible, depending on how many
> processes are involved in the pipeline, that async error reporting
> could help reduce latency a bit if it let the kernel report the error
> directly to the last process in the chain.  However, I'm not convinced
> the potential for userspace programmer error is worth it..  That said,
> I'm happy to leave that up to the actual video experts. (I just do 3D)
> 
> > Nothing of course would prevent V4L2 framework to generically handle
> > out_fence from other producers. It does not even handle implicit fences
> > at the moment, which is already quite problematic (I've seen glitches
> > on i.MX6/8 and Raspberry Pi 4).
> > 
> > In that specific case, if the fences from etnaviv, vc graphic drivers
> > was exposed, we could solve this issue in userspace. Right now it's
> > implicit, so we rely on all DMABuf driver to have proper support, which
> > is not the case. There is V4L2 support for that coming, but the wait is
> > done synchronously in userspace call that was normally non-blocking. So
> > that is unlikely to fly.
> 
> Yeah... waits in userspace aren't what anyone wants.
> 
> > Small note, stateless video decoders don't have this issue. The
> > bitstream is validated by userspace, and userspace controls the
> > "decode" operation. This one would be a good case for bidirectional
> > fencing.
> 
> Good to know.
> 
> > > > I must admit that V4L is a bit of an odd case since the kernel driver
> > > > is the producer and not the consumer.
> > > 
> > > Note that V4L2 can be a consumer too. Video output with V4L2 is less
> > > frequent than video capture (but it still exists), and codecs and other
> > > memory-to-memory processing devices (colorspace converters, scalers,
> > > ...) are both consumers and producers.
> > > 
> > > > > In the normal case buffers are processed in sequence, but if
> > > > > an error occurs during capture, they can be recycled internally and put
> > > > > to the back of the queue.
> > > > 
> > > > Are those errors something that can happen at any time in the middle
> > > > of a frame capture?  If so, that does make things stickier.
> > > 
> > > Yes it can. Think of packet loss when capturing from a USB webcam for
> > > instance.
> > > 
> > > > > Unless I'm mistaken, this problem also exists
> > > > > with stateful codecs. And if you don't know in advance which buffer you
> > > > > will receive from the device, the usefulness of fences becomes very
> > > > > questionable :-)
> > > > 
> > > > Yeah, if you really are in a situation where there's no way to know
> > > > until the full frame capture has been completed which buffer is next,
> > > > then fences are useless.  You aren't in an implicit synchronization
> > > > setting either; you're in a "full flush" setting.  It's arguably worse
> > > > for performance but perhaps unavoidable?
> > > 
> > > Probably unavoidable in some cases, but nothing that should get in the
> > > way for the discussion at hand: there's no need to migrate away from
> > > implicit sync when there's implicit sync in the first place :-)
> > > 
> > > I think we need to analyse the use cases here, and figure out at least
> > > guidelines for userspace, otherwise applications will wonder what
> > > behaviour to implement, and we'll end up with a wide variety of them.
> > > Even just on the kernel side, some V4L2 capture driver will pass
> > > erroneous frames to userspace (thus guaranteeing ordering, but without
> > > early notification of errors), some will require the frame
> > > automatically, and at least one (uvcvideo) has a module parameter to
> > > pick the desired behaviour.
> > 
> > Also, from a userspace point of view, the synchronization with the
> > "next frame" in V4L2 isn't implicit. We can poll() the device, just
> > like we'd do with a fence FD. What the explicit fence gives, is a
> > unified object we can pass to another driver, or other userspace, so we
> > can delegate the wait.
> > 
> > You refer to performance in few places. In streaming, this is often
> > measure as real-time throughput. Implicit/explicit fences don't really
> > play any role for us in this regard. V4L2 drivers, like m2m drivers,
> > works with buffer queues. So you can queue in advance many buffers on
> > the OUTPUT device side (which is the input of the m2m), and userspace
> > will queue in advance pretty much all free buffers available on the
> > CAPTURE side. The driver is never starved in that model, at the cost of
> > very large memory consumption of course. Maybe a more visual
> > representation would be:
> > 
> >   [pending job] -> [M2M Worker] -> [pending results]
> > 
> > So as long as userspace keep the pending job queue non-empty, and that
> > it consumes and give back buffers back to write the results into, the
> > driver will keep running un-interrupted. Performance remains optimal.
> > What isn't optimal is the latency. And what bugs right now is when a
> > DMAbuf implicit out fence is put back into the pending results queue,
> > since the fence is ignored.
> 
> Yes, that makes sense.  In 3D land, we're very concerned about
> latency.  Any time anyone has to stall for anything, it's a potential
> hitch in someone's game.  Being delayed by a single extra frame can be
> problematic; 2-3 frames puts the gamer at a significant disadvantage.
> In video, as long as audio and video are in sync and you aren't
> dropping frames, no one really cares about latency as long as hitting
> the pause button doesn't take too long.

Just a note, there exist low latency use cases for streaming too (sub-
frame latency between two devices). But everything I'm ware is
downstream. The one I have in mind uses a special AXI feature to
synchronize between two HW component, but the implementation is not
using either implicit or explicit fence, in fact they didn't bother
adding a specific kernel object, you have to know when you use these
downstream drivers. We are a bit far from being able to make generic
software on top of that.

The use case was less prone to capture error, since instead of a
camera, they have SDI or HDMI receiver.

> 
> What concerns me the most, I think is actually the interop issues.
> You mentioned issues with the raspberry pi.  Right now, if someone is
> rendering frames using a Vulkan driver and trying to pass those on to
> V4L for encode or to some other api such as VA-API, we don't really
> have a plan for synchronization.  Thanks to dma-buf extensions we at
> least have most of a plan for sharing the memory and negotiating image
> layouts (strides, tiling, etc.) but no plan for synchronization at

I didn't know there was plan for that, this is nice. Right now every
userspace carry this information in a slightly different and
incompatible way, translating, extrapolation, etc. It's all very error
prone.

> all.  The only thing you can do today is to use a VkFence to CPU wait
> for the 3D rendering to be 100% done and then pass the image on to the
> encoder.
> 
> The more I look over the various hacks we've done over the course of
> the last 4 years to make window systems work, the less confident I am
> that I want to expose ANY of them as an official Vulkan extension that
> we support long-term.  The one we do have which I'm reasonably happy
> to be stuck with is sync_file import/export.  That said, it's sounding
> like V4L doesn't support dma-buf implicit sync at all so maybe CPU
> waiting with a VkFence is the current state-of-the-art?
> 
> --Jason
> 
> 
> > > > Trying to understand. :-)
> > > 
> > > So am I :-)
> > 
> > Hehe, same here.
> > 
> > > > > > There is other elements that would implement fencing, notably kmssink, but no
> > > > > > one actually dared porting it to atomic KMS, so clearly there is very little
> > > > > > comunity interest. glimagsink could clearly benifit. Right now if we import a
> > > > > > DMABuf, and that this DMAbuf is used for render, a implicit fence is attached,
> > > > > > which we are unaware. Philippe Zabbel is working on a patch, so V4L2 QBUF would
> > > > > > wait, but waiting in QBUF is not allowed if O_NONBLOCK was set (which GStreamer
> > > > > > uses), so then the operation will just fail where it worked before (breaking
> > > > > > userspace). If it was an explcit fence, we could handle that in GStreamer
> > > > > > cleanly as we do for new APIs.
> > > > > > 
> > > > > > > > ## Chicken and egg problems
> > > > > > > > 
> > > > > > > > Ok, this is where it starts getting depressing.  I made the claim
> > > > > > > > above that Wayland has an explicit synchronization protocol that's of
> > > > > > > > questionable usefulness.  I would claim that basically any bit of
> > > > > > > > plumbing we do through window systems is currently of questionable
> > > > > > > > usefulness.  Why?
> > > > > > > > 
> > > > > > > > From my perspective, as a Vulkan driver developer, I have to deal with
> > > > > > > > the fact that Vulkan is an explicit sync API but Wayland and X11
> > > > > > > > aren't.  Unfortunately, the Wayland extension solves zero problems for
> > > > > > > > me because I can't really use it unless it's implemented in all of the
> > > > > > > > compositors.  Until every Wayland compositor I care about my users
> > > > > > > > being able to use (which is basically all of them) supports the
> > > > > > > > extension, I have to continue carry around my pile of hacks to keep
> > > > > > > > implicit sync and Vulkan working nicely together.
> > > > > > > > 
> > > > > > > > From the perspective of a Wayland compositor (I used to play in this
> > > > > > > > space), they'd love to implement the new explicit sync extension but
> > > > > > > > can't.  Sure, they could wire up the extension, but the moment they go
> > > > > > > > to flip a client buffer to the screen directly, they discover that KMS
> > > > > > > > doesn't support any explicit sync APIs.
> > > > > > > 
> > > > > > > As per the above correction, Wayland compositors aren't nearly as bad
> > > > > > > off as I initially thought.  There may still be weird screen capture
> > > > > > > cases but the normal cases of compositing and displaying via
> > > > > > > KMS/atomic should be in reasonably good shape.
> > > > > > > 
> > > > > > > > So, yes, they can technically
> > > > > > > > implement the extension assuming the EGL stack they're running on has
> > > > > > > > the sync_file extensions but any client buffers which come in using
> > > > > > > > the explicit sync Wayland extension have to be composited and can't be
> > > > > > > > scanned out directly.  As a 3D driver developer, I absolutely don't
> > > > > > > > want compositors doing that because my users will complain about
> > > > > > > > performance issues due to the extra blit.
> > > > > > > > 
> > > > > > > > Ok, so let's say we get KMS wired up with implicit sync.  That solves
> > > > > > > > all our problems, right?  It does, right up until someone decides that
> > > > > > > > they wan to screen capture their Wayland session via some hardware
> > > > > > > > media encoder that doesn't support explicit sync.  Now we have to
> > > > > > > > plumb it all the way through the media stack, gstreamer, etc.  Great,
> > > > > > > > so let's do that!  Oh, but gstreamer won't want to plumb it through
> > > > > > > > until they're guaranteed that they can use explicit sync when
> > > > > > > > displaying on X11 or Wayland.  Are you seeing the problem?
> > > > > > > > 
> > > > > > > > To make matters worse, since most things are doing implicit
> > > > > > > > synchronization today, it's really easy to get your explicit
> > > > > > > > synchronization wrong and never notice.  If you forget to pass a
> > > > > > > > sync_file into one place (say you never notice KMS doesn't support
> > > > > > > > them), it will probably work anyway thanks to all the implicit sync
> > > > > > > > that's going on elsewhere.
> > > > > > > > 
> > > > > > > > So, clearly, we all need to go write piles of code that we can't
> > > > > > > > actually properly test until everyone else has written their piece and
> > > > > > > > then we use explicit sync if and only if all components support it.
> > > > > > > > Really?  We're going to do multiple years of development and then just
> > > > > > > > hope it works when we finally flip the switch?  That doesn't sound
> > > > > > > > like a good plan to me.
> > > > > > > > 
> > > > > > > > 
> > > > > > > > ## A proposal: Implicit and explicit sync together
> > > > > > > > 
> > > > > > > > How to solve all these chicken-and-egg problems is something I've been
> > > > > > > > giving quite a bit of thought (and talking with many others about) in
> > > > > > > > the last couple of years.  One motivation for this is that we have to
> > > > > > > > deal with a mismatch in Vulkan.  Another motivation is that I'm
> > > > > > > > becoming increasingly unhappy with the way that synchronization,
> > > > > > > > memory residency, and command submission are inherently intertwined in
> > > > > > > > i915 and would like to break things apart.  Towards that end, I have
> > > > > > > > an actual proposal.
> > > > > > > > 
> > > > > > > > A couple weeks ago, I sent a series of patches to the dri-devel
> > > > > > > > mailing list which adds a pair of new ioctls to dma-buf which allow
> > > > > > > > userspace to manually import or export a sync_file from a dma-buf.
> > > > > > > > The idea is that something like a Wayland compositor can switch to
> > > > > > > > 100% explicit sync internally once the ioctl is available.  If it gets
> > > > > > > > buffers in from a client that doesn't use the explicit sync extension,
> > > > > > > > it can pull a sync_file from the dma-buf and use that exactly as it
> > > > > > > > would a sync_file passed via the explicit sync extension.  When it
> > > > > > > > goes to scan out a user buffer and discovers that KMS doesn't accept
> > > > > > > > sync_files (or if it tries to use that pesky media encoder no one has
> > > > > > > > converted), it can take it's sync_file for display and stuff it into
> > > > > > > > the dma-buf before handing it to KMS.
> > > > > > > > 
> > > > > > > > Along with the kernel patches, I've also implemented support for this
> > > > > > > > in the Vulkan WSI code used by ANV and RADV.  With those patches, the
> > > > > > > > only requirement on the Vulkan drivers is that you be able to export
> > > > > > > > any VkSemaphore as a sync_file and temporarily import a sync_file into
> > > > > > > > any VkFence or VkSemaphore.  As long as that works, the core Vulkan
> > > > > > > > driver only ever sees explicit synchronization via sync_file.  The WSI
> > > > > > > > code uses these new ioctls to translate the implicit sync of X11 and
> > > > > > > > Wayland to the explicit sync the Vulkan driver wants.
> > > > > > > > 
> > > > > > > > I'm hoping (and here's where I want a sanity check) that a simple API
> > > > > > > > like this will allow us to finally start moving the Linux ecosystem
> > > > > > > > over to explicit synchronization one piece at a time in a way that's
> > > > > > > > actually correct.  (No Wayland explicit sync with compositors hoping
> > > > > > > > KMS magically works even though it doesn't have a sync_file API.)
> > > > > > > > Once some pieces in the ecosystem start moving, there will be
> > > > > > > > motivation to start moving others and maybe we can actually build the
> > > > > > > > momentum to get most everything converted.
> > > > > > > > 
> > > > > > > > For reference, you can find the kernel RFC patches and mesa MR here:
> > > > > > > > 
> > > > > > > > https://lists.freedesktop.org/archives/dri-devel/2020-March/258833.html
> > > > > > > > 
> > > > > > > > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4037
> > > > > > > > 
> > > > > > > > At this point, I welcome your thoughts, comments, objections, and
> > > > > > > > maybe even help/review. :-)