[PATCH RFC wayland-protocols] unstable/linux-dmabuf: add wp_linux_dmabuf_device_hint

Mon Nov 12 12:48:19 UTC 2018

On Mon, 12 Nov 2018 12:16:04 +0000
Simon Ser <contact at emersion.fr> wrote:

> On Monday, November 12, 2018 10:18 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> > On Sat, 10 Nov 2018 13:34:31 +0000
> > Simon Ser <contact at emersion.fr> wrote:
> >  
> > > On Monday, November 5, 2018 9:57 AM, Pekka Paalanen <ppaalanen at gmail.com> wrote:  
> > > > > > Yeah. Another option is to send a wl_array of modifiers per format and
> > > > > > tranch.  
> > > > >
> > > > > True. Any reason why this hasn't been done in the global?  
> > > >
> > > > For formats? Well, it is simpler without a wl_array, and there might be
> > > > a lot of formats.
> > > >
> > > > Could there be a lot of modifiers per format? Would a wl_array make
> > > > anything easier? Just a thought.  
> > >
> > > It's true that for this list of formats sorted by preference, we'll probably
> > > need to split modifiers anyway so I don't think we'd benefit a lot from this
> > > approach.  
> >
> > Hi Simon,
> >
> > just to be clear, I was thinking of something like:
> >
> > event(uint format, wl_array(modifiers))
> >
> > But I definitely do not insist on it if you don't see any obvious
> > benefits with it.  
> 
> Yeah. I think the benefits would not be substantial as we need to "split" these
> to order them by preference. So it would look like so:
> 
>   event(format1, wl_array(modifiers))
>   barrier()
>   event(format1, wl_array(modifiers))
>   event(format2, wl_array(modifiers))
>   barrier()
>   event(format1, wl_array(modifiers))
>   barrier()
> 
> Also this is not consistent with the rest of the protocol. Maybe we can discuss
> this again for linux-dmabuf-unstable-v2.
> 
> > It seems you and I made very different assumptions on how the hints
> > would be sent, I only realized it just now. More about that below.
> >  
> > > > > > I suppose it will be enough to send tranches for just the currently
> > > > > > used format? Otherwise it could be "a lot" of data.  
> > > > >
> > > > > What do you mean by "the currently used format"?  
> > > >
> > > > This interface is used to send clients hints after they are already
> > > > presenting, which means they already have a format chosen and probably
> > > > want to stick with it, just changing the modifiers to be more optimal.  
> > >
> > > If we only send the modifiers for the current format, how do clients tell the
> > > difference between the initial hints (which don't have a "currently used
> > > format") and the subsequent hints?  
> >
> > I'm not sure I understand why they would need to see the difference.
> > But yes, I was short-sighted here and didn't consider the
> > initialization when a surface is not mapped yet. I didn't expect that
> > hints can be calculated if the surface is not mapped, but of course a
> > compositor can provide some defaults. I suppose the initial default
> > hints would boil down to what is most efficient to composite.
> >  
> > > > > I expect clients to bind to this interface and create a surface hints object
> > > > > before the surface is mapped. In this case there's no "currently used format".  
> > > >
> > > > Right, that's another use case.
> > > >  
> > > > > It will be a fair amount of data, yes. However it's just a list of integers.
> > > > > When we send strings over the protocol (e.g. toplevel title in xdg-shell) it's
> > > > > about the same amount of data I guess.  
> > > >
> > > > If the EGLConfig or GLXFBConfig or GLX visual lists are of any
> > > > indication... yes, they account for depth, stencil, aux, etc. but then
> > > > we will have modifiers.
> > > >
> > > > We already advertise the list of everything supported of format+modifer
> > > > in the linux_dmabuf extension. Could we somehow minimize the number of
> > > > recommended format+modifiers in hints? Or maybe that's not a concern
> > > > for the protocol spec?  
> > >
> > > I'm not sure.
> > >
> > > After this patch, I'm not even sure how the formats+modifiers advertised by the
> > > global work. Are these formats+modifiers supported on the GPU the compositor
> > > uses for rendering? Intersection or union of formats+modifiers supported on all
> > > GPUs?  
> >
> > The format+modifier advertised by the global before this patch are the
> > ones that can work at all, or the compositor is willing to make them
> > work at least in the worst fallback case. This patch must not change
> > that meaning. These formats also must always work regardless of which
> > GPU a client decides to use, but that is already implied by the
> > compositor being able to import a dmabuf. The compositor does not need
> > to try to factor in what other GPUs on the system might be able to
> > render or not, that is for the client to figure out when it knows the
> > formats the compositor can accept and is choosing a GPU to render with.
> > It is theoretically possible that a client tries to use a GPU that
> > cannot render any formats the compositor can use, but that is the
> > client's responsibility to figure out.  
> 
> Okay, that makes sense. And if a GPU doesn't support direct scan-out for some
> format+modifier, then it can always fallback to good ol' compositing.
> 
> > So clearly the formats from the global can be used by a client at any
> > time. The hint formats OTOH has no reason to list absolutely
> > everything the compositor supports, but a compositor can choose on its
> > own judgement to send only a sub-set it would prefer.  
> 
> Yes, this makes sense.
> 
> > However, after a client has picked a format and used it, then there
> > should be hints with that format, at least if they can make any
> > difference.  
> 
> Okay, I get it now. :)
> 
> > I'm not sure. Not listing everything always was my intuitive
> > assumption, and I believe you perhaps assumed the opposite so that a
> > client has absolutely all the information to e.g. optimize the modifier
> > of a format that the compositor would not prefer at all even though it
> > does work.
> >
> > It would be simpler to always send everything, but that will be much
> > more protocol traffic. Would it be too much? I don't know, could you
> > calculate some examples of how many bytes a typical hints update would
> > be if sending everything always?  
> 
> If I'm understanding the protocol marshalling right, each event would have:
> 
> * A 2*32-bit header for sender, size and opcode
> * One 32-bit field for the format
> * A 2*32-bit field for the modifier
> 
> Total 5*32 = 160 bits, ie. 20 bytes per event.
> 
> My current setup lists ~100 format+modifier combinations. So that means each
> time a client binds to wp_linux_dmabuf, ~2KB of data is sent. That's a lot, your
> concerns are correct.
> 
> (In a future version of the protocol, maybe we could use shared memory, just
> like the wl_keyboard keymap.)
> 
> In the meantime, we could decide to do as you suggest. So the compositor would
> always advertise a subset of the supported modifiers. When the hints object is
> created, the compositor would send its preferred format+modifier pairs. When the
> client submits a buffer with a new format, the compositor can decide to send the
> preferred modifiers for this format. I wonder how we should phrase this in the
> protocol (can/should/must?). Thoughts?

Yeah. It can be left to be decided by the compositor implementation on
which format+modifiers it suggests, while clients are free to pick any
at all supported format+modifier they want.

Quite likely we need to revisit this in any case. Using shared memory
feels complicated, but OTOH it *is* relatively lot of data. Even the
kernel UABI does not use a flat list of format+modifier but a fairly
"interesting" bitfield encoding. That's probably not appropriate for
Wayland though, so maybe we have to use shared memory for it.

I wonder if there could be a yet another option...

> In the case where a client is scanned out and the GPU used for scan-out
> supports more formats than the GPU used for compositing, as discussed before the
> compositor won't be able to advertise these additional formats, because the
> client could keep using them when not scanned out anymore.

Yes. It is actually unavoidable, because the compositor makes the
decision to stop scanning out first while compositing already, then
sends the new set of hints, and only afterwards the client could use the
new hints.

Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20181112/07c55e5d/attachment-0001.sig>