Full-motion zero-copy screen capture in Weston

[Pekka Paalanen]

On Mon, 10 Jun 2024 14:18:39 +0000
"Hoosier, Matt" <Matt.Hoosier at garmin.com> wrote:

> Yes, the native Linux driver for the hardware still does end up being
> responsible for one or more planes.
> 
> Other than trying to arrange the pieces so that there's some API that
> offers the client an option to guarantee that the source of the
> screen capture involves the writeback connector (similar to what
> you've done with weston_output_capture), I don't really think it
> would be a good idea to make Weston explicitly aware of any of this
> funny hypervisor business going on.
> 

I very much agree.

> The title on this email conversation was probably poorly chosen, in
> retrospect. I'm not so much concerned with keeping absolutely to a
> zero-copy mechanism as I am to using hardware mechanisms (GL
> rendering, DRI writeback, hardware-accelerated blits as necessary)
> all the way through.
> 
> After seeing the way that the Pipewire backend handles streaming
> (especially with
> https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1366),
> I'm almost thinking that it would be preferable to maybe structure
> the design of this feature something like this:
> 
> * Add some sort of API on weston_output by which it can advertise the
> ability to lay hands on the explicit renderbuffer (e.g.
> gbm_surface_get_front_buffer()). This roughly equates to whether it's
> a primary, non-nested backend. That is, DRM.
> 
> * When processing the weston.ini mirror-of relationships at startup,
> check whether the source output of the mirror-of declaration supports
> that ability above. If so:
>   - Wire up a signal so that the source output announces each newly
> rendered frame to any/all mirror-of outputs.
>   - The virtual output's backend allocates its own buffer pool in
> exactly the same way that MR 1366 already does.
>   - Upon receipt of each signal announcing a new frame's renderbuffer
> from the source output, the virtual output does a very cheap
> glBlitFramebuffer() to get the contents into its own buffer pool.
> This avoids the possibility of an unresponsive client tying up the
> source output's buffer.
> 
> * If the source output isn't one that supports exposing the
> underlying renderbuffer, then the mirror-of relationship continues as
> with MR 1476 just to invoke an explicit weston_renderer pass to draw
> the correct logical contents.
> 
> How does this strike you?

Sorry, I don't understand how that idea is relevant to the KMS
writeback case. Did you imply that DRM-backend could deliver a
KMS-writeback FB instead of the rendered FB?

This is not the current plan for mirror-of. It does not allow the
mirror output to run on its own pace independently of the source
output. Re-using the source output's rendered FB would also be a
problem for color management. The FB is rendered for that output, and
the color properties of the mirror may be different, needing an
independent rendering anyway.

The fundamental difference is who defines the properties of the stream.
KMS writeback steam properties are necessarily defined by the KMS
output. Mirror-of is for the opposite, for full flexibility. E.g. a
remote mirror may not want to run at the full framerate, the physical
monitor resolution, or with the color capabilities of the source output
in order to save bandwidth and improve latency.

It seems to me that we will need two different mechanisms for the two
cases. I believe KMS writeback streaming is worthwhile to support, but
not via mirror-of key. Maybe the writeback streaming should be built
into the DRM-backend as a special pipewire source? Then it would also
be guaranteed to be KMS writeback. It is some amount of code and
testing duplication, but I think it would be the simplest.

I don't see KMS writeback streaming as specific to your curious
virtualization case. I can imagine it being useful in general, to
ensure that display controller output is correct for example, or when
overall rendering efficiency is more important than optimal stream
format.


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20240612/594bf669/attachment-0001.sig>