Full-motion zero-copy screen capture in Weston

[Hoosier, Matt]

Hi,

Yeah. I agree that although I can prototype something quick and dirty here as a change to weston_output_capture_v1, it's probably not a good fit there.

Drew or Simon, does either of https://github.com/swaywm/wlroots/blob/master/protocol/wlr-screencopy-unstable-v1.xml or https://github.com/swaywm/wlroots/blob/master/protocol/wlr-export-dmabuf-unstable-v1.xml strike you as a good fit for the use-case of getting a streamed copy of an output? They seem very similar – not sure what differentiates them from each other.

My goal is to implement this screen capture with a guarantee that the copy comes from a KMS writeback connector. I know this sounds like an odd thing to insist on. Let's say that in my industry, the system is rarely only running Linux directly on the bare metal. Using the writeback hardware on the display controller allows to get a copy of content from all the virtual machines in the system.

Frankly, weston_output_capture_v1's model that clients allocate the buffers would make it very difficult to support efficient screen capture for more than one simultaneous client. You can only target one writeback framebuffer per page flip. Having the compositor manage the buffers' lifetimes and just publishing out handles (in the style of those two wlr extensions) to those probably fits better.

-Matt

________________________________
From: Pekka Paalanen
Sent: Friday, May 31, 2024 3:02 AM
To: Hoosier, Matt
Cc: Marius Vlad; wayland-devel at lists.freedesktop.org
Subject: Re: Full-motion zero-copy screen capture in Weston

On Thu, 30 May 2024 13:40:15 +0000
"Hoosier, Matt" <Matt.Hoosier at garmin.com> wrote:

> Okay, interesting thoughts on all of that.
>
> I'm not sure how far I'm going to get toward a complete overhaul of
> the capture mechanism. Maybe it would be useful to know a couple
> things about the current one (weston_output_capture_v1), so that I
> don't commit mistakes that somebody already considered and guarded
> against:
>
> * Why was it explicitly picked only for still shots? I can image that
> it wouldn't have been a whole lot more work to design this API with a
> bounce-buffering scheme rather than a setup/do-it-once/destroy model.
>

There was no need for streaming, as it was purely for the test suite.

The test suite is very particular about when and how the shot is taken:
it must be produced by the defined source, and it must be produced on
the very next repaint of the output, exactly once.

Did you mean a buffer pool (queueing/dequeueing) rather than
bounce-buffering? Sure.

> * Why was wl_shm explicitly picked for the buffer type? I was
> thinking here that it would have worked equally well to specify that
> the client must supply a linear-layout buffer manufactured through
> zwp_linux_dmabuf. This would be eligible for direct targeting by the
> GL renderer/KMS writeback, but also can be mapped with gbm_bo_map()
> in the compositor's Pixman backend and/or a client who wants to map
> it to the CPU upon delivery.

Because the test suite specifically needs CPU access to the screenshot.
There was no use yet for dmabuf, and GL-renderer was already doing
glReadPixels() for screenshots.

IOW, all the limitations are just "was not needed yet".

Note, that re-using or extending the protocol extension
weston_capture_v1 for streaming outputs for non-test-suite use cases
may not be the best idea. If the interface needs to be a Wayland
extension, then maybe something from the wlr extensions would suit
better. OTOH, for e.g. Pipewire there would not be a Wayland extension
used at all.

The internal API (weston_output_capture) is very much geared for
weston_capture_v1 only. The internal API might take improving rather
than weston_capture_v1.


Thanks,
pq
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20240531/cdf7c96a/attachment.htm>