Full-motion zero-copy screen capture in Weston

[Simon Ser]

On Friday, May 31st, 2024 at 15:26, Hoosier, Matt <Matt.Hoosier at garmin.com> wrote:

> Drew or Simon, does either of https://github.com/swaywm/wlroots/blob/master/protocol/wlr-screencopy-unstable-v1.xml or https://github.com/swaywm/wlroots/blob/master/protocol/wlr-export-dmabuf-unstable-v1.xml strike you as a good fit for the use-case of getting a streamed copy of an output? They seem very similar – not sure what differentiates them from each other.

screencopy is designed around client-allocated buffers. For
compositor-allocated buffers, export-dmabuf is what's used in the
wlroots ecosystem.

> My goal is to implement this screen capture with a guarantee that the copy comes from a KMS writeback connector. I know this sounds like an odd thing to insist on. Let's say that in my industry, the system is rarely only running Linux directly on the bare metal. Using the writeback hardware on the display controller allows to get a copy of content from all the virtual machines in the system.
> Frankly, weston_output_capture_v1's model that clients allocate the buffers would make it very difficult to support efficient screen capture for more than one simultaneous client. You can only target one writeback framebuffer per page flip. Having the compositor manage the buffers' lifetimes and just publishing out handles (in the style of those two wlr extensions) to those probably fits better.

IIRC Weston is not interested in supporting the wlroots protocol, they'd
prefer to use e.g. PipeWire. Should be possible to pass the result of
the writeback into a PipeWire stream.

> From: Pekka Paalanen
> Sent: Friday, May 31, 2024 3:02 AM
> To: Hoosier, Matt
> Cc: Marius Vlad; wayland-devel at lists.freedesktop.org
> Subject: Re: Full-motion zero-copy screen capture in Weston
> On Thu, 30 May 2024 13:40:15 +0000
> "Hoosier, Matt" <Matt.Hoosier at garmin.com> wrote:
> 
> > Okay, interesting thoughts on all of that.
> >
> > I'm not sure how far I'm going to get toward a complete overhaul of
> > the capture mechanism. Maybe it would be useful to know a couple
> > things about the current one (weston_output_capture_v1), so that I
> > don't commit mistakes that somebody already considered and guarded
> > against:
> >
> > * Why was it explicitly picked only for still shots? I can image that
> > it wouldn't have been a whole lot more work to design this API with a
> > bounce-buffering scheme rather than a setup/do-it-once/destroy model.
> >
> 
> There was no need for streaming, as it was purely for the test suite.
> 
> The test suite is very particular about when and how the shot is taken:
> it must be produced by the defined source, and it must be produced on
> the very next repaint of the output, exactly once.
> 
> Did you mean a buffer pool (queueing/dequeueing) rather than
> bounce-buffering? Sure.
> 
> > * Why was wl_shm explicitly picked for the buffer type? I was
> > thinking here that it would have worked equally well to specify that
> > the client must supply a linear-layout buffer manufactured through
> > zwp_linux_dmabuf. This would be eligible for direct targeting by the
> > GL renderer/KMS writeback, but also can be mapped with gbm_bo_map()
> > in the compositor's Pixman backend and/or a client who wants to map
> > it to the CPU upon delivery.
> 
> Because the test suite specifically needs CPU access to the screenshot.
> There was no use yet for dmabuf, and GL-renderer was already doing
> glReadPixels() for screenshots.
> 
> IOW, all the limitations are just "was not needed yet".
> 
> Note, that re-using or extending the protocol extension
> weston_capture_v1 for streaming outputs for non-test-suite use cases
> may not be the best idea. If the interface needs to be a Wayland
> extension, then maybe something from the wlr extensions would suit
> better. OTOH, for e.g. Pipewire there would not be a Wayland extension
> used at all.
> 
> The internal API (weston_output_capture) is very much geared for
> weston_capture_v1 only. The internal API might take improving rather
> than weston_capture_v1.
> 
> 
> Thanks,
> pq