Full-motion zero-copy screen capture in Weston

[Hoosier, Matt]

Thanks, Pekka.

So what would be an upstream-friendly way to put hooks deep down into the DRM backend to exploit the KMS writeback connectors, but still allow an XDG Desktop portal implementation to be usable on all the backends? Or maybe that's an unnecessary goal to begin with; a desktop session somewhat implies the compositor is driving the physical hardware?

> -----Original Message-----
> From: Pekka Paalanen <pekka.paalanen at collabora.com>
> Sent: Wednesday, May 29, 2024 2:43 AM
> To: Marius Vlad <marius.vlad at collabora.com>; Hoosier, Matt
> <Matt.Hoosier at garmin.com>
> Cc: wayland-devel at lists.freedesktop.org
> Subject: Re: Full-motion zero-copy screen capture in Weston
> 
> On Tue, 28 May 2024 20:04:45 +0300
> Marius Vlad <marius.vlad at collabora.com> wrote:
> 
> > On Tue, May 28, 2024 at 03:53:23PM +0000, Hoosier, Matt wrote:
> > > Hi Marius,
> > Hi,
> > >
> > > Okay, I guess that answers the bit about needing to screen-scrape to get
> content into Pipewire now.
> > >
> > > But I'm still a little unclear about a couple things, if I were to try to build on
> this PW backend as a starting point:
> > >
> > > First, it looks to me like when you use the PW backend to Weston,
> > > that becomes your display. That is, rendering directly targets it. I
> > > was hoping for a way to get it to broadcast the very same
> > > framebuffer(s) that are getting scanned out for the current frame by
> > > the DRM backend.
> > With
> > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341
> > the PipeWire output can mirror out the native DRM one. It aims at
> > having a way to configure VNC, RDP and PipeWire to screen-share the
> output.
> > >
> > > Second, I'm don't see the path to getting this to leverage the
> > > DRM_MODE_CONNECTOR_WRITEBACK hardware (like the
> > > weston_screen_recorder does).  I think that any layering would be
> > > forced to be offloaded to the GPU ahead of time. Maybe I missed some
> > > implication of what you were pointing out here?
> > No sorry, I haven't really implied that, just pointed that out there's
> > some work for PipeWire gaining dmabuf.
> >
> > Screen-sharing an output is done
> > with multiple backends, and configuring Weston front-end is it with
> > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341.
> 
> Hi Matt, Marius,
> 
> multiple backends necessarily means that each backend renders its own
> weston_output independently. While it's fine for most screen-sharing use
> cases, I don't think it can ever be as efficient as KMS writeback could be.
> 
> Getting a dmabuf-based video stream from DRM-backend using KMS
> writeback needs integration in the DRM-backend. I don't know how that
> would look like, but I suspect these ideas would be good:
> 
> - builds on and extends the existing writeback support instead of
>   something new on the side
> 
> - does not use the DRM-backend "virtual output" API; this was a
>   workaround for the lack of multi-backend support in the past and has
>   the same disadvantages as multi-backend. (I don't think the DRM
>   virtual output API has any reason to exist anymore, multi-backend
>   should supersede it after converting the users.)
> 
> I also think that implementing the usual desktop portals in Weston would be a
> good thing, and not something to avoid, if those can provide a nice external
> interface for the use case.
> 
> 
> Thanks,
> pq