Full-motion zero-copy screen capture in Weston

[Pekka Paalanen]

On Fri, 7 Jun 2024 14:16:32 +0000
"Hoosier, Matt" <Matt.Hoosier at garmin.com> wrote:

> > What do you mean you can capture all virtual machines with KMS
> > writeback?
> >
> > What's the architecture there? How do virtual machines access KMS
> > hardware? Why would Weston be able to capture their outputs at all?
> 
> The world of virtualization on commercially supported embedded SOCs
> for big-scale production use is wild. Every vendor typically has only
> a narrow range of supported hypervisors. Usually, one -- and an
> in-house model at that. There will typically be at least one bizarre
> twist on the virtualization method for display controllers.
> 
> One fad is for one of the virtual machines -- typically, the one
> running a normal-style GNU/Linux Yocto system -- to own privileged
> I/O maps of most of the hardware, and it more or less runs the same
> drivers inside this VM that the SOC maker has already written for
> their bare-metal Linux deployment. Most hardware peripherals are just
> exposed to the other guest VMs by having the privileged Linux VM host
> export some sort of hypervisor-integrated VirtIO-style backend for
> the hardware. The guests see VirtIO devices. This approach goes by
> the name of "paravirtualization". 
> 
> But for graphics and display, there is almost always some additional
> mechanism to sidestep this pure paravirtualization because it's
> perceived as too expensive. So the vendor may do something like
> designate some subset of planes on each connector to be "directly"
> manipulated by the non-GNU/Linux guest VMs. The hypervisor executive
> runs a tiny little server that receives the stream of plane updates,
> and during the vsync it programs the appropriate display controller
> hardware registers to refer to the new frame's contents. It's very
> limited -- the guests VMs whose scene updates are couched using this
> mechanism are not able to do modesets or reconfigure the overall DRI
> scene topology.
> 
> The key point here is that because Linux is running a full physical
> driver, the writeback connector gets the results of blending all the
> layers -- even those whose contents are programmed using the awkward
> side channel.
> 
> I'm not a big fan on this approach. But it is there, and I'd like to
> cope with it. I have a use-case that requires Linux to get a complete
> picture of the physical contents getting scanned out by the connector.
> 

I see. On such connectors, does Weston still paint at least one KMS
plane?

If we can pretend that the connector is still a normal output for
Weston, just with less planes, I feel that this should be implementable
in upstream Weston.


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/wayland-devel/attachments/20240610/0ff25b46/attachment.sig>