Screen shooting and recording protocols (Re: Authorized clients)

[Maarten Baert]

On 10/01/14 17:10, Pekka Paalanen wrote:
> What you call capturing a window, is just capturing a sub-rect of the
> screen for me. For me, capturing a window is a different thing, it
> gets the original window contents. 
Okay, window capturing is probably not the right word to describe it.
It's essentially bounding box picking followed by recording a part of
the screen.

> It was visible when the user started the capture, then switched
> virtual desktops to do something else while the window was being
> recorded, e.g. some automatic long GUI test run. Again, that goes to
> my interpretation of what window capture does. 
I suspect automatic GUI testing is much easier to integrate in the
toolkit being used (GTK or Qt) or even the application itself. The
compositor can send events, but it knows nothing about buttons and
textboxes, and you would need some pretty complex OCR to validate the
output in the first place. Do people actually want to do this?

> It could be optional. But sure, it will be very complicated, and that
> is what "window capturing" means to me in the broadest sense. Maybe we
> should forget window capturing.
I agree, at least for now. We can always add it later if it is really
needed, it is totally different from normal screen capturing so it would
need lots of extra code anyway.

> Yeah... but supporting resizing is a must for window recording, since
> you can resize the window. One more nail to the coffin of window
> recording. 
I have the same problem with OpenGL recording. It's not as bad because
most games use just a single window for everything (no tooltips and all
that). But it definitely complicates memory management (you can't use
simple preallocated buffers when the size is unknown), and the recording
application is forced to add scaling or some other workaround to deal
with changing sizes.

> That probably requires a video mode switch, which is totally possible.
> Video mode is in complete control of the compositor.
>
> There has been talk about 10-bit output, so I guess it would be useful
> to someone. And support for it is coming if not there already AFAIK.
>
> IOW, you would be willing to downgrade the image quality on the monitor
> while recording is on. That is perfectly doable, if it is wanted.
Users already accept that video encoding takes a lot of CPU power and
will usually degrade the frame rate of their games somewhat. The quality
of the final video is usually more important than the quality of the
image on the screen. And let's be realistic, almost every program uses
8-bit colors, so a 10-bit display isn't going to look any better than an
8-bit one unless you're running some very special application that can
make use of 10-bit colors.

The alternative is that the compositor does an additional pixel format
conversion, and that would add a lot of overhead (essentially making
zero-overhead capturing impossible).

I read that Wayland compositors are required to accept BGRA (and BGRX)
buffers from applications. Does that mean the compositor is also
required to support a BGRA video mode? Or is that unrelated?


On 10/01/14 18:29, Sebastian Wick wrote:
> Recording a window instead of the whole screen is something I really,
> really want to do.
I want it too, but I think it is too complex to do in the compositor.
Besides, for the simple cases it can already be done in the client
itself, this is how I do OpenGL recording today (with or without
Wayland). It's trivial to extend the existing OpenGL code to Wayland SHM
buffers as long as the application uses only a single buffer (i.e. no
subwindows, menus or tooltips). Even some of the more complex cases can
be handled if the client can figure out what the relative position of
the windows is. And I assume it will, how else can the client make
tooltips and menus appear in the right place?

As I said, we can always come back to this later and add window
capturing. But I think it makes more sense to implement normal screen
capturing first.