Screen shooting and recording protocols (Re: Authorized clients)

[Pekka Paalanen]

On Thu, 09 Jan 2014 15:15:48 +0100
Martin Peres <martin.peres at free.fr> wrote:

> Le 09/01/2014 14:41, Pekka Paalanen a écrit :
> > On Thu, 09 Jan 2014 13:05:28 +0100
> > Martin Peres <martin.peres at free.fr> wrote:
> >
> >> There is a way to limit the memory consumption of apps that don't
> >> consume buffers. We could have a small ring buffer of wl_buffer or
> >> dma-buf (if we want 0 copy) on the compositor side. When the
> >> ring buffer is full, the compositor should not copy or store any
> >> new buffer. This way, it won't use up all the video memory.
> > Yeah, that would be part of the failsafe, except the implementation
> > details and protocol get complicated as you probably do not want the
> > compositor to depend on the client in allocating the buffers the
> > compositor is rendering into, which means you cannot use wl_drm.
> >
> > Or maybe you could... compositor has its own 2-3 private buffers it
> > normally uses, and the recorder client could give it some "empty"
> > buffers. The compositor renders into client's buffers one by one if
> > they are suitable and not busy, *and* flips them on screen. As the
> > fallback it can use its own buffers. That way the client is the one
> > that gets "killed on OOM". Maybe that'd work.
> Oh, very interesting idea. I wouldn't have thought of that.

Mmh, the architecture would certainly be interesting, I'm not sure all
the pieces are there yet for this kind of use, API-wise.

> >> Then, we need a way for the compositor to know when the user
> >> consumes a buffer. Having a simple "get_next_frame()" function
> >> returning the next wl_buffer or the next dma-buf fd would be
> >> enough. We could also tell that the semantics of the call means
> >> that the application should be done using a frame before
> >> requiring a new one. If we really want to be conservative, we could
> >> only use 2 entries in the ring buffer.
> >>
> >> This way, I don't think we would need any other form of
> >> synchronization, would we?
> > Such a function would be higher level API than the protocol, since
> > the wire protocol has no concept of a "return value".
> >
> > Yes, that should be all that's needed. "Here it is, just say when
> > you're done."
> 
> Oh, ok, so we would need two events then:
> - Compositor -> client: new image available in buffer index %i
> - Client -> compositor: buffer index %i has been processed and can be 
> re-used

No need for indices, the protocol is object oriented. You can just send
references to the buffers. In Wayland we have specific meanings for
requests and events: events always go server->client, and requests
client->server.

> That should be enough to map that to the PUT/GET pointers of a ring
> buffer. However, the ring buffer would need to be implemented on both
> side (compositor
> and client).

It's more like a pool than a ring. No need to maintain any
specific order, and you can grow/shrink the pool at will. The client
only needs to note in which order the compositor says the buffers are
filled. Preferrably the event carries a timestamp.

> >> In order to reduce the frame rate, another function in the protocol
> >> is needed to set the desired framerate otherwise a lot of copies
> >> will be made for no good reason, as Maarten rightly suggested!
> > Not really. We want zero-copy, and then it's ok to push frames
> > at the compositor's repaint rate (which is variable!). If the client
> > doesn't need some of them, it just acks them without processing. I'm
> > assuming a frame is sent only when the compositor repaints, i.e.
> > something on screen changes. That's how the current recorder works,
> > too.
> 
> If we achieve 0-copy, then this is indeed what should be done.
> 
> If we have to do copies, then it would be nice not to slow down
> memory by doing useless copies. It would thus need an indication
> of how many frames per second should be sent (or how many frames
> should be dropped).

Right... so that'd be a framerate hint, which the compositor only uses
if it needs to do a significant amount of work for each sent frame. We
probably want to keep most of the rate control still in the encoding
program, so it can pick the best frames to drop or duplicate.


Thanks,
pq