[Spice-devel] [spice v13 10/29] server: Handle and recover from GStreamer encoding errors

[Francois Gouget]

On Fri, 3 Jun 2016, Christophe Fergeau wrote:
[...]
> For what it's worth, I tried asking GStreamer people if they had
> suggestions how we could handle what is being done in this patch, but
> they did not have a silver bullet :)

I'm not surprised. The GStreamer doc says you're not supposed to expect 
to get one buffer out for each buffer you push into the pipeline. This 
totally makes sense because the input buffers may not correspond to a 
frame each (think data coming from the network) and because encoders may 
have a non-zero latency (i.e. need to see multiple frames before they 
can put out a new compressed frame). These are all things we carefully 
avoid. More importantly we have to because the encoding is done as part 
of the marshalling step in Spice, and thus is synchronous.

So the right fix (from the GStreamer point of view) would be to push 
frames to be encoded earlier, and then once the GStreamer pipeline gives 
us a compressed buffer, push it somehow on a queue to be sent over the 
network. Then when the pipeline drops a buffer the corresponding 
resources would simply be freed and we would never get a callback to 
send the frame to the client. So no deadlock. It also means the client 
would never get these dropped frames unlike now where we have a 
non-stream fallback. So that may be a problem. Obviously the builtin 
MJPEG encoder would have to also work with this new architecture too.

This would be a pretty major change so I think the current solution is 
better. Maybe one day if someone feels like breaking everything...


-- 
Francois Gouget <fgouget at codeweavers.com>