[Spice-devel] aspeed: video frames pass-through

[Francois Gouget]

On Tue, 27 Oct 2015, Anton D. Kachalov wrote:
[...]
> I've started with Francois Gouget's https://github.com/fgouget/spice 
> repo with GStreamer support. It were my first steps with Spice. I've 
> added one more encoder: "aspeed" and one more image type: "AST" and 
> tries to pass fake image update command from the server command's 
> queue 
> (https://github.com/ya-mouse/spice/blob/gst/server/tests/ast_base.c ) 
> based on test_display_no_ssl to trigger stream encoding. From the 
> server side I stucked at "pixman" decoding at "draw_copy", so I've 
> just skip copy/blitting on semi-invalid image and read actual data 
> from the temporary file during `aspeed_encoder_encode_frame' call. 
> It's very-very ugly, but I need just a working concept.

>From my limited understanding of your setup and Spice I think it would 
makes sense to treat the whole screen as a video and send it from the 
server to the client as a video stream.

On the client you'd need a GStreamer decoder (*) that can interpret the 
frame update instructions to return the full bitmap to be shown on 
screen. That would not be very different from every other inter-frame 
video compression codec.

On the server side it's more tricky because the video encoder 
normally gets full frames from the generic Spice codec. I guess 
that's why you had to add new bitmap and compression types. Maybe an 
alternative would be to have a GStreamer video source that would grab 
the relevant video stream on its own and thus bypass the rest of Spice. 
This might cause other issues though.

As an extra note, the drawback of passing the Aspeed video stream as is 
is that the client will need to have the relevant codec to decode it 
(either in the form of the relevant GStreamer decoder or a Spice client 
with the right support compiled in). But avoiding this would mean either 
converting the Aspeed instructions into primitives supported by Spice, 
or to rebuild the full frame but then efficiently sending it to the 
client would likely require reencoding it as a video stream.


(*) This could also take the form of a Spice-specific video decoder but 
    I don't think doing it as a generic GStreamer decoder would be much 
    different and it would make reuse in other contexts easier.

-- 
Francois Gouget <fgouget at codeweavers.com>