[Spice-devel] video passthrough ideas

[Marc-André Lureau]

Hi Zeeshan,

(quick reply)

----- Original Message -----
> Hi everyone,
> 1. VAAPI (Video Acceleration API): The idea here is to implement a
> custom VAAPI driver for the guest that redirects all decoding/playback
> calls to the client through a spice channel. On the client side, we
> create a channel handler based on gstreamer and its playbin2 element.
> 
> a. We still need to figure out how our VAAPI driver running in
> user-space (its loaded as shared-object in the application) will
> communicate to the spice server running inside Qemu?

I think that's what spicevmc channels are for. Someone else should tell you the details, or you'll have to investigate a little the agent code.

> e. Haven't yet figured how the interaction with QXL/windowing-system
> on client can/will work (i-e telling QXL to leave the part of the
> screen where the video is being played on as blank etc), need to read
> more of QXL and VAAPI code to figure i guess.

It should be "composited" like we do with other surfaces (including mjpeg).

My concern for VAAPI though (and other acceleration API) is that it is not a sink, which handle completely decoding and displaying. I understand it can take care of some expensive operations, but the result probably has to be returned to the server/guest. If that's the case, that won't fly well. Perhaps a trick would be to keep and send the original data, and find a way to use it in the client... (btw, I wouldn't be surprised if V4L2 has some way to send and display MPEG streams)

> 2. GStreamer all the way: This is the idea that I originally came-up
> with when Christophe suggested a "magic GStreamer element" on IRC. The

This is indeed a solution we have discussed in the past. It has several advantages:

- at the moment there are just too many video acceleration API to choose one, so going higher level sounds reasonable
- we have a fairly good idea how to do it, since we know GStreamer etc..
- we could send the whole stream to the client, which give us same advantage for audio, easy a/v sync, and a separate audio channel for free
- it will work for any client platform  (spice-gtk already works with GStreamer on Windows)
- if the solution is proved to be robust and simple, for Windows, a DirectShow sink could convert the stream to a GStreamer flow, which will be handled in GDP the same way
- for MacOS, I suppose we could also write a simple QuickTime component doing the same conversion

Just my 2 cents, keep looking!

-- 
Marc-André Lureau