[Spice-devel] Thoughts about improving streaming video

Mon Jun 27 01:50:01 PDT 2011

Hey,

Fwiw,
http://blogs.adobe.com/penguinswf/2010/01/solving_different_problems.html
and
http://www.splitted-desktop.com/static/en/pdf/actu/Linux_with_Gnash.pdf
might be interesting readings.

Christophe

On Mon, Jun 27, 2011 at 08:32:52AM +0300, Yaniv Kaul wrote:
> Licensing and patent concerns of x264 aside (see http://mailman.videolan.org/pipermail/x264-devel/2010-July/007508.html),
> the more I think about it, the more it makes sense to me to have
> H.264 offload feature in QXL:
> 1. Many (Flash, HTML5, others) are moving to H.264 - and take
> advantage of hardware acceleration if such is available, for
> complete decoding of the stream. If QXL advertises itself as being
> able to HW accel. the stream, we need to just pick the stream and
> move it to the client. On the client, we can either download it to
> the local GPU which will HW accelerate its decoding, or use
> software-based decoding (x264, for example).
> 2. Then we can revisit x264 and see if it really provides a
> noticeable advantage over mjpeg - clearly having one video solution
> is cleaner and less hassle, and x264 seems to have some nice
> properties we can use (see http://x264dev.multimedia.cx/archives/249
> regarding low latency).
> 
> Both are not trivial tasks, I'm afraid.
> Y.
> 
> 
> On 06/25/2011 06:33 PM, John A. Sullivan III wrote:
> >Hello, all.  Andrea Celestino was kind enough to email me off list about
> >streaming video performance as reducing SPICE bandwidth consumption for
> >streaming video is his project as a Computer Engineering student.  He
> >agreed that I could repost our conversation to invite input from others
> >on the list.  The (top posted - sorry) thread is below - John
> >
> >
> >Hi, Andrea.  No problem with your English.  It is certainly better than
> >my very rusty Italian!
> >
> >I know little about video other than as a network engineer and I am not
> >a programmer but I do know what we need.  From that ignorant
> >perspective, I see several possibilities.
> >
> >Video is inherently brutal when it comes to network bandwidth and
> >processing.  There are also different usage scenarios.  For example, it
> >makes sense to stream when the video needs to be played only once and
> >started immediately.  However, if it is going to be viewed multiple
> >times, the Citrix approach of downloading and playing locally makes more
> >sense.  I've mentioned on the list how we can do this with technology
> >already in the X2Go project.  It is probably the least desirable option
> >and the least interesting for your project.  Of course, we have to
> >handle all this complexity in a way which does not confuse the end user
> >who just wants to click on their file or web site and have it work!
> >
> >For streaming the video between the SPICE server and client, I think we
> >also have a couple of options.  I don't think there is any magic that
> >can be done.  It is a matter of using the most appropriate codec for the
> >transmission.  The challenge, as already mentioned on the list, is
> >encoding on the fly.  The clients need to decode no matter what but, in
> >non-SPICE usage, the file is pre-encoded.  So, in our selection of
> >codecs, speed of encoding versus decoding becomes critical.  We must
> >also not only choose one which uses an open source license but one which
> >is not patent encumbered.  I believe that rules out X264.  The most
> >likely candidate is probably VP8 with Theora as an alternative.
> >
> >Once a codec (or a selection of codecs) has been chosen, the next
> >challenge is how to switch.  One option is to simply make it a setting
> >in the SPICE parameters for qemu.  The two advantages are that it is the
> >simplest and it allows the system administrator (who hopefully knows his
> >system well) to make the balance between CPU utilization and bandwidth.
> >The disadvantage is that it does not adapt to the users environment and
> >may require a reboot to change.
> >
> >The second option is to select the codec dynamically.  Not only would we
> >select whether to use a lossy or lossless codec based upon the pixel
> >change rate, but we would then choose the codec based upon the bandwidth
> >to the client balanced against the bitrate of the video.  There is
> >already precedent for this logic in that SPICE already chooses whether
> >to use lossy or lossless codecs based upon detected bandwidth.  From a
> >previous post by Marian Krcmarik:
> >
> >'Once client connects to a guest, "spice server" determines client's
> >bandwidth (look at code for details, I believe It could be more
> >appropriate and guys are working on it). "jpeg-wan-compression" and
> >"zlib-glz-wan-compression" are enabled in case the client's bandwidth is
> ><10Mbps, otherwise It's disabled. When jpeg compression and zlib over
> >glz compression are enabled then photo-like bitmaps are compressed by
> >lossy jpeg compression and textual/artificial bitmaps are compressed by
> >lossy zlib on top of GLZ.'
> >
> >SPICE would compare the bandwidth to the bandwidth requirements of the
> >video and, if the video bandwidth requirements are some percentage less
> >than the available bandwidth (to leave some of other traffic - I do not
> >know what that number should be - 80%?), it uses MJPEG for better
> >quality and lower CPU utilization.  If not, it uses VP8 or whatever we
> >choose.  This could be multiple choice depending on bitrate/bandwidth if
> >that makes sense and not just MJPEG vs VP8.
> >
> >This would be the most interesting approach although we should sacrifice
> >our programming interest to the best solution for the end users.  It
> >does have the advantage of not needing to be tweaked by the admin and
> >adapting to whether the user is on the end of a 3G cell connection or a
> >100 Mbps WAN link.
> >
> >As I think about it, we already have precedent for making it the best of
> >all worlds by using an auto or fixed setting like SPICE does for other
> >parameters - auto activates the automated routine above and a fixed
> >value (MJPEG or VP8) specifically locks it to always use the specified
> >codec.  This is probably a good idea in general but would be
> >particularly helpful in the early stages while we tweak what that magic
> >percentage is for allowing other traffic, i.e., does the higher
> >utilization codec kick in at 80%, 60%, 95%.
> >
> >Would you mind if I posted this response to the list to see if others
> >have helpful input? Thanks - John
> >
> >On Fri, 2011-06-24 at 11:18 +0200, Andrea Celestino wrote:
> >>Hi,
> >>
> >>first of all I am studying the source coude of spice, because I need
> >>to understand how spice detects video streaming, how it sends stream
> >>data to client with socket and how it compress data with mjpeg.
> >>I have not any idea at the moment, I need to do some tests.
> >>I am very interested.
> >>Do you have an idea?
> >>
> >>
> >>Thanks.
> >>
> >>
> >>p.s. sorry for my english :)
> >>
> >>
> >>
> >>
> >>
> >>2011/6/23 John A. Sullivan III<jsullivan at opensourcedevel.com>
> >>         Certo! Hope I remembered that correctly.  I would be extremely
> >>         interested. What did you have in mind? Thanks - John
> >>
> >>         On Thu, 2011-06-23 at 10:29 +0200, Andrea Celestino wrote:
> >>         >  Hi,
> >>         >  I have read your message on Spice mailing lists and I am
> >>         interesting
> >>         >  in streaming video performance with spice. I am an italian
> >>         student in
> >>         >  computer engineering, during this period I am working on
> >>         reducing
> >>         >  bandwidth usage with streaming video in Spice. I see that
> >>         you are
> >>         >  interested in the same problem. I think that we can
> >>         cooperate. What do
> >>         >  you think about it?
> >>
> >>
> >>
> >>
> >
> >
> >_______________________________________________
> >Spice-devel mailing list
> >Spice-devel at lists.freedesktop.org
> >http://lists.freedesktop.org/mailman/listinfo/spice-devel
> 
> _______________________________________________
> Spice-devel mailing list
> Spice-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/spice-devel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/spice-devel/attachments/20110627/66f17b71/attachment-0001.pgp>