[Spice-devel] Thoughts about improving streaming video

Fri Jul 22 02:38:33 PDT 2011

Talking about improving streaming video in spice, I am working on a thesis
project about Spice, in particular how it handles streaming video, only in
Linux.

In particular I would like to know if there are possibilities to improve it
by
- replacing mjpeg with another encoder?
- or changing how stream data are transmitted to the client? for example
using a separate socket for the video and give it higher priority?

Do you think that is quite complex to implement this works ? The second
solution seems easier, but I don't know if can improve something.
I read the spice code and all streaming-related stuff, but I would like to
know in advance if this work is too complex before starting it.

Thanks.

2011/6/28 Yaniv Kaul <ykaul at redhat.com>

> On 06/28/2011 11:14 AM, John A. Sullivan III wrote:
>
>> On Tue, 2011-06-28 at 09:30 +0200, David Jaša wrote:
>>
>>> Dne 28.6.2011 00:39, John A. Sullivan III napsal(a):
>>>
>>>> On Tue, 2011-06-28 at 00:14 +0200, David Jaša wrote:
>>>>
>>>>> Dne 27.6.2011 20:45, John A. Sullivan III napsal(a):
>>>>>
>>>>>> On Mon, 2011-06-27 at 08:32 +0300, Yaniv Kaul wrote:
>>>>>>
>>>>>>> Licensing and patent concerns of x264 aside (see
>>>>>>> http://mailman.videolan.org/**pipermail/x264-devel/2010-**
>>>>>>> July/007508.html<http://mailman.videolan.org/pipermail/x264-devel/2010-July/007508.html>
>>>>>>> ),
>>>>>>> the more I think about it, the more it makes sense to me to have
>>>>>>> H.264
>>>>>>> offload feature in QXL:
>>>>>>>
>>>>>> This sounds great (gaming on SPICE!) but that's a pretty big aside.
>>>>>>  How
>>>>>> would we handle the patent issues? - John
>>>>>>
>>>>>>  Gaming needs 3D acceleration, video needs video acceleration. That's
>>>>> two
>>>>> distinct features despite being implemented by the same transistors. :)
>>>>>
>>>>> Patent issues can be solved pretty easily in an ugly way - make it
>>>>> possible to pass any video stream to client if the client claims it
>>>>> supports the container + codec, but do not include any patented stuff
>>>>> in
>>>>> client by default. Users could observe the U.S. patent law and buy
>>>>> codecs from e.g. Fluendo, who will pay the royalties for them, or if
>>>>> they don't mind crossing it, they can add support of patented codecs
>>>>> themselves.
>>>>>
>>>>>  <snip>
>>>> <grin>   I was making a reference to the X264 dev in they hyperlink who
>>>> exclaimed, "ideoconferencing?  Pah!  I’m playing Call of Duty 4 over a
>>>> live video stream!".  Interesting idea though!
>>>>
>>>> However, more seriously, that is ugly.  Is it really something we would
>>>> expect our users to do? Would it completely torpedo most commercial
>>>> installation in the contract legal review stage?
>>>>
>>>>  It's exactly what Fedora does:
>>>
>>> ===========================
>>>
>>> Patent licenses usually require the licensee to pay royalties based on
>>> the number of users. Since Fedora is free and open source software, the
>>> effective number of users is essentially unrestricted. Patent holders are
>>> generally unwilling to give a blanket patent license for unlimited use;
>>> moreover, the royalty payments would be too high for it to be practical for
>>> the Fedora Project, or its sponsors, to pay them. Proprietary operating
>>> systems like Microsoft Windows include the costs of third-party patent
>>> licenses paid by Microsoft in the pricing of the product as sold to end
>>> users. Fedora is not sold commercially, so there is no way to recoup these
>>> substantial expenses.
>>>
>>> ...
>>>
>>> Note that Fluendo offers a MP3 plugin for the Gstreamer multimedia
>>> framework (used by Totem, Rhythmbox and other multimedia applications) for
>>> free and other codecs and DVD player for a price that includes patent
>>> licenses. Fedora does not include or endorse these options but you can
>>> choose to use them with Fedora if you want to.
>>>
>>> ==========================
>>>
>>> (from http://fedoraproject.org/wiki/**Software_Patents#Can.27t_you_**
>>> pay_the_patent_license_fees_**for_patent_encumbered_codecs.**3F<http://fedoraproject.org/wiki/Software_Patents#Can.27t_you_pay_the_patent_license_fees_for_patent_encumbered_codecs.3F>)
>>>
>>>    From reading the referenced hyperlink about x264
>>>> (http://x264dev.multimedia.cx/**archives/249<http://x264dev.multimedia.cx/archives/249>)
>>>> it sounds like an excellent
>>>> solution but only if we can practically surmount the patent issues.
>>>> Thoughts?
>>>>
>>> <snip>
>> Thanks for the reference,  Having read it, I wonder if we should follow
>> its advice and focus on VP8 or Theora.  What does RedHat do to deal with
>> the issue in large commercial accounts as opposed to Fedora? I suppose
>> the big question is if VP8 or Theora has implemented a similar
>> on-the-fly encoding scheme as outlined in the referenced x264dev page.
>>
>
> - I cannot and should not speak on behalf of Red Hat regarding patents, but
> the whole idea (or at least my idea) of using x264 was to offload H.264
> decoding entirely - from the guest all the way to the client. Using VP8 or
> any other decoder would just replace MJPEG's functionality today - the guest
> decompresses (in software) and spice re-compresses (in software). I don't
> think it's good enough.
> - Doesn't seem like the other video scheme are prevalent enough nor have
> hardware acceleration support (yet - VP8 has something in the works -
> http://blog.webmproject.org/**2011/06/expanding-vp8-**
> hardware-decoder-for-full.html<http://blog.webmproject.org/2011/06/expanding-vp8-hardware-decoder-for-full.html>. I highly doubt it'll ever get as popular as H.264 HW support is).
>
>
> Y.
>
>  If no one knows off hand, I wonder if that would be a good place for
>> Andrea's project to start - evaluating if those codecs have similar
>> implementations to see if they could be used in SPICE as an MJPEG
>> alternative - John
>>
>> ______________________________**_________________
>> Spice-devel mailing list
>> Spice-devel at lists.freedesktop.**org <Spice-devel at lists.freedesktop.org>
>> http://lists.freedesktop.org/**mailman/listinfo/spice-devel<http://lists.freedesktop.org/mailman/listinfo/spice-devel>
>>
>
> ______________________________**_________________
> Spice-devel mailing list
> Spice-devel at lists.freedesktop.**org <Spice-devel at lists.freedesktop.org>
> http://lists.freedesktop.org/**mailman/listinfo/spice-devel<http://lists.freedesktop.org/mailman/listinfo/spice-devel>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/spice-devel/attachments/20110722/d62c97ee/attachment.html>