[Spice-devel] [PATCH spice-server 00/28] adaptive video streaming

[Yonit Halperin]

On 02/26/2013 02:49 PM, Marc-André Lureau wrote:
>
>
> ----- Mensaje original -----
>> Hi,
>>
>> On 02/26/2013 01:40 PM, Marc-André Lureau wrote:
>>> Hi
>>>
>>> ----- Mensaje original -----
>>>> The adaptive video streaming is implemented by the following
>>>> heuristic:
>>>> Given a bit rate, we calculate the best combination of mjpeg
>>>> quality
>>>> and frame rate (henceforth, the stream parameters) for this
>>>> bit rate. In order to decide this combination, we evaluate the
>>>> encoding size for different jpeg
>>>> qualities by applying them on successive frames.
>>>
>>> But no downscaling?
>> Good point. However, the default libjpeg sampling factors values are
>> already 2 for luminance components and 1 for chrominance components
>> (both horizontal and vertical), while 4 is the highest value, and
>> larger
>> factor means higher resolution. It is worth trying decreasing the
>> luminance to 1.
>
> According to wikipedia, the jpeg YUV downscaling is:
>
> "The ratios at which the downsampling is ordinarily done for JPEG images are 4:4:4 (no downsampling), 4:2:2 (reduction by a factor of 2 in the horizontal direction), or (most commonly) 4:2:0"
>
> So luminance is not downsampled. Is that what you said? According to my experience with video codecs (not so much with jpeg itself), downsampling really makes a difference to most codecs, both in term of overally quality and cpu usage.
I understand from libjpeg documentation that by default the luminance is 
downsampled by a factor of 2, and the chrominance by a factor of 4. So 
we can try downsampling the luminance by a factor of 4.
>
>>> How is this going to work with multiple client?
>> Each client have a separate rate-controller and a different stream.
>> Ages
>> ago, we (alon an I) talked about clustering clients according to
>> their
>> network setup, and encode each message only once per cluster. I think
>> that when we have such mechanism, we can use one rate controller per
>> cluster, and then we can base our estimations on the "weakest" client
>> in
>> the cluster.
>
> Ah good, just wanted to make sure this was covered somehow.
>
>>>> - Add vp8 encoding
>>>
>>> Video codec is really bad, it will really take a lot of CPU, unless
>>> it can be done on GPU?
>>>
>> I haven't tried it yet, is it even worse than the libjpeg-turbo
>> performance? For high resolution videos, the cpu usage for
>> libjpeg-turbo
>> is very high.
>> About hardware acceleration (at least for decoding):
>> http://en.wikipedia.org/wiki/WebM#Hardware
>
> For high resolution video (>720p) the videos codecs are _real_ hogs. I use to run cpu/qos quality of video codecs, and libvp8 at that time wasn't able to do real time 1080p even on our fastest hardware (using 8 xeon cores iirc). It wasnt't very good at running on multiple threads. I can imagine it may have improved for the past 2 years. (x264 with realtime quality level was ~4x faster iirc).
>
It is worth trying. I know that there are live streaming applications 
that use it, so it may have improved in the last years.