[Spice-devel] Adaptive compression choice? [was: Re: [PATCH spice 1/3] dcc_compress_image: Handle NULL drawable]

[Frediano Ziglio]

> Hi Frediano,

> On Čt, 2016-01-14 at 12:52 -0500, Frediano Ziglio wrote:
> > > > On Thu, 2016-01-14 at 12:07 -0500, Frediano Ziglio wrote: > > > > > >
> > > > On
> > > > Thu, Jan 14, 2016 at 10:27:02AM -0500, Frediano Ziglio wrote: > > > >
> > > > Had a small discussion with Pavel. > > > > We agree that original code
> > > > is quite complicated and is hard to > > > > understand > > > > the
> > > > final
> > > > compression format used. > > > > > > > > So we would like to have some
> > > > public discussion about the topic. > > > > > > > > I personally agree
> > > > we
> > > > should have a single code deciding the > > > > compression > > > > to
> > > > use. > > > > > > I definitely agree here. For one, having different
> > > > compression being > > > used depending on whether the qxl driver is
> > > > used
> > > > or not is unexpected > > > (eg if you set image compression to glz, lz
> > > > will still be used during > > > initial bootup, and then will 'switch'
> > > > to glz later on. I haven't looked > > > at the code, so there might be
> > > > good reasons for that). > > > > > > > > > > > This is the list of
> > > > actual
> > > > compressions: > > > > - AUTO_GLZ; > > > > - AUTO_LZ; > > > > - QUIC; >
> > > > >
> > > > > > - GLZ; > > > > - LZ; > > > > - LZ4. > > > > A client can also
> > > > > > decide
> > > > to disable compression. > > > > > > > > The AUTO_XXX looks like they
> > > > should use QUIC as a fallback if XXX is > > > > not > > > > possible or
> > > > if an image with high graduality is detected. > > > > > > (side
> > > > question, do we have numbers on compression ratio and cpu usage > > >
> > > > for quic/lz/glz/lz4?) > > > > > > > Brief and raw of a Windows replay
> > > > capture > > > >         Images  MB before   MB after  Ratio     CPU
> > > > time
> > > > > > LZ4     193     24.21       2.43      10.04%    0.04 > > QUIC
> > > > > > 204
> > > > 23.11       1.66       7.18%    0.44 > > GLZ     190     20.05
> > > > 1.2        5.99%    0.14 > > LZ      202     20.42       2.04
> > > > 9.99%    0.15 > > > > So why use Quic ? > > Interesting data.
> > > > Indeed, QUIC seems to be the worst choice. from this data, > it > seems
> > > > that you'd want GLZ if you were optimizing for network bandwidth, and >
> > > > LZ4 > if you're optimizing for CPU usage. Might be nice to see data for
> > > > a slightly > larger sample as well. > > Out of curiosity, did you write
> > > > a little utility for doing this benchmark, or > did you just modify the
> > > > code in-place?? Having a little benchmark utility > that > you could
> > > > run
> > > > on different replay captures might be a useful thing to have in > the
> > > > repository... > > Jonathon > > No code modification at all. Compile
> > > > with
> > > > COMPRESS_STAT enabled, run replay
> 
> > utility with SPICE_DEBUG_LEVEL=3 set at the end you see a similar table
> 
> > (I added just ratio with LibreOffice calc).
> 
> > Oh... you just need to use -C replay option with
> 
> > - 4 quic
> 
> > - 5 glz
> 
> > - 6 lz
> 
> > - 7 lz4
> 
> > (not sure about 5/6, maybe swapped).
> 

> would you mind running with no compression so that we can get CPU baseline?
> FWIW I've put inverted numbers to a chart (1/cpu time,
> orig_size/compressed_size, so that greater number is better) and the result
> is here:

> You can imagine the numbers as a number of VMs you can squeeze into a single
> host given a cpu/network constraint.

Good graph. 

I discovered in the meantime that I was running my test with -O0 (so no optimization) so turned out these 
data are completely wrong (lz4 is coded in a different library so not affected by -O setting). 

> I was wondering, if the current code is indeed messy if it couldn't be
> replaced with an adaptive algorithm e.g. starting with some "middle ground"
> algorithm (LZ4 looks like the candidate) and move up if server detects
> packet loss or move right if server can't compress images fast enough...

> > I think would be really helpful to collect different replay captures of
> 
> > normal day job.
> 

> IIRC VDI benchmark could be the tool to get such a capture.

> David
Frediano 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/spice-devel/attachments/20160125/26c4e5aa/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: spice_compression.png
Type: image/png
Size: 13391 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/spice-devel/attachments/20160125/26c4e5aa/attachment-0001.png>