[Spice-devel] Adaptive compression choice? [was: Re: [PATCH spice 1/3] dcc_compress_image: Handle NULL drawable]

Mon Jan 25 10:58:05 PST 2016

> Hi,

> On Mon, 2016-01-25 at 06:48 -0500, Frediano Ziglio wrote:

> > > Hi Frediano,
> > 
> 

> > > On Čt, 2016-01-14 at 12:52 -0500, Frediano Ziglio wrote:
> > 
> 
> > > > > > On Thu, 2016-01-14 at 12:07 -0500, Frediano Ziglio wrote: > > > > >
> > > > > > >
> > > > > > On
> > > > > > Thu, Jan 14, 2016 at 10:27:02AM -0500, Frediano Ziglio wrote: > > >
> > > > > > >
> > > > > > Had a small discussion with Pavel. > > > > We agree that original
> > > > > > code
> > > > > > is quite complicated and is hard to > > > > understand > > > > the
> > > > > > final
> > > > > > compression format used. > > > > > > > > So we would like to have
> > > > > > some
> > > > > > public discussion about the topic. > > > > > > > > I personally
> > > > > > agree
> > > > > > we
> > > > > > should have a single code deciding the > > > > compression > > > >
> > > > > > to
> > > > > > use. > > > > > > I definitely agree here. For one, having different
> > > > > > compression being > > > used depending on whether the qxl driver is
> > > > > > used
> > > > > > or not is unexpected > > > (eg if you set image compression to glz,
> > > > > > lz
> > > > > > will still be used during > > > initial bootup, and then will
> > > > > > 'switch'
> > > > > > to glz later on. I haven't looked > > > at the code, so there might
> > > > > > be
> > > > > > good reasons for that). > > > > > > > > > > > This is the list of
> > > > > > actual
> > > > > > compressions: > > > > - AUTO_GLZ; > > > > - AUTO_LZ; > > > > -
> > > > > > QUIC;
> > > > > > >
> > > > > > >
> > > > > > > > - GLZ; > > > > - LZ; > > > > - LZ4. > > > > A client can also
> > > > > > > > decide
> > > > > > to disable compression. > > > > > > > > The AUTO_XXX looks like
> > > > > > they
> > > > > > should use QUIC as a fallback if XXX is > > > > not > > > >
> > > > > > possible
> > > > > > or
> > > > > > if an image with high graduality is detected. > > > > > > (side
> > > > > > question, do we have numbers on compression ratio and cpu usage > >
> > > > > > >
> > > > > > for quic/lz/glz/lz4?) > > > > > > > Brief and raw of a Windows
> > > > > > replay
> > > > > > capture > > > >         Images  MB before   MB after  Ratio     CPU
> > > > > > time
> > > > > > > > LZ4     193     24.21       2.43      10.04%    0.04 > > QUIC
> > > > > > > > 204
> > > > > > 23.11       1.66       7.18%    0.44 > > GLZ     190     20.05
> > > > > > 1.2        5.99%    0.14 > > LZ      202     20.42       2.04
> > > > > > 9.99%    0.15 > > > > So why use Quic ? > > Interesting data.
> > > > > > Indeed, QUIC seems to be the worst choice. from this data, > it >
> > > > > > seems
> > > > > > that you'd want GLZ if you were optimizing for network bandwidth,
> > > > > > and
> > > > > > >
> > > > > > LZ4 > if you're optimizing for CPU usage. Might be nice to see data
> > > > > > for
> > > > > > a slightly > larger sample as well. > > Out of curiosity, did you
> > > > > > write
> > > > > > a little utility for doing this benchmark, or > did you just modify
> > > > > > the
> > > > > > code in-place?? Having a little benchmark utility > that > you
> > > > > > could
> > > > > > run
> > > > > > on different replay captures might be a useful thing to have in >
> > > > > > the
> > > > > > repository... > > Jonathon > > No code modification at all. Compile
> > > > > > with
> > > > > > COMPRESS_STAT enabled, run replay
> > > 
> > 
> 
> > > > utility with SPICE_DEBUG_LEVEL=3 set at the end you see a similar table
> > > 
> > 
> 
> > > > (I added just ratio with LibreOffice calc).
> > > 
> > 
> 
> > > > Oh... you just need to use -C replay option with
> > > 
> > 
> 
> > > > - 4 quic
> > > 
> > 
> 
> > > > - 5 glz
> > > 
> > 
> 
> > > > - 6 lz
> > > 
> > 
> 
> > > > - 7 lz4
> > > 
> > 
> 
> > > > (not sure about 5/6, maybe swapped).
> > > 
> > 
> 

> > > would you mind running with no compression so that we can get CPU
> > > baseline?
> > > FWIW I've put inverted numbers to a chart (1/cpu time,
> > > orig_size/compressed_size, so that greater number is better) and the
> > > result
> > > is here:
> > 
> 

> > > You can imagine the numbers as a number of VMs you can squeeze into a
> > > single
> > > host given a cpu/network constraint.
> > 
> 

> > Good graph.
> 

> > I discovered in the meantime that I was running my test with -O0 (so no
> > optimization) so turned out these
> 
> > data are completely wrong (lz4 is coded in a different library so not
> > affected by -O setting).
> 

> I collected some more data (i did not count uncompressed images) and run it
> with -O2 and it seems that glz is always better.

This really surprise me! I did some tests and turn out glz is surely the winner for compression (twice as lz4) but also 
lz4 is kind of 80% faster (could be I did some other mistakes). 

Which applications where you trying? Which OS and version? 

Frediano 

> But as Frediano said it would be nice to have a recording of a daily usage
> [0] and summarize it.

> Pavel

> [0]
> http://www.spice-space.org/docs/manual/#_recording_replaying_spice_server_traffic

> > > I was wondering, if the current code is indeed messy if it couldn't be
> > > replaced with an adaptive algorithm e.g. starting with some "middle
> > > ground"
> > > algorithm (LZ4 looks like the candidate) and move up if server detects
> > > packet loss or move right if server can't compress images fast enough...
> > 
> 

> > > > I think would be really helpful to collect different replay captures of
> > > 
> > 
> 
> > > > normal day job.
> > > 
> > 
> 

> > > IIRC VDI benchmark could be the tool to get such a capture.
> > 
> 

> > > David
> > 
> 
> > Frediano
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/spice-devel/attachments/20160125/0ed71668/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: spice_compression.png
Type: image/png
Size: 13391 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/spice-devel/attachments/20160125/0ed71668/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: compression.png
Type: image/png
Size: 15385 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/spice-devel/attachments/20160125/0ed71668/attachment-0003.png>