[Spice-devel] Adaptive compression choice? [was: Re: [PATCH spice 1/3] dcc_compress_image: Handle NULL drawable]

Mon Jan 25 23:26:19 PST 2016

On Mon, 2016-01-25 at 13:58 -0500, Frediano Ziglio wrote:
> 
> Hi,
> 
> On Mon, 2016-01-25 at 06:48 -0500, Frediano Ziglio wrote:
> 
> 
> Hi Frediano,
> 
> On Čt, 2016-01-14 at 12:52 -0500, Frediano Ziglio wrote:
> > 
> > On Thu, 2016-01-14 at 12:07 -0500, Frediano Ziglio wrote:
> > > > 
> > > > On Thu, Jan 14, 2016 at 10:27:02AM -0500, Frediano Ziglio
> wrote:
> > > > > Had a small discussion with Pavel.
> > > > > We agree that original code is quite complicated and is hard
> to
> > > > > understand
> > > > > the final compression format used.
> > > > > 
> > > > > So we would like to have some public discussion about the
> topic.
> > > > > 
> > > > > I personally agree we should have a single code deciding the
> > > > > compression
> > > > > to use.
> > > > 
> > > > I definitely agree here. For one, having different compression
> being
> > > > used depending on whether the qxl driver is used or not is
> unexpected
> > > > (eg if you set image compression to glz, lz will still be used
> during
> > > > initial bootup, and then will 'switch' to glz later on. I
> haven't looked
> > > > at the code, so there might be good reasons for that).
> > > > 
> > > > > 
> > > > > This is the list of actual compressions:
> > > > > - AUTO_GLZ;
> > > > > - AUTO_LZ;
> > > > > - QUIC;
> > > > > - GLZ;
> > > > > - LZ;
> > > > > - LZ4.
> > > > > A client can also decide to disable compression.
> > > > > 
> > > > > The AUTO_XXX looks like they should use QUIC as a fallback if
> XXX is
> > > > > not
> > > > > possible or if an image with high graduality is detected.
> > > > 
> > > > (side question, do we have numbers on compression ratio and cpu
> usage
> > > > for quic/lz/glz/lz4?)
> > > > 
> > > 
> > > Brief and raw of a Windows replay capture
> > > 
> > >         Images  MB before   MB after  Ratio     CPU time
> > > LZ4     193     24.21       2.43      10.04%    0.04
> > > QUIC    204     23.11       1.66       7.18%    0.44
> > > GLZ     190     20.05       1.2        5.99%    0.14
> > > LZ      202     20.42       2.04       9.99%    0.15
> > > 
> > > So why use Quic ?
> > 
> > Interesting data. Indeed, QUIC seems to be the worst choice. from
> this data,
> > it
> > seems that you'd want GLZ if you were optimizing for network
> bandwidth, and
> > LZ4
> > if you're optimizing for CPU usage. Might be nice to see data for a
> slightly
> > larger sample as well.
> > 
> > Out of curiosity, did you write a little utility for doing this
> benchmark, or
> > did you just modify the code in-place?? Having a little benchmark
> utility
> > that
> > you could run on different replay captures might be a useful thing
> to have in
> > the repository...
> > 
> > Jonathon
> > 
> > 
> 
> No code modification at all. Compile with COMPRESS_STAT enabled, run
> replay
> utility with SPICE_DEBUG_LEVEL=3 set at the end you see a similar
> table
> (I added just ratio with LibreOffice calc).
> Oh... you just need to use -C replay option with
> - 4 quic
> - 5 glz
> - 6 lz
> - 7 lz4
> (not sure about 5/6, maybe swapped).
> 
> would you mind running with no compression so that we can get CPU
> baseline? FWIW I've put inverted numbers to a chart (1/cpu time,
> orig_size/compressed_size, so that greater number is better) and the
> result is here:
> 
> You can imagine the numbers as a number of VMs you can squeeze into a
> single host given a cpu/network constraint.
> 
> Good graph.
> 
> I discovered in the meantime that I was running my test with -O0 (so
> no optimization) so turned out these
> data are completely wrong (lz4 is coded in a different library so not
> affected by -O setting).
> 
> I collected some more data (i did not count uncompressed images) and
> run it with -O2 and it seems that glz is always better.
> 
> This really surprise me! I did some tests and turn out glz is surely
> the winner for compression (twice as lz4) but also
> lz4 is kind of 80% faster (could be I did some other mistakes).
> 
> Which applications where you trying? Which OS and version?
My bad, I put a wrong data for lz4 /o\ . It was a ~5 minutes usage of a
rhel7 guest, I used firefox for a while - read some articles,
played video on youtube (streaming is off). Graph with correct data:
Pavel
> > > > Frediano> > But as Frediano said it would be nice to have a recording of a daily usage [0] and summarize it. 
> > Pavel
> > [0] http://www.spice-space.org/docs/manual/#_recording_replaying_spice_server_traffic> > 
> I was wondering, if the current code is indeed messy if it couldn't be replaced with an adaptive algorithm e.g. starting with some "middle ground" algorithm (LZ4 looks like the candidate) and move up if server detects packet loss or move right if server can't compress images fast enough...
> > I think would be really helpful to collect different replay captures of
> normal day job.
> 


> 
> IIRC VDI benchmark could be the tool to get such a capture.
> > 
> David
> > Frediano
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/spice-devel/attachments/20160126/dc09ccf4/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: spice_compression.png
Type: image/png
Size: 13391 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/spice-devel/attachments/20160126/dc09ccf4/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: compress.png
Type: image/png
Size: 14419 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/spice-devel/attachments/20160126/dc09ccf4/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: compression.png
Type: image/png
Size: 15385 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/spice-devel/attachments/20160126/dc09ccf4/attachment-0005.png>