<html><head></head><body><div>On Mon, 2016-01-25 at 13:58 -0500, Frediano Ziglio wrote:</div><blockquote type="cite"><div style="font-family: times new roman, new york, times, serif; font-size: 12pt; color: #000000"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><div><br></div><div>Hi,</div><div><br></div><div>On Mon, 2016-01-25 at 06:48 -0500, Frediano Ziglio wrote:</div><blockquote><div style="font-family: times new roman, new york, times, serif; font-size: 12pt; color: #000000"><div><br></div><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><div><br></div> Hi Frediano,<br><div><br></div> On Čt, 2016-01-14 at 12:52 -0500, Frediano Ziglio wrote: <blockquote><pre><span style="color: #737373;">> </span> <span style="color: #737373;">> On Thu, 2016-01-14 at 12:07 -0500, Frediano Ziglio wrote:</span> <span style="color: #737373;">> > > </span> <span style="color: #737373;">> > > On Thu, Jan 14, 2016 at 10:27:02AM -0500, Frediano Ziglio wrote:</span> <span style="color: #737373;">> > > > Had a small discussion with Pavel.</span> <span style="color: #737373;">> > > > We agree that original code is quite complicated and is hard to</span> <span style="color: #737373;">> > > > understand</span> <span style="color: #737373;">> > > > the final compression format used.</span> <span style="color: #737373;">> > > > </span> <span style="color: #737373;">> > > > So we would like to have some public discussion about the topic.</span> <span style="color: #737373;">> > > > </span> <span style="color: #737373;">> > > > I personally agree we should have a single code deciding the</span> <span style="color: #737373;">> > > > compression</span> <span style="color: #737373;">> > > > to use.</span> <span style="color: #737373;">> > > </span> <span style="color: #737373;">> > > I definitely agree here. For one, having different compression being</span> <span style="color: #737373;">> > > used depending on whether the qxl driver is used or not is unexpected</span> <span style="color: #737373;">> > > (eg if you set image compression to glz, lz will still be used during</span> <span style="color: #737373;">> > > initial bootup, and then will 'switch' to glz later on. I haven't looked</span> <span style="color: #737373;">> > > at the code, so there might be good reasons for that).</span> <span style="color: #737373;">> > > </span> <span style="color: #737373;">> > > > </span> <span style="color: #737373;">> > > > This is the list of actual compressions:</span> <span style="color: #737373;">> > > > - AUTO_GLZ;</span> <span style="color: #737373;">> > > > - AUTO_LZ;</span> <span style="color: #737373;">> > > > - QUIC;</span> <span style="color: #737373;">> > > > - GLZ;</span> <span style="color: #737373;">> > > > - LZ;</span> <span style="color: #737373;">> > > > - LZ4.</span> <span style="color: #737373;">> > > > A client can also decide to disable compression.</span> <span style="color: #737373;">> > > > </span> <span style="color: #737373;">> > > > The AUTO_XXX looks like they should use QUIC as a fallback if XXX is</span> <span style="color: #737373;">> > > > not</span> <span style="color: #737373;">> > > > possible or if an image with high graduality is detected.</span> <span style="color: #737373;">> > > </span> <span style="color: #737373;">> > > (side question, do we have numbers on compression ratio and cpu usage</span> <span style="color: #737373;">> > > for quic/lz/glz/lz4?)</span> <span style="color: #737373;">> > > </span> <span style="color: #737373;">> > </span> <span style="color: #737373;">> > Brief and raw of a Windows replay capture</span> <span style="color: #737373;">> > </span> <span style="color: #737373;">> > Images MB before MB after Ratio CPU time</span> <span style="color: #737373;">> > LZ4 193 24.21 2.43 10.04% 0.04</span> <span style="color: #737373;">> > QUIC 204 23.11 1.66 7.18% 0.44</span> <span style="color: #737373;">> > GLZ 190 20.05 1.2 5.99% 0.14</span> <span style="color: #737373;">> > LZ 202 20.42 2.04 9.99% 0.15</span> <span style="color: #737373;">> > </span> <span style="color: #737373;">> > So why use Quic ?</span> <span style="color: #737373;">> </span> <span style="color: #737373;">> Interesting data. Indeed, QUIC seems to be the worst choice. from this data,</span> <span style="color: #737373;">> it</span> <span style="color: #737373;">> seems that you'd want GLZ if you were optimizing for network bandwidth, and</span> <span style="color: #737373;">> LZ4</span> <span style="color: #737373;">> if you're optimizing for CPU usage. Might be nice to see data for a slightly</span> <span style="color: #737373;">> larger sample as well.</span> <span style="color: #737373;">> </span> <span style="color: #737373;">> Out of curiosity, did you write a little utility for doing this benchmark, or</span> <span style="color: #737373;">> did you just modify the code in-place?? Having a little benchmark utility</span> <span style="color: #737373;">> that</span> <span style="color: #737373;">> you could run on different replay captures might be a useful thing to have in</span> <span style="color: #737373;">> the repository...</span> <span style="color: #737373;">> </span> <span style="color: #737373;">> Jonathon</span> <span style="color: #737373;">> </span> <span style="color: #737373;">> </span> No code modification at all. Compile with COMPRESS_STAT enabled, run replay utility with SPICE_DEBUG_LEVEL=3 set at the end you see a similar table (I added just ratio with LibreOffice calc). Oh... you just need to use -C replay option with - 4 quic - 5 glz - 6 lz - 7 lz4 (not sure about 5/6, maybe swapped). </pre></blockquote><br> would you mind running with no compression so that we can get CPU baseline? FWIW I've put inverted numbers to a chart (1/cpu time, orig_size/compressed_size, so that greater number is better) and the result is here:<br><img src="cid:1453793179.2657.7.camel@redhat.com" align="bottom" border="0" data-inline="" data-name="spice_compression.png"><br> You can imagine the numbers as a number of VMs you can squeeze into a single host given a cpu/network constraint.<br></blockquote><div><br></div><div>Good graph.<br></div><div><br></div><div>I discovered in the meantime that I was running my test with -O0 (so no optimization) so turned out these<br></div><div>data are completely wrong (lz4 is coded in a different library so not affected by -O setting).<br></div></div></blockquote><div><br></div><div>I collected some more data (i did not count uncompressed images) and run it with -O2 and it seems that glz is always better.</div></blockquote><div><br></div><div>This really surprise me! I did some tests and turn out glz is surely the winner for compression (twice as lz4) but also<br></div><div>lz4 is kind of 80% faster (could be I did some other mistakes).<br></div><div><br>Which applications where you trying? Which OS and version?<br></div></div></blockquote><div><br></div><div>My bad, I put a wrong data for lz4 /o\ . It was a ~5 minutes usage of a rhel7 guest, I used firefox for a while - read some articles,</div><div>played video on youtube (<b>streaming</b> is <b>off</b>). Graph with correct data:</div><div><img src="cid:1453793179.2657.8.camel@redhat.com"><br></div><div>Pavel</div><div><br></div><blockquote type="cite"><div style="font-family: times new roman, new york, times, serif; font-size: 12pt; color: #000000"><div><br></div><div><br></div><div><br></div><div>Frediano<br></div><div><br></div><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><div>But as Frediano said it would be nice to have a recording of a daily usage [0] and summarize it. </div><div><br></div><div><img src="cid:1453793179.2657.9.camel@redhat.com" data-inline="" data-name="compression.png"><br></div><div>Pavel</div><div><br></div><div>[0] <a href="http://www.spice-space.org/docs/manual/#_recording_replaying_spice_server_traffic" target="_blank">http://www.spice-space.org/docs/manual/#_recording_replaying_spice_server_traffic</a><br data-mce-bogus="1"></div><div><br></div><blockquote><div style="font-family: times new roman, new york, times, serif; font-size: 12pt; color: #000000"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"> I was wondering, if the current code is indeed messy if it couldn't be replaced with an adaptive algorithm e.g. starting with some "middle ground" algorithm (LZ4 looks like the candidate) and move up if server detects packet loss or move right if server can't compress images fast enough...<br><div><br></div><blockquote><pre>I think would be really helpful to collect different replay captures of normal day job. </pre></blockquote><br> IIRC VDI benchmark could be the tool to get such a capture.<br><div><br></div> David</blockquote><div><br></div><div>Frediano</div></div></blockquote></blockquote><div><br></div></div></blockquote></body></html>