<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On 7 Mar 2017, at 16:37, Frediano Ziglio <<a href="mailto:fziglio@redhat.com" class="">fziglio@redhat.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div style="" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class=""><br class="Apple-interchange-newline"><br class="">Hi Snir,<br class=""><br class="">Regarding compression and another topic recently evoked, endianness, I<br class="">wonder<br class="">if encoding using LEB128 had been discussed?<br class=""><br class="">LEB128 encodes any value below 128 as 1 bytes, any value below ~16000 as 2<br class="">bytes, and so on. It is size and endianness independent. This means that we<br class="">can extend a field from uint16 to uint32 without protocol incompatibility.<br class="">It is much less computationally expensive than full compression. I think it<br class="">would apply very well to a lot of our meta-data, but would be less<br class="">effective<br class="">than regular compression algorithms with image data proper. So it’s<br class="">addressing another part of the problem.<br class=""><br class="">I have no real feeling yet for how important that other part is in the<br class="">streams, but my guts tell me “probably not much” (i.e. I expect much more<br class="">image data than meta-data). Still, it might be worth trying. But it’s<br class="">probably not just a capability: if we want to encode most fields that way,<br class="">this means a protocol bump. Not sure it’s worth it.<br class=""><br class="">The code to encode and decode is really short. Here is an example for<br class="">writing:<br class=""><a href="https://github.com/c3d/XL-programming-language/blob/master/xlr/serializer.cpp#L219" class="">https://github.com/c3d/XL-programming-language/blob/master/xlr/serializer.cpp#L219</a>.<br class="">Here is an example for reading:<br class=""><a href="https://github.com/c3d/XL-programming-language/blob/master/xlr/serializer.cpp#L406" class="">https://github.com/c3d/XL-programming-language/blob/master/xlr/serializer.cpp#L406</a>.<br class=""><br class=""><br class="">Christophe<br class=""><br class=""></blockquote><br class="">Maybe helpful however Snir is working more on payload compression than<br class="">structured data so the gain would be not great (he use few fields).<br class="">However maybe some statistics can be done. Our "mini" header is 16 bit<br class="">type and 32 bit length. For instance you can add some statistics recording<br class="">- number of messages<br class="">- total messages payload<br class="">- total LEB128 encoded header (the not encoded is number of messages * 6).<br class="">If you want also to try compressing (or doing statistics on) fields you<br class="">could add some code to the code generated by python scripts in spice-common.<br class=""><br class=""></blockquote><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">I instrumented a bit the code to understand how this LEB128 could affect</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">fields (not the header).</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">The algorithm remove about 50-60% of field data however this affect the</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">total bandwidth utilization by about 0.2-0.4% mainly due to the display</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">channel. Maybe for other channels we could have better gain.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""></div></div></blockquote><div class=""><br class=""></div><div class="">Thanks for testing this. I’m not too surprised by them given the nature of the data.</div><div class=""><br class=""></div><div class="">Christophe</div><br class=""><blockquote type="cite" class=""><div class=""><div style="" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Frediano</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class=""><br class=""><blockquote type="cite" class=""><blockquote type="cite" class="">On 2 Mar 2017, at 17:53, Snir Sheriber <<a href="mailto:ssheribe@redhat.com" class="">ssheribe@redhat.com</a>> wrote:<br class=""><br class="">This series of patches allows compression of messages over<br class="">selected spice channels<br class=""><br class="">Few notes:<br class=""><br class="">*Currently lz4 stream and regular compression are in use for small and<br class="">large<br class="">messages accordingly. packets are being sent in common msg type that was<br class="">added,<br class="">and it utilize previous compressed message structure.<br class=""><br class="">*The stream compression & decompression dictionary is based on previous<br class="">messages,<br class="">there for messages that has compressed\decompressed in stream mode are<br class="">being saved<br class="">in continuous pre-allocated buffer and a pointer is used for tracking<br class="">current<br class="">position.<br class=""><br class="">*Currently all channels are allowed to be compressed, we might want to<br class="">avoid<br class="">compression for some channels (for good or just in specific cases)<br class="">***please<br class="">notice that usbredir compression was not revert yet so meanwhile i added<br class="">2<br class="">small<br class="">patches to disable its compression caps.<br class=""><br class="">*Multiple clients- basically it should work with more than one client,<br class="">although adding<br class="">compression/decompression to just part of the clients could theoretically<br class="">make it<br class="">worse (good for compression performance testing though)<br class=""><br class="">-If someone has encountered issues with the combination of compression<br class="">and<br class="">other spice features please let me know<br class="">-Suggestions and comments are welcome, especially for improving the<br class="">messages sending :)<br class=""><br class="">Snir.<br class=""><br class="">Spice components:<br class="">server,client,spice-common,spice-protocol<br class=""><br class=""></blockquote></blockquote></blockquote><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Spice-devel mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""><a href="mailto:Spice-devel@lists.freedesktop.org" class="">Spice-devel@lists.freedesktop.org</a></span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""><a href="https://lists.freedesktop.org/mailman/listinfo/spice-devel" class="">https://lists.freedesktop.org/mailman/listinfo/spice-devel</a></span></div></div></blockquote></div><br class=""></div></body></html>