[Mesa-dev] [PATCH 2/4] util/disk_cache: compress individual cache entries

[Grigori Goronzy]

On 2017-03-02 10:08, Timothy Arceri wrote:
> On 02/03/17 18:45, Tobias Droste wrote:
>> Hi Timothy,
>> 
>> if you plan to support multiple compression algorithms, shouldn't 
>> "struct
>> cache_entry_file_data" contain some info about what compression 
>> algorithm was
>> used to compress the data? Or is this already there and I missed it?
> 
> I don't plan to support more than one. I'm just saying it's a
> possibility for the future, depending on further analysis and
> requirements from different hardware. But right now I would just like
> to land zlib support so we have a baseline to work from.
> 

Like outlined on IRC, for the time being can you reconsider the 
compression level, though?

The cache currently quite noticeably affects shader loading time on 
first hit, i.e. when the cache is cold. Cache I/O and compression is 
right now done synchronously, so it even happens without compression on 
a system with fast SSD. Here's a summary of the numbers I gathered from 
the initial loading screens of DE:MD on an Athlon X4 860k:

No Cache                                 215 sec

Cold Cache zlib BEST_COMPRESSION         285 sec
Warm Cache zlib BEST_COMPRESSION         33 sec

Cold Cache zlib BEST_SPEED               264 sec
Warm Cache zlib BEST_SPEED               33 sec

Cold Cache no compression                266 sec
Warm Cache no compression                34 sec

The total cache size for that game is 48 MiB with BEST_COMPRESSION, 56 
MiB with BEST_SPEED and 170 MiB with no compression.

What In conclude from these numbers is that

a) the cache works really well! A warmed cache cuts down loading time by 
an order of magnitude,
b) compression does a good job of reducing the cache size (which 
probably helps with traditional spinning disk HDDs, and when disk space 
is limited),
c) decompression always seems Fast Enough On My Computer™.

However, I also notice that BEST_COMPRESSION doesn't really affect the 
total size of the cache files much compared to BEST_SPEED, while it does 
noticeably increase the cold cache shader loading time. So in the end, 
BEST_SPEED might be a better compromise, particularly for systems with a 
slow CPU.

Apart from that, consider the series
Reviewed-by: Grigori Goronzy <greg at chown.ath.cx>

Best regards
Grigori

>> 
>> Am Donnerstag, 2. März 2017, 03:20:05 CET schrieb Matt Turner:
>>> On Wed, Mar 1, 2017 at 2:19 PM, Timothy Arceri 
>>> <tarceri at itsqueeze.com>
>> wrote:
>>>> IMO we should go with zlib and people can provide future patches 
>>>> with
>>>> justifications/stats for using a different library over zlib just 
>>>> like we
>>>> do for any other performance based patch.
>>> 
>>> Yes, agreed. "Which compression should we use?" is one of the easiest
>>> bikesheds to paint.
>>> _______________________________________________
>>> mesa-dev mailing list
>>> mesa-dev at lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev