[Mesa-dev] [PATCH 2/4] util/disk_cache: compress individual cache entries

[Marek Olšák]

It should be very easy to move the compression and I/O to another thread
using gallium/util/u_queue. It's a perfect match for this kind of thing.

Marek

On Mar 2, 2017 11:12 PM, "Grigori Goronzy" <greg at chown.ath.cx> wrote:

> On 2017-03-02 10:08, Timothy Arceri wrote:
>
>> On 02/03/17 18:45, Tobias Droste wrote:
>>
>>> Hi Timothy,
>>>
>>> if you plan to support multiple compression algorithms, shouldn't "struct
>>> cache_entry_file_data" contain some info about what compression
>>> algorithm was
>>> used to compress the data? Or is this already there and I missed it?
>>>
>>
>> I don't plan to support more than one. I'm just saying it's a
>> possibility for the future, depending on further analysis and
>> requirements from different hardware. But right now I would just like
>> to land zlib support so we have a baseline to work from.
>>
>>
> Like outlined on IRC, for the time being can you reconsider the
> compression level, though?
>
> The cache currently quite noticeably affects shader loading time on first
> hit, i.e. when the cache is cold. Cache I/O and compression is right now
> done synchronously, so it even happens without compression on a system with
> fast SSD. Here's a summary of the numbers I gathered from the initial
> loading screens of DE:MD on an Athlon X4 860k:
>
> No Cache                                 215 sec
>
> Cold Cache zlib BEST_COMPRESSION         285 sec
> Warm Cache zlib BEST_COMPRESSION         33 sec
>
> Cold Cache zlib BEST_SPEED               264 sec
> Warm Cache zlib BEST_SPEED               33 sec
>
> Cold Cache no compression                266 sec
> Warm Cache no compression                34 sec
>
> The total cache size for that game is 48 MiB with BEST_COMPRESSION, 56 MiB
> with BEST_SPEED and 170 MiB with no compression.
>
> What In conclude from these numbers is that
>
> a) the cache works really well! A warmed cache cuts down loading time by
> an order of magnitude,
> b) compression does a good job of reducing the cache size (which probably
> helps with traditional spinning disk HDDs, and when disk space is limited),
> c) decompression always seems Fast Enough On My Computer™.
>
> However, I also notice that BEST_COMPRESSION doesn't really affect the
> total size of the cache files much compared to BEST_SPEED, while it does
> noticeably increase the cold cache shader loading time. So in the end,
> BEST_SPEED might be a better compromise, particularly for systems with a
> slow CPU.
>
> Apart from that, consider the series
> Reviewed-by: Grigori Goronzy <greg at chown.ath.cx>
>
> Best regards
> Grigori
>
>
>>> Am Donnerstag, 2. März 2017, 03:20:05 CET schrieb Matt Turner:
>>>
>>>> On Wed, Mar 1, 2017 at 2:19 PM, Timothy Arceri <tarceri at itsqueeze.com>
>>>>
>>> wrote:
>>>
>>>> IMO we should go with zlib and people can provide future patches with
>>>>> justifications/stats for using a different library over zlib just like
>>>>> we
>>>>> do for any other performance based patch.
>>>>>
>>>>
>>>> Yes, agreed. "Which compression should we use?" is one of the easiest
>>>> bikesheds to paint.
>>>> _______________________________________________
>>>> mesa-dev mailing list
>>>> mesa-dev at lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
>>>>
>>> _______________________________________________
>> mesa-dev mailing list
>> mesa-dev at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
>>
> _______________________________________________
> mesa-dev mailing list
> mesa-dev at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20170304/040146d6/attachment.html>