Using libpci for reading PCI vendor/device/etc

[Daniel Drake]

Daniel Drake wrote:
> For the pci.ids.gz run:
> 572kb RSS is used on the heap (entirely private dirty). libc has 260kb 
> of r-xp RSS. The libz maps occupy a further 52kb RSS.

Another data point:

I got curious and decided to look into how zlib's decompression 
(inflation) works, in terms of memory and buffering.

When libpci calls gzopen(), some initialization and buffer/structure 
allocation takes place.

The first time libpci calls gzread(), a sliding inflation window is 
allocated. I think this is 32kb in size.

There are no other allocations in the file read path. gzread() uses a 
16kb buffer allocated at gzopen() time to fread() the gzipped file 
contents into.

When it actually comes to inflating the data from the 16kb buffer, the 
code sometimes directly copies into the output buffer that was supplied 
to gzread(), but sometimes has to include the sliding window as a 
middle-man.

This certainly isn't as pure as the mmap approach, but I have at least 
convinced myself that zlib isn't at all messy with memory buffering, the 
space overhead of zlib inflation is low (48kb), and as zlib makes no 
allocations in the read path, there's no risk of memory fragmentation or 
anything like that.

Additionally, the structures, window and buffers created in the gzopen() 
path are freed on gzclose(). libpci does this soon after it does 
gzopen() (i.e. it doesn't leave the file descriptor around, it only 
needs it once).

Daniel