Need help with temporary files

[Caolán McNamara]

On Wed, 2015-07-22 at 13:15 +0200, Noel Grandin wrote:
> 
> On 2015-07-22 01:07 PM, Caolán McNamara wrote:
> > On Wed, 2015-07-22 at 10:43 +0100, Caolán McNamara wrote:
> >> On Wed, 2015-07-22 at 10:28 +0100, Caolán McNamara wrote:
> >>> On Tue, 2015-07-21 at 22:41 +0200, Matúš Kukan wrote:
> >>>> Hi there,
> >>>>
> >>>> I am working on a bug around saving big file in Writer:
> >>>> https://bugs.documentfoundation.org/show_bug.cgi?id=88314
> >>>>
> >> E_MFILE, too many open files, so the problem is a file handle leak.
> >
> > See https://gerrit.libreoffice.org/#/c/17289/ for a possible solution.
> > That odt has > 14k files in it and in parallel deflate mode each one
> > gets a separate ZipOutputEntry which all exist at the same time until
> > the threads are completed. Each ZipOutputEntry has an open temp file so
> > it runs out of file handles.
> >
> 
> We should be using some kind of task-manager/thread-pool to limit the number of active threads to something reasonable.
> Running more than approx no_cores * 3 threads is going to __reduce__ performance.

That's already the case, there are only no_cores active threads at a
time, its just that the final .zip is stitched up from 14k Entries (each
with an open stream) at the end of the process. While in non-thread mode
there is each entry is created, processed and disposed of serially so
there's only one entry alive at a time.

C.