Re: package / tmp file usage ...

[Attila Szűcs]

Thx Michael, you are right, if an other thread would deflate (to disk) while SvXMLExport create the xml stream (in memory), and they would keep the memory stream under a size limitation, then it would be great.
I seen this producer / consumer pattern like thing in XBufferedThreadedStream .. and it is called in import time from ScXMLImportWrapper. (i do not rechecked it now.. somehow i remember that still wrote the stream into a file)
But in this export case, as i seen, the same thread do both of them, and deflate started only after SvXMLExport finished.

the idea of having ready deflated streams on disk to pack into the zip archive sounds good, i bet in basic cases it is not a problem... im not sure about complex cases, like encription

On Wednesday, May 17, 2023 16:46 BST, Michael Meeks <michael.meeks at collabora.com> wrote:
 
On 17/05/2023 15:42, Attila Szűcs wrote:
> I can see this file on my hard drive, so it is really here.
> i can view its content and it is a half made xml file
> content.xml (the whole file) ~110mb big, but the compressed ods is only
> ~670kb.
> so if it would be stored only in memory, we could avoid a lot of disk
> writing.

Streaming to a file is fine - and in fact good =) -but- I expect that
if we have another thread that deflates this as we do the streaming out
- then we could save quite a chunk of I/O I think and perhaps get the
compression 'for free' (and deflate is far from free unfortunately).

Of course, the idea of having ready deflated streams on disk to pack
into the zip archive may not fit perfectly into whatever conceptual
model but ... =)

> i can understand that in most cases this extra file is not a big
> problem... and when the file is big enought to be a problem, than maybe
> the memory could be a bigger problem.. :)

=)

Michael.

--
michael.meeks at collabora.com <><, GM Collabora Productivity
Hangout: mejmeeks at gmail.com, Skype: mmeeks
(M) +44 7795 666 147 - timezone usually UK / Europe

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice/attachments/20230517/00719d6e/attachment.htm>