Patch to huge memory consumption in LO Calc

[William Bonnet]

Hello Kohei,

Thanks for your feedback. I'll do my best to be C++03 compliant for my 
next patches . So far i haven't found an equivalent to shrink_to_fit, 
but i think i have another to do it... i'll given it a look during this 
week end.

Meanwhile, i think we can improve things a little with the patch i 
attached to this email.

I had a look to the insert_empty_impl and set_new_block_to_middle method 
and i noticed that the way element_block_func::assign_values_from_block 
is called can be optimized. Especially when working with very large 
documents.

What is currently done after the new block is created is (in short) :

1/ "copy right part of the old block to the new one"
2/ "shrink left part"

When working with huge document (mine is ~100 000 rows and 100 columns), 
it happends very often to insert at a position at the very beginning. In 
such case, we will have to copy something like ~100 000 items to the new 
block and resize the old one to something like less than 10.

What i suggest is to add a test on left vs right block size, and then 
copy the smallest one instead of the right one.

Now the smaller block content is copyied to the new block, and the 
bigger on is either resized or items are poped from front. If needed 
pointer are swapped. This save some cycles when you work with huge sheets :)

I made some timing measurements, still using my 100 000 x 100 sheet and 
subtotal, average execution time of the ScTable::InsertRow method goes 
from 0.607 second down to 0.289 seconds (Phenom II box quad core and 
8gigs of ram).

If you agree with the solution i proposed, i'll check if other method 
need the same kind of optimization and i will submit a patch.

Cheers,

-- 
William BONNET
Directeur Technique / CTO LINAGORA
Linagora 80 rue Roque de Fillol / Puteaux 92800 F
Tél. +33 (0)810 251 251
GSM +33 (0)689 376 977
Twitter @wbonnet

http://www.linagora.com/ | http://www.08000linux.com/

Découvrez OBM, La messagerie Libre : http://www.obm.org/

La présente transmission contient des informations confidentielles appartenant à Linagora, exclusivement destinées au(x) destinataire(s) identifié(s) ci-dessus. Si vous n'en faites pas partie, toute reproduction, distribution ou divulgation de tout ou partie des informations de cette transmission, ou toute action effectuée sur la base de celles-ci vous sont formellement interdites. Si vous avez reçu cette transmission par erreur, nous vous remercions de nous en avertir et de la détruire de votre système d'information.

The present transmission contains privileged and confidential information belonging to Linagora, exclusively intended for the recipient(s) thereabove identified. If you are not one of these aforementioned recipients, any reproduction, distribution, disclosure of said information in whole or in part, as well as any action undertaken on the basis of said information are strictly prohibited. If you received the present transmission by mistake, please inform us and destroy it from your messenging and information systems.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: multi_type_vector_def.inl.patch
Type: text/x-patch
Size: 5601 bytes
Desc: not available
URL: <http://lists.freedesktop.org/archives/libreoffice/attachments/20140529/6a8ff886/attachment.bin>