[Libreoffice-commits] core.git: Changes to 'feature/calc-parallel'

Fri Oct 13 21:12:40 UTC 2017

New branch 'feature/calc-parallel' available with the following commits:
commit 64cb548a14c172bde6128532240e4f5a211df347
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Oct 4 22:55:19 2017 +0300

    Avoid unused private field warning in the NDEBUG case
    
    Change-Id: I5e37b9a8325af35a15c01409f9eaa2f92459cc28

commit fdc97d82770e4c4821c7ad194127e649f3318f6b
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Oct 4 16:55:59 2017 +0300

    Move ScDocument::GetNonThreadedContext() inline
    
    Did not have any impact on performance, though.
    
    Change-Id: I7e769b4a74e0ff9e0aabfb7e291fc4b987441954

commit 699b9b5ba41604d30e85c595ea8e8ef2573cc4bc
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Oct 4 13:11:10 2017 +0300

    Display the threaded calculation state in Help:About
    
    Change-Id: I299e555392bb4b09325ad2c92f843b1e12ee4d31

commit c269dfb87a3d5a08450fce42d4947692927ffba1
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Oct 4 12:40:20 2017 +0300

    Make threaded calculation the default (when OpenCL is not used)
    
    Introduce a configuration setting to turn it off. For now, can also be
    turned off with the environment variable SC_NO_THREADED_CALCULATION,
    but that is probably not something we want to keep or guarantee
    staility of. (LO looks at way too many environment variables already.)
    
    Change-Id: I469cde259eda72cc2d630814a25f707f1210b0ab

commit e106b31f27ccd31c716935597dbc23476fbd3a37
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Oct 4 10:55:13 2017 +0300

    Need more ScInterpreterContexts
    
    Change-Id: I1dd679156661bb5cb025ca6cb46d19783524d5a4

commit 90072f2424f4d118a37611dd6791fca0a3ed0c60
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Oct 4 08:48:05 2017 +0300

    -Werror,-Wunused-parameter
    
    Change-Id: If10c6a58f5b6f196f3644f6c592dd6d1dc0d860c

commit cc5e75eb79c51b4042764a3ce90b62f2a4904805
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Oct 4 08:47:02 2017 +0300

    -Werror,-Wsign-compare
    
    Change-Id: Ide03e0ae1fe97e1a09a767908a981a1e803a3474

commit 319e69afdc2d4ec8ad60b030d6692361650e683d
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Oct 4 00:12:31 2017 +0300

    Introduce ScInterpreterContext
    
    When calculating a formula group in multiple threads in parallel, we
    need to use separate SvNumberFormatters in the threads. Possibly later
    also other things that need to be thread-local can be handled through
    the ScInterpreterContext.
    
    Why handle some thread-local things through the
    ScDocument::maNonThreaded and ScDocument::maThreadSpecific mechanism,
    and others through this ScInterpreterContext? Good question.
    
    Change-Id: I372e5fbd9a19785f55f0faf4a4bedc5fc1ef3e03

commit dc6dbd1087f048d594d7cecc03a1319a35b88a39
Author: Tor Lillqvist <tml at collabora.com>
Date:   Fri Sep 29 17:43:24 2017 +0300

    Disable formula group threading for macros and table ops
    
    Those are likely highly problematic to do in parallel.
    
    Change-Id: I50cc32eb72f6b7951d247ecd787b2942cd7d9163

commit b05335a6227cf18eaef3d1ebfa832ffcd3f13e9d
Author: Tor Lillqvist <tml at collabora.com>
Date:   Fri Sep 29 17:01:03 2017 +0300

    Move nInterpreterTableOpLevel back to ScDocument
    
    Change-Id: I4de0051d9fa5de9147954c6021d47076145a3e59

commit 38f09c94cfe877be2f60085df1745a97e78bb916
Author: Tor Lillqvist <tml at collabora.com>
Date:   Fri Sep 29 16:00:18 2017 +0300

    Move nMacroInterpretLevel back to ScDocument
    
    Change-Id: I48748434c845af963af160f8bbd75e4ab7ce95bd

commit 146e13ec24629e2cd4e31dbcfb78e70c225263ee
Author: Tor Lillqvist <tml at collabora.com>
Date:   Fri Sep 29 14:26:29 2017 +0300

    Move nInterpretLevel back to ScDocument
    
    Move the calls to increment and decrement it out of InterpretTail(),
    to those call sites that aren't reached during parallelized
    calculations.
    
    Change-Id: Ie1bd03dd62aea5f6c71c383df21afff29391dade

commit d645f4b603e1eabfc37963e55aa3c306dea4dbd4
Author: Tor Lillqvist <tml at collabora.com>
Date:   Fri Sep 29 11:04:17 2017 +0300

    Re-work how the thread-specific data in ScDocument works
    
    We can use normal thread_local data for it as a thread can only be
    acting on one ScDocument in parallelized formula group calculation
    anyway. Use separate data instance for the non-threaded data, and when
    a thread starts, copy that to the thread-specific data.
    
    Change-Id: I3e58320a728d1c5639a8a078748b3d4d7a451b25

commit 10504ebd8e9e5984f242941a47ccff6ad96e17bf
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Aug 16 13:01:38 2017 +0300

    Move some of the fields in ScDocument into a thread-local struct
    
    This is part of the work to enable having several ScInterpreters
    working on the same object in parallel. Perhaps a more thorugh
    re-factoring would be better, maybe these fields should not be part of
    ScDocument at all, but part of ScInterpreter?
    
    Without this change the parallelism unit test occasionally fails.
    
    Change-Id: I70500b4d2b513e134d22b4b98c84a3ec00dad291

commit 9d147d2e9f19959a4aa799802e16cd505471721b
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Sep 20 15:14:21 2017 +0300

    Guard pFormatExchangeList from mutation during threaded calculation
    
    Change-Id: I645630fae8a5fcd09d8af4c4ec9aac94edbd2f25

commit d270208ffd1d580336d35f7db710dfd7ea83a09e
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Sep 20 15:05:24 2017 +0300

    Guard pValidationList from mutation
    
    Change-Id: I9bd889523b5e5374776d94534ae2171e7a5a635d

commit 849b93952b81135fca3e674372e6c96ccd5c060c
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Sep 20 13:56:06 2017 +0300

    Guard pColorList from mutation
    
    Change-Id: I53a13bebf018c15a1d94c15adbfa7b68632c77b0

commit 428334b024f81a753022563282dc60eb7cb09bfd
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Sep 20 12:54:09 2017 +0300

    Use explicit flag to indicate threaded formula group calculation being active
    
    Don't just check whether we are in the "main" thread, as that is not
    true in the tiled rendering unit test, for instance.
    
    Change-Id: I4f3ca1a1b1cb3c92856104fffe8100f18cd7b66c

commit b3917222b5db99b9dbc410dd5eaa2610f6e67cc5
Author: Tor Lillqvist <tml at collabora.com>
Date:   Thu Sep 14 15:06:14 2017 +0300

    Assert that we don't call MaybeInterpret() in a threaded calculation
    
    Change-Id: Ied03498333ee531e6e31e05fa2734056673364f2

commit 4d5c8568c9953ffd9fdbda1b30e3ad83c16cab8f
Author: Tor Lillqvist <tml at collabora.com>
Date:   Tue Sep 12 12:49:53 2017 +0300

    Guard mpDrawLayer from mutation
    
    Change-Id: I8b9a4680691d47732be2dd159ffc5d56d12a7a95

commit 1e1fa0038dd1f634493283601b84a7fd72c2af06
Author: Tor Lillqvist <tml at collabora.com>
Date:   Tue Sep 12 11:07:34 2017 +0300

    Guard mpShell from mutation
    
    Change-Id: I7c12301f5a435dfca733b5f87b4b083f98a5292a

commit dec7da418ce2af864de8e2fcf7e354339712bea3
Author: Tor Lillqvist <tml at collabora.com>
Date:   Mon Sep 11 15:30:10 2017 +0300

    Guard mpNoteEngine from mutation
    
    Change-Id: I174bca167379b96083476a394d8ff5d7f00a8021

commit e531300d425a220ad091978c6a704842318e2251
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Aug 30 15:51:02 2017 +0300

    Set fill-column in Emacs mode line
    
    Change-Id: I102b02e93ff06f3e53b9a1194277f5a6fae9f1ed

commit 81ae580b08e847d791fc58ddbba31eb2b0416d4a
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Aug 30 09:28:59 2017 +0300

    More work on the ScDependantsCalculator
    
    Might now actually do what it should.
    
    Change-Id: Ibf38560a37910924c4fade79cbbf4553d6dbd077

commit 617393664c0a8fe27137c3c6051c6a1c63ac2ef1
Author: Tor Lillqvist <tml at collabora.com>
Date:   Tue Aug 29 22:22:19 2017 +0300

    Use a static const bool when checking an env var for existence

commit 65e7ec212a04d21b8136b73fdb576af25e7c56ad
Author: Tor Lillqvist <tml at collabora.com>
Date:   Tue Aug 29 22:11:06 2017 +0300

    Make formula group weight take number of cells referenced into account
    
    And not just the number of rows in the group. This means that even
    relatively short formula groups that calculate over large ranges of
    cells will be eligible for parallelized calculation. The weight of a
    formula group is the number of rows in the group times the weight of
    the formula.
    
    The weight of a formula is for now the number of cells referenced by
    all cell ranges in the formula, divided by 10. For instance, the
    weight of SUM(A1:B100) would be 20. If no cell ranges are used, the
    formula's weight is one.
    
    Change-Id: Ib77e403961d8f487d580eea6b901fa4f5e4102b0

commit 47204077f5013d1f31241b2cca135bd6f813fc43
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Aug 23 16:33:50 2017 +0300

    Check whether it is the multi-threaded case explicitly
    
    Change-Id: I96bed3ebd90af9a8ab2c17b678384cf340ba2802

commit 3cc30455f0311b08c34c49797ae252d3409ddc43
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Aug 23 14:05:13 2017 +0300

    Bin some YAGNI code
    
    We don't do anything at the moment to check whether a formula group
    calculation thread "failed", I don't even know what it actually would
    mean. So just bin that code for now.
    
    Change-Id: Ie4b15a3c20169d08d338e255a71a139913528a16

commit 3135a1cf9cc71089d617dd5ab098f73f745f6ce3
Author: Tor Lillqvist <tml at collabora.com>
Date:   Tue Aug 22 14:26:29 2017 +0300

    Move formula tree manipulation out of multi-threaded parts of code
    
    Change-Id: I2f7e6fb747b6a74172a81f9db9bc210ef6a27342

commit cdfc83d18e6357fcb60aa0a0e6095de103b41679
Author: Tor Lillqvist <tml at collabora.com>
Date:   Fri Aug 18 16:46:44 2017 +0300

    Always create the DocumentLinkManager, avoids a mutation assertion failure
    
    Change-Id: If968792c4b98afbe0cc956e9cab0bcfde47bf1c1

commit 3418fd256f50b1ffc40831de6e2e780d12253569
Author: Tor Lillqvist <tml at collabora.com>
Date:   Fri Sep 22 18:55:37 2017 +0300

    Add "mutation guard" API for ScDocument
    
    USe by adding a ScMutationGuard object to a section of code that
    shouldn't be run during threaded calculation of a formula group in any
    of the calculation threads. There are currently several "classes" of
    mutation as bits of ScMutationGuardFlags, althouh I am not sure
    whether that will be useful, so far I use just the one same (CORE) in
    all cases.
    
    Currently implemented using mutexes. Possibly a simple bool field in
    ScDocument would be enough, the multiple flags and mutexes might be
    over-complicating it? Maybe I misunderstood what I want.
    
    Add such mutation guards for some fields of ScDocument.
    
    Change-Id: If2a8223c49d36143f2984e0449798271f2b6440d

commit 0345e373772d5f7f425347990ff02a98ac6ca7ad
Author: Tor Lillqvist <tml at collabora.com>
Date:   Wed Aug 16 12:21:53 2017 +0300

    First steps for Calc parallelism
    
    For now, formula group calculations are done in parallel threads when
    1) OpenCL is not used, and 2) the environment variable
    CPU_THREADED_CALCULATION is set. This commit is a surely broken first
    step and does not actually work that well at all.
    
    Change-Id: Ia7e5019703ba89bff0695faef0f7504765061149