[PATCH libreoffice-3-6] prevent out of bounds vector access in ooo62493-1.xls recalc...

[Eike Rathke (via Code Review)]

Hi,

I have submitted a patch for review:

    https://gerrit.libreoffice.org/3339

To pull it, you can do:

    git pull ssh://gerrit.libreoffice.org:29418/core refs/changes/39/3339/1

prevent out of bounds vector access in ooo62493-1.xls recalculation

The document https://issues.apache.org/ooo/attachment.cgi?id=34418
attached to https://issues.apache.org/ooo/show_bug.cgi?id=62493 crashes
in a dbgutil build due to an invalid subtotal count that leads to an out
of bounds access in a vector, in this case with offset -1

This fixes the symptom, not the cause why the subtotal count would be
wrong.

Change-Id: If96c2eadba8753ea3e3166db0a81441818c89b2d
(cherry picked from commit 58885eda4f15913fc46243dab726632f5904aab5)
---
M sc/source/core/data/dpoutput.cxx
1 file changed, 2 insertions(+), 0 deletions(-)



diff --git a/sc/source/core/data/dpoutput.cxx b/sc/source/core/data/dpoutput.cxx
index 2b86a69..ea43c5f 100644
--- a/sc/source/core/data/dpoutput.cxx
+++ b/sc/source/core/data/dpoutput.cxx
@@ -1782,6 +1782,8 @@
             // if a subtotal is included, clear the result flag for the columns/rows that the subtotal includes
             sal_Int32 nStart = nPos - rSubtotal[nPos];
             OSL_ENSURE( nStart >= 0, "invalid subtotal count" );
+            if (nStart < 0)
+                nStart = 0;
 
             for (sal_Int32 nPrev = nStart; nPrev < nPos; nPrev++)
                 rResult[nPrev] = false;

-- 
To view, visit https://gerrit.libreoffice.org/3339
To unsubscribe, visit https://gerrit.libreoffice.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If96c2eadba8753ea3e3166db0a81441818c89b2d
Gerrit-PatchSet: 1
Gerrit-Project: core
Gerrit-Branch: libreoffice-3-6
Gerrit-Owner: Eike Rathke <erack at redhat.com>