[Libreoffice-commits] core.git: Branch 'libreoffice-7-1' - sal/rtl

[Eike Rathke (via logerrit)]

 sal/rtl/math.cxx |   14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

New commits:
commit 73d98236ea83296018e6da30d0d7ec0219313776
Author:     Eike Rathke <erack at redhat.com>
AuthorDate: Thu Dec 3 18:18:23 2020 +0100
Commit:     Eike Rathke <erack at redhat.com>
CommitDate: Fri Dec 4 00:52:54 2020 +0100

    Better accuracy in rtl_math_approxValue(), tdf#138360 related
    
    Similar to commit 5abb1890ffafe5a2212076208a1c6e226f1ffa4e for
    rtl_math_round() use the reciprocal value in an inverse operation
    for negative exponents to not use the inexact 1e-1
    0.10000000000000001 and so on factors.
    
    Change-Id: I05b852e06f2c31d6e0ce622b07277a81a5690833
    Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107172
    Reviewed-by: Eike Rathke <erack at redhat.com>
    Tested-by: Jenkins
    (cherry picked from commit 520949f17a91c531ea0c8b3856ffcf3c7ac8a3b2)
    Reviewed-on: https://gerrit.libreoffice.org/c/core/+/107195

diff --git a/sal/rtl/math.cxx b/sal/rtl/math.cxx
index a296927635bf..46a3e925b95b 100644
--- a/sal/rtl/math.cxx
+++ b/sal/rtl/math.cxx
@@ -1299,16 +1299,24 @@ double SAL_CALL rtl_math_approxValue( double fValue ) SAL_THROW_EXTERN_C()
 
     int nExp = static_cast< int >(floor(log10(fValue)));
     nExp = 14 - nExp;
-    double fExpValue = getN10Exp(nExp);
+    double fExpValue = getN10Exp(abs(nExp));
+
+    if (nExp < 0)
+        fValue /= fExpValue;
+    else
+        fValue *= fExpValue;
 
-    fValue *= fExpValue;
     // If the original value was near DBL_MIN we got an overflow. Restore and
     // bail out.
     if (!std::isfinite(fValue))
         return fOrigValue;
 
     fValue = std::round(fValue);
-    fValue /= fExpValue;
+
+    if (nExp < 0)
+        fValue *= fExpValue;
+    else
+        fValue /= fExpValue;
 
     // If the original value was near DBL_MAX we got an overflow. Restore and
     // bail out.