[Libreoffice-commits] core.git: sc/source

[Mike Kaganski (via logerrit)]

 sc/source/core/data/table4.cxx |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

New commits:
commit d5ebe7c3089ab9f4d3fe0707169fc1ce024cdb70
Author:     Mike Kaganski <mike.kaganski at collabora.com>
AuthorDate: Mon Sep 21 10:21:24 2020 +0200
Commit:     Mike Kaganski <mike.kaganski at collabora.com>
CommitDate: Mon Sep 21 11:12:07 2020 +0200

    tdf#129606: Round the mean of the two subtractions
    
    This improves handling of cases when the two subtraction methods give
    these values:
    
    c = -0.10000000000000053
    d = -0.10000000000000006
    
    Rounding c would give -0.10000000000000100, while rounding the mean
    gives -0.10000000000000001.
    
    Change-Id: I9805bc78bd1dec8ed77b660df1e366d36348f68b
    Reviewed-on: https://gerrit.libreoffice.org/c/core/+/103048
    Tested-by: Jenkins
    Reviewed-by: Mike Kaganski <mike.kaganski at collabora.com>

diff --git a/sc/source/core/data/table4.cxx b/sc/source/core/data/table4.cxx
index 9403d1b913a7..09a17546c002 100644
--- a/sc/source/core/data/table4.cxx
+++ b/sc/source/core/data/table4.cxx
@@ -208,7 +208,8 @@ double approxDiff( double a, double b )
     // tdf#129606: Limit precision to the 16th significant digit of the least precise argument.
     // Cf. mnMaxGeneralPrecision in sc/source/core/data/column3.cxx.
     const int nExpArg = static_cast<int>(floor(log10(std::max(aa, ab)))) - 15;
-    return rtl::math::round(c, -std::max(nExp, nExpArg));
+    // Round the mean of the two subtractions
+    return rtl::math::round((c + d) / 2, -std::max(nExp, nExpArg));
 }
 }