[Libreoffice-bugs] [Bug 144156] New: Slight inaccuracy / deviation in SKEW() with uniform integer sequences for Calc pre-7.2

[bugzilla-daemon at bugs.documentfoundation.org]

https://bugs.documentfoundation.org/show_bug.cgi?id=144156

            Bug ID: 144156
           Summary: Slight inaccuracy / deviation in SKEW() with uniform
                    integer sequences for Calc pre-7.2
           Product: LibreOffice
           Version: 7.0.6.2 release
          Hardware: x86-64 (AMD64)
                OS: Linux (All)
            Status: UNCONFIRMED
          Severity: normal
          Priority: medium
         Component: Calc
          Assignee: libreoffice-bugs at lists.freedesktop.org
          Reporter: stephane.guillou at member.fsf.org

Description:
I am reporting this even though it is fixed in 7.2, because:
- no bug report seem to already exist on the topic, so we might want to have
one in order to link the fix to a commit
- the issue being an inaccuracy in calculation, I thought we might want to have
that fix in 7.1 as well - depending on how disruptive the commit is.

The issue is with the SKEW() function giving a result slightly off 0 when it
should be exactly 0, depending on the size of the uniform integer sequence.

Steps to Reproduce:
1. Open attachment

Actual Results:
Observe that the skewness is correct for some sets of integers but not for
others, e.g. correct all the way to 10, but starts being off when 11 is
included, then back to correct when 19 is included...

Expected Results:
The skewness is 0 for all rows.


Reproducible: Always


User Profile Reset: No



Additional Info:
This is reproducible in:

Version: 7.0.6.2
Build ID: 144abb84a525d8e30c9dbbefa69cbbf2d8d4ae3b
CPU threads: 8; OS: Linux 5.4; UI render: default; VCL: gtk3
Locale: en-AU (en_AU.UTF-8); UI: en-US
Calc: threaded

and:

Version: 7.1.5.2 / LibreOffice Community
Build ID: 85f04e9f809797b8199d13c421bd8a2b025d52b5
CPU threads: 8; OS: Linux 5.4; UI render: default; VCL: gtk3
Locale: en-AU (en_AU.UTF-8); UI: en-US
Calc: threaded

but is NOT reproduced in:

Version: 7.2.0.4 / LibreOffice Community
Build ID: 9a9c6381e3f7a62afc1329bd359cc48accb6435b
CPU threads: 8; OS: Linux 5.4; UI render: default; VCL: gtk3
Locale: en-AU (en_AU.UTF-8); UI: en-US
Calc: threaded

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice-bugs/attachments/20210829/7bcc745c/attachment.htm>