[PATCH] fdo#51083 calc: fix mathematical errors in BinomDist, B, Cri...

[Werner Körner (via_Code_Review)]

Hi,

I have submitted a patch for review:

    https://gerrit.libreoffice.org/1213

To pull it, you can do:

    git pull ssh://gerrit.libreoffice.org:29418/core refs/changes/13/1213/1

fdo#51083 calc: fix mathematical errors in BinomDist, B, CritBinom and NegBinomDist

In these functions the precision is bad when pow(1-p,n) is a denormalized number.
If pow(1-p,n) is 0 an alternative using pow(p,n) is used.
Changed to select the calculation method depending on size of p.

Change-Id: I00a882799ec8790a6232c00cf3f9b23d8f4f03c0
---
M sc/source/core/tool/interpr3.cxx
1 file changed, 90 insertions(+), 60 deletions(-)


--
To view, visit https://gerrit.libreoffice.org/1213
To unsubscribe, visit https://gerrit.libreoffice.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I00a882799ec8790a6232c00cf3f9b23d8f4f03c0
Gerrit-PatchSet: 1
Gerrit-Project: core
Gerrit-Branch: master
Gerrit-Owner: Werner Körner <wk661lo at gmail.com>