[Libreoffice] [PATCH] Fix for OO Basic Million-to-one bug #i76852#

[John LeMoyne Castle]

Adds '=' to edge test in sbxscan.cxx ImpCvtNum so exponent prints and 1
million is 1e+6 instead of just 1 for singles.  
Same one char change does similar fix for OO Basic doubles and 1e+14. 

When I saw #i76852# -- http://qa.openoffice.org/issues/show_bug.cgi?id=76852
-- describe how and when OOBasic prints as single with value 1 000 000  as 1
(with no million) I thought right off: edge case ==> missing equals.  

Using OpenGrok, I drilled in through the basic interface for the CStr()
function.  CStr() is implicitly called by print in the OOBasic interpreter
when it needs to print a number.  Straight away found these two lines in
sbxscan.cxx ImpCvtNum method: 

    double dMaxNumWithoutExp = (nPrec == 6) ? 1E6 : 1E14;
    myftoa( nNum, p, nPrec,( nNum &&( nNum < 1E-1 || nNum >
dMaxNumWithoutExp ) ) ? 4:0,

The top line gives the two problem cases identified in #i76852 .  The double
type fails at the other number 100 000 000 000 000 = 1e+14.

I added the equals to make the boolean clause true at 1 000 000 - (as: nNum
>= dMaxNumWithoutExp )  and OO Basic prints 1e+6 instead of just the one. 
Built and ran and now CSng(CStr(1000000)) = 1000000.  Relative error down
from ~1 (utter fail) to < 1 ppm (as good as it gets for single). Absolute
error down from 99999 to <1 - a million to one reduction. 

Thought it was too easy and so reviewed the code to understand what that
parameter exponent width = ( ( nNum && ( nNum < 1E-1 || nNum >
dMaxNumWithoutExp ) ) ? 4:0 ) was all about -- for numbers w/ abs value <1
or abs value >= dMaxNumWithoutExp a non zero exponent width of four is
passed to myftoa.  In myftoa if zero width then no exp prints : hence 1
million became 1 (no million) without the equals.  Grokked *some* of parser
logic in myftoa.  

Wanted to check that nearby numbers (within round/trunc for single precision
limit)  weren't messed up so wrote macro in attached spreadsheet.  In ods is
macro M2one - does a check on all the numbers in the A column of singles
sheet and generates (clobbers) values into cols B,C,D... Tested against OOo
3.2 (fail). Did not complete the double test at 1e14 in 2nd sheet+macro, but
tested doubles with previous print macro and tested single and double
against LibO dev yesterday (fail at 1e+14).  If you try it and need to
enable macros then re-load the worksheet after your settings change.  

Seems that the OO issue zilla is a good source of bugs to fix... 

Have fun, 
LeMoyne
JLCastle

http://nabble.documentfoundation.org/file/n1839430/0001-OO-Basic-Million-to-One-bug-i76852-fixed.patch
0001-OO-Basic-Million-to-One-bug-i76852-fixed.patch 

http://nabble.documentfoundation.org/file/n1839430/Million-to-one_SingleOnly_OOo3.2.ods
Million-to-one_SingleOnly_OOo3.2.ods 
-- 
View this message in context: http://nabble.documentfoundation.org/PATCH-Fix-for-OO-Basic-Million-to-one-bug-i76852-tp1839430p1839430.html
Sent from the Dev mailing list archive at Nabble.com.