[Libreoffice] [PATCH] Fix for OO Basic Million-to-one bug #i76852#
John LeMoyne Castle
jlc at mail2lee.com
Wed Nov 3 21:47:56 PDT 2010
Adds '=' to edge test in sbxscan.cxx ImpCvtNum so exponent prints and 1
million is 1e+6 instead of just 1 for singles.
Same one char change does similar fix for OO Basic doubles and 1e+14.
When I saw #i76852# -- http://qa.openoffice.org/issues/show_bug.cgi?id=76852
-- describe how and when OOBasic prints as single with value 1 000 000 as 1
(with no million) I thought right off: edge case ==> missing equals.
Using OpenGrok, I drilled in through the basic interface for the CStr()
function. CStr() is implicitly called by print in the OOBasic interpreter
when it needs to print a number. Straight away found these two lines in
sbxscan.cxx ImpCvtNum method:
double dMaxNumWithoutExp = (nPrec == 6) ? 1E6 : 1E14;
myftoa( nNum, p, nPrec,( nNum &&( nNum < 1E-1 || nNum >
dMaxNumWithoutExp ) ) ? 4:0,
The top line gives the two problem cases identified in #i76852 . The double
type fails at the other number 100 000 000 000 000 = 1e+14.
I added the equals to make the boolean clause true at 1 000 000 - (as: nNum
>= dMaxNumWithoutExp ) and OO Basic prints 1e+6 instead of just the one.
Built and ran and now CSng(CStr(1000000)) = 1000000. Relative error down
from ~1 (utter fail) to < 1 ppm (as good as it gets for single). Absolute
error down from 99999 to <1 - a million to one reduction.
Thought it was too easy and so reviewed the code to understand what that
parameter exponent width = ( ( nNum && ( nNum < 1E-1 || nNum >
dMaxNumWithoutExp ) ) ? 4:0 ) was all about -- for numbers w/ abs value <1
or abs value >= dMaxNumWithoutExp a non zero exponent width of four is
passed to myftoa. In myftoa if zero width then no exp prints : hence 1
million became 1 (no million) without the equals. Grokked *some* of parser
logic in myftoa.
Wanted to check that nearby numbers (within round/trunc for single precision
limit) weren't messed up so wrote macro in attached spreadsheet. In ods is
macro M2one - does a check on all the numbers in the A column of singles
sheet and generates (clobbers) values into cols B,C,D... Tested against OOo
3.2 (fail). Did not complete the double test at 1e14 in 2nd sheet+macro, but
tested doubles with previous print macro and tested single and double
against LibO dev yesterday (fail at 1e+14). If you try it and need to
enable macros then re-load the worksheet after your settings change.
Seems that the OO issue zilla is a good source of bugs to fix...
View this message in context: http://nabble.documentfoundation.org/PATCH-Fix-for-OO-Basic-Million-to-one-bug-i76852-tp1839430p1839430.html
Sent from the Dev mailing list archive at Nabble.com.
More information about the LibreOffice