[Libreoffice-bugs] [Bug 107769] New: spell checking should normalize data first

[bugzilla-daemon at bugs.documentfoundation.org]

https://bugs.documentfoundation.org/show_bug.cgi?id=107769

            Bug ID: 107769
           Summary: spell checking should normalize data first
           Product: LibreOffice
           Version: 5.4.0.0.alpha1+ Master
          Hardware: All
                OS: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: medium
         Component: Linguistic
          Assignee: libreoffice-bugs at lists.freedesktop.org
          Reporter: martin_hosken at sil.org

Words to be spell checked should be converted to NFKC first so that spell
checking dictionaries don't need to hold all forms (NFD, NFC, mixed) of a word.

I'm going to sketch my thoughts on how to do it here in case I can't get back
to the bug for a while. Anyone want to take it further?

In SpellChecker::GetSpellFailure in lingucomponent/source/spell/sspellimpl.cxx,
rather than doing a poor man's hand created NFK into nWord, start with an nWord
created something like:

icu::UnicodeString rIn(reinterpret_case<const UChar *>(rWord.getStr()),
rWord.getLength());
icu::UnicodeString normal;
UErrorCode rCode;
icu::Normalizer(rIn, UNORM_NFKC, normal, rCode);
OUString nWord(U_SUCCESS(rCode) ? OUString(reinterpret_case<Sal_Unicode
*>(normal.getBuffer()), normal.length()) : OUString());

then use nWord instead of rWord for the rest of the function.

Need to find a test for this.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/libreoffice-bugs/attachments/20170511/31225f9c/attachment.html>