[PUSHED][EasyHack] #44681 port to CLucene from java/Lucene

[Caolán McNamara]

On Tue, 2012-02-21 at 10:21 +0100, G.H.M.Valkenhoef, van wrote:
> Great to hear that it works & that it will make it into master :-).
> 
> Any thoughts on this one:
> 
>  3) When creating the CLucene FileReader (HelpIndexer.cxx), the path
> is converted to plain ASCII, that's probably dangerous. There is
> probably a way to work around this, but I haven't gotten around to it
> yet.

Think I sorted out all these minor quibbles. I did an all lang build and
a windows build and both work fine now. I double tested the CJK issue
and that still works with clucene as well. So this is integrated in
master now. 

Basically it's great and gives me the warm and fuzzies :-) The
introduction of a java indexer right after we had replaced the original
java HelpCompiler and HelpSearch stuff with a c++ replacement (which had
knocked something like 10 hours of my 26-hour Linux ALL language build
back in the day when gcj was the only option for me for java) depressed
me no end, so it cheers me up endlessly to see the back of it.

Only interesting problem I encountered was that clucene's defaults
overflowed for our "ja" help and threw an exception on creating the help
indexes. Apparently, according to the documentation, lucene has the same
limit except silently drops what doesn't fit, so now that I've doubled
the limit I presume this means we get better Japanese indexing as well
as a freebie.

Memory management in clucene is sort of weird, but all our uses of it
are "by-the-book" wrt their documentation and examples.

C.