[poppler] poppler::ustring encoding issue

[suzuki toshiya]

Dear Albert,

Thank you for response. From now, I would post proposed patch
as attachment - if too big, I would file to bugzilla. Here I
attach pt1 and pt2 as separated attachments.

>> In fact, I'm unfamiliar with how the cpp-frontend users think
>> about a BOM in ustring object. If there are so many existing
>> implementations assuming as if ustring always starts with a BOM
>> (and they have their own routines for the concatenation, splicing
>> and replacing), we should care for that. Please let me hear how
>> the users think.
> 
> I guess this is not very important, I mean after all it was broken so i guess 
> noone could make it really work?

OK... I found a few comments from Unicode on BOM:
	http://www.unicode.org/faq/utf_bom.html#bom6
It seems that handing BOM during the string is not straight-
forward (just removing it can cause semantically difference),
so I suggest to use non-BOM approach (of current patch),
until some complains from the users/reviewers.

> what we need to do is document exactly how it behaves (since it seems to be a 
> bit under documented now).

I see. where is the appropriate place to add a document of
poppler::ustring class itself?

Regards,
mpsuzuki

P.S.
Thanks to Jeroen too for the info how to check the commit quickly,
it is very helpful to make the attachments of this :-)

Jeroen Ooms wrote:
> On Mon, Mar 26, 2018 at 10:06 PM, Albert Astals Cid <aacid at kde.org> wrote:
>> El diumenge, 25 de març de 2018, a les 5:39:18 CEST, suzuki toshiya va
>> escriure:
>>> Hi all,
>>>
>>> Finally I think I found the root of issue and I can propose a fix.
>>> pre-patch situation is like this:
>>> https://travis-ci.org/mpsuzuki/poppler/builds/357212162
>>>
>>> post-patch situation is like this:
>>> https://travis-ci.org/mpsuzuki/poppler/builds/357956103
>>>
>>> My fix consists from 2 parts.
>> Can you post the patches either to bugzilla or as attachments to the mailing
>> list? I don't feel confortable using github.
> 
> One easy way to get patches from Github without actually having to
> pull in the repository is simply appending ".diff" or  ".patch to the
> Github commit URLs that he posted:
> 
>  - https://github.com/mpsuzuki/poppler/commit/7404f5effa8e303399e5101d54ff954ee5153e44.diff
>  - https://github.com/mpsuzuki/poppler/commit/b3230c7098b891da0b92742264d78c9bd86750bd.diff
> _______________________________________________
> poppler mailing list
> poppler at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/poppler

-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix-ustring_pt1.diff
Type: text/x-patch
Size: 5712 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/poppler/attachments/20180327/59f16f0c/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix-ustring_pt2.diff
Type: text/x-patch
Size: 3148 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/poppler/attachments/20180327/59f16f0c/attachment-0001.bin>