[PATCH] dim: avoid early decode on email body.

[Jani Nikula]

On Wed, 16 Sep 2020, Daniel Vetter <daniel at ffwll.ch> wrote:
> On Wed, Sep 16, 2020 at 8:59 AM Jani Nikula <jani.nikula at linux.intel.com> wrote:
>>
>> On Tue, 15 Sep 2020, Rodrigo Vivi <rodrigo.vivi at intel.com> wrote:
>> > On Tue, Sep 15, 2020 at 08:33:25PM +0300, Jani Nikula wrote:
>> >> On Tue, 15 Sep 2020, Rodrigo Vivi <rodrigo.vivi at intel.com> wrote:
>> >> > On Tue, Sep 15, 2020 at 11:05:24AM +0300, Jani Nikula wrote:
>> >> >> On Tue, 15 Sep 2020, Daniel Vetter <daniel at ffwll.ch> wrote:
>> >> >> Please test this on a message with Content-Transfer-Encoding set before
>> >> >> applying.
>> >> >
>> >> > I tested and it doesn't work... but it also doesn't work without this patch
>> >> > and with python2. It doesn't work anyways...
>> >> >
>> >> > Is this a case that we should care?
>> >>
>> >> Yes. Even if the sender does not use it, some mail transfer agents may,
>> >> for example, base64 encode the body. We do need to decode the mails.
>> >
>> > Yes, you are right... but it never worked so it means we never supported
>> > this case. why should we start supporting it now?
>>
>> I'm pretty sure it has worked in the past, and *must* continue to
>> work. Please just git blame the history near there and you'll find
>> base64 references.
>>
>> >> Anyway, the problem doesn't seem to be specifically about the decoding
>> >> itself, I think it's about the difference in python 2 vs. 3 unicode. In
>> >> your example, it looks like a binary blob.
>> >
>> > indeed, but I didn't find a clean way to modify the decode part itself.
>> > and removing the decode works for all of our current cases.
>>
>> Ahem, all the cases you just tried. :p
>
> I feel like we need to start having some testcases ... like set up an
> example dim, then do a bunch of these special cases and make sure it's
> all parsed correctly.
>
> Maybe not full-up tests but built-in tests at the bash level, like the
> test-dim.sh script sources dim (so it can call all internal functions)
> and then throws a bunch of examples at the various functions. I
> totally forgotten the base64 case, and we have a ton of these corner
> cases in parsing emails :-/

In all of our copious free time...

...seems as likely as converting the whole thing to python. :/

Anyway, I've sent what I believe is the proper fix to the issue.


BR,
Jani.


> -Daniel
>
>>
>>
>> BR,
>> Jani.
>>
>>
>> --
>> Jani Nikula, Intel Open Source Graphics Center

-- 
Jani Nikula, Intel Open Source Graphics Center