[PATCH] dim: avoid early decode on email body.

[Daniel Vetter]

On Wed, Sep 16, 2020 at 8:59 AM Jani Nikula <jani.nikula at linux.intel.com> wrote:
>
> On Tue, 15 Sep 2020, Rodrigo Vivi <rodrigo.vivi at intel.com> wrote:
> > On Tue, Sep 15, 2020 at 08:33:25PM +0300, Jani Nikula wrote:
> >> On Tue, 15 Sep 2020, Rodrigo Vivi <rodrigo.vivi at intel.com> wrote:
> >> > On Tue, Sep 15, 2020 at 11:05:24AM +0300, Jani Nikula wrote:
> >> >> On Tue, 15 Sep 2020, Daniel Vetter <daniel at ffwll.ch> wrote:
> >> >> Please test this on a message with Content-Transfer-Encoding set before
> >> >> applying.
> >> >
> >> > I tested and it doesn't work... but it also doesn't work without this patch
> >> > and with python2. It doesn't work anyways...
> >> >
> >> > Is this a case that we should care?
> >>
> >> Yes. Even if the sender does not use it, some mail transfer agents may,
> >> for example, base64 encode the body. We do need to decode the mails.
> >
> > Yes, you are right... but it never worked so it means we never supported
> > this case. why should we start supporting it now?
>
> I'm pretty sure it has worked in the past, and *must* continue to
> work. Please just git blame the history near there and you'll find
> base64 references.
>
> >> Anyway, the problem doesn't seem to be specifically about the decoding
> >> itself, I think it's about the difference in python 2 vs. 3 unicode. In
> >> your example, it looks like a binary blob.
> >
> > indeed, but I didn't find a clean way to modify the decode part itself.
> > and removing the decode works for all of our current cases.
>
> Ahem, all the cases you just tried. :p

I feel like we need to start having some testcases ... like set up an
example dim, then do a bunch of these special cases and make sure it's
all parsed correctly.

Maybe not full-up tests but built-in tests at the bash level, like the
test-dim.sh script sources dim (so it can call all internal functions)
and then throws a bunch of examples at the various functions. I
totally forgotten the base64 case, and we have a ton of these corner
cases in parsing emails :-/
-Daniel

>
>
> BR,
> Jani.
>
>
> --
> Jani Nikula, Intel Open Source Graphics Center



-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch