v4l2videodec with imx6 coda

[Nicolas Dufresne]

Le lundi 28 juillet 2014 à 11:15 +0200, Philipp Zabel a écrit :
> Hi Nicolas,
> 
> Am Sonntag, den 27.07.2014, 17:47 +0100 schrieb Nicolas Dufresne:
> >  Le Samedi 26 Juillet 2014 10:49 EDT, Philipp Zabel <pza at pengutronix.de> a écrit: 
> > > I see two issues, currently:
> > > a) v4l2videodec should do a TRY_FMT/S_FMT on the V4L2 capture side.
> > >    Currently only the I420 output format works because that happens
> > >    to be the default. Also, the future CODA JPEG decoder will allow
> > >    downscaling (by factors of 2, 4, and 8) during decompression.
> > 
> > I've created bugs to track these feature requests.
> > 
> > For the negotiation of output color format:
> > https://bugzilla.gnome.org/show_bug.cgi?id=733827
> > 
> > For the negotiation of output size:
> > https://bugzilla.gnome.org/show_bug.cgi?id=733828
> 
> Thank you.
> 
> > Patch are welcome. Also, be aware that I'm looking forward to see the probing of output capabilities
> > to be documented, sanitized and well implemented in drivers. Currently your TRY_FMT proposition
> > would imply too many combination to try on the width/height side.
> 
> Would it work if we were to implement ENUM_FMT on the output side to
> limit the reported frame sizes to those that can be produced from the
> currently set format on the input side?

This could work, though it's not documented in V4L API, I'd like to
interest the maintainers so we know we create a standard for this. We
also need to make sure ENUM_FMT won't fail before the output format is
set, as it would otherwise be hard to probe the driver and figure-out
it's a decoder. On GStreamer side, we will need a mechanism to ignore
the cached probed caps after the output format is set. Not too
difficult, but a bit orthogonal to the old and aging caps probing
design.

> 
> >  The format part is relatively trivial, all one need is few lines of code and a driver to test it. Patches are welcome.
> > 
> > > b) At stream end I get a hardware timeout because the v4l2videodec doesn't
> > >    send a V4L2_DEC_CMD_STOP decoder command to tell the CODA it may flush
> > >    its bitstream buffer.
> > 
> > This is unfortunate, but it would seem decoders drivers don't agree on the draining method.
> > The s5p-mfc decoder, expect that an empty buffer is pushed to drain at least 1 buffer out of the
> > decoder. Then the decoder will indicate that all buffers has been drain by sending en empty buffer
> > on the capture side. This is more powerful, as it allow implementing partial draining for timeout base
> > real time decoding.
> 
> The s5p-mfc decoder does support VIDIOC_DECODER_CMD / V4L2_DEC_CMD_STOP
> to enter its stream-end state. After dequeueing the last decoded frame,
> the driver will generate a V4L2_EVENT_EOS event.

I think I got tricked by a really stupid bug. See:

 drivers/media/platform/s5p-mfc/s5p_mfc_dec.c:832

They switch on V4L2_ENC_CMD_STOP instead of V4L2_DEC_CMD_STOP. That's
why I never found that support (and the the code I initially wrote never
worked). Please feel free to propose patch for that (fix kernel MFC, and
propose support to GStreamer). I'd like to keep a fallback to empty
buffers (for older kernel with MFC if possible). On the capture side,
was the conclusion to use an EOS event or an EOS flag ? That also need
to be implemented in GStreamer. Events are currently not implemented at
all, that includes the very recent EVENT_SOURCE_CHANGE, used for
resolution change.

> 
> I chose to also use this method after the discussion in this thread:
> https://patchwork.linuxtv.org/patch/18486/
> 
> >  But isn't very well documented. Your method is not documented for men-2-men
> > devices either though, but is slightly simpler.

The empty buffer was documented in some Samsung slide, but no patch was
indeed proposed for the v4l2 spec.

> 
> Do you think the VIDIOC_DECODER_CMD documentation should explicitly
> mention capture and mem2mem devices?

Yes, I think some clarification would be really nice to have, like
documented what is to be expected on the capture side when done
draining.

> 
> >  It implies that all drivers have some thread running I think.
> > Could you raise this issue to the linux-media mailing list, or at least agree with Kamil on a shared method ?
> 
> Yes, I'll bring it up again on linux-media.

Thank you for your time, I'm glad to find that MFC isn't so different,
that they just fat finger ENC and DEC in their implementation. That's
mainly because they patched it, but kept using the old empty buffers
method.

thanks,
Nicolas