AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6

[Nicolas Dufresne]

Le jeudi 13 décembre 2018 à 14:08 +0000, Maurer, Marie a écrit :
> Hi Nicolas,
> 
> many thanks for your answer. I am currently building Gstreamer 1.14.4 (and some more). When it is done, I can retest.
> We don't use Yocto, but I had not yet locally updated to 1.14.4.
> 
> The complete pipeline is big, but I think I can reduce it to following overview (queue, caps removed, where not explicitly needed):
> 
> appsrc -> tee T1
> 
> T1 -> v4l2h264enc -> tee T2
> 
> T2#1 -> "Streaming"
> 
> T2#2 -> ("Video Recording to a file") h264parse -> CAPS streaming-format=avc -> ...
> 
> Video Recording to a file contains the h264parse, which reports these warnings.
> 
> We attach v4l2h264enc (to T1) only when streaming and/or video recording is enabled to save resources
> and detach it when both are switched off.
> We attach streaming (to T2) only, when it is enabled and detach it when streaming is disabled.
> We attach "video recording to a file" (also to T2) only when user wants it and detach it when user switches it off.
> At T1 we have more parts connected, which consume frames.
> 
> The main problem I have (beside the reported warnings) is that sometimes, h264parse and following caps can't be linked.
> 
> !!!Speculation!!!
> This could be, but I am not sure yet, that h264parse reports streaming-format=byte-stream, which does not fit to caps with streaming-format=avc.
> We use h264parse to change streaming-format byte-stream (before) to avc (afterwards).
> Is there a more intelligent way needed after h264parse instead of just forcing caps to solve link problem?

Normally that constraint should come form the next element, like a
muxer. If you use a recorder base on ISOMP4 (qtmux), the muxer would
force AVC conversion automatically.

> 
> Another thing I am currently unsure: When we dynamically add h264parse, is there some synchronization to frame data needed, so we start exactly at the beginning of a GOP?
> But we are already on compressed side, so is it relevant at all?

Normally, h264parse will drop data until it reaches a state where at
least one SPS and one PPS has been seen. Everything before that is
dropped. I would suggest to review closely you're dynamic linking path,
you have have introduce a race.

> 
> Many thanks!
> 
> Best regards,
> 
> Marie
> 
> -----Ursprüngliche Nachricht-----
> Von: gstreamer-devel [mailto:gstreamer-devel-bounces at lists.freedesktop.org] Im Auftrag von Nicolas Dufresne
> Gesendet: Mittwoch, 12. Dezember 2018 18:39
> An: Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>
> Betreff: Re: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6
> 
> Le mercredi 12 décembre 2018 à 17:12 +0000, Maurer, Marie a écrit :
> > Hello,
> >  
> > environment is GStreamer 1.14.2 on Linux with i.MX6 (most of it latest 
> > mainlined versions)
> 
> Please, try again with 1.14.4, as there is some fixes in the 1.14 stable branch. And if you can, complain to Yocto people who seem not to stay up-to-date with stable.
> 
> >  
> > I see a lot messages like the following:
> >  
> > WARN   h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame:<RecorderH264Parser> broken/invalid nal Type: 1 Slice, Size: 81703 will be dropped
> >  
> > Size is changing, but always around between 50000 and 80000.
> >  
> > I think the relevant part of my pipeline is a v4l2h264enc and after some queues and tees a h264parse.
> >  
> > Is it correct that somehow the parser does not understand the data which the v4l2h264enc is generating?
> > Or the data which the encoder is generating gets sometimes somehow corrupted till it is received by h264parse?
> 
> Hard to say, though it's better to update first, and see if any of the upstream bugfixes improves the situation.
> 
> >  
> > Any idea how it can be solved?
> 
> You would have to share the pipeline you are using, the settings and all. Ideally if you can reproduce in gst-launch-1.0 form, then I could try reproduce locally.
> 
> >  
> > Best regards,
> >  
> > Marie
> >  
> > _______________________________________________
> > gstreamer-devel mailing list
> > gstreamer-devel at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> 
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel