AW: AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6
Maurer, Marie
marie.maurer at zeiss.com
Fri Dec 14 17:18:09 UTC 2018
Hello Nicolas,
just a short follow-up: Same problem (warnings and error with gst_element_link) with 1.14.4.
Tried these two commits:
https://github.com/GStreamer/gst-plugins-bad/commit/4f7fe897b9be77c983f935538d30a5c90cecc947
and
https://lists.freedesktop.org/archives/gstreamer-commits/2018-December/109067.html
-> no difference
25xOK (only 3 shown here), 1x Error:
Line 639452: 0:05:08.615060370 523 0x71d9db50 DEBUG h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
Line 667789: 0:05:21.600504371 523 0x9e7d20 DEBUG h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
Line 696382: 0:05:34.680368373 523 0x71d13950 DEBUG h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
Line 724887: 0:05:47.681662041 523 0x6ff35980 DEBUG h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: (NULL)
OK:
0:05:34.679142373 523 0x71d13950 DEBUG h264parse gsth264parse.c:334:gst_h264_parse_format_from_caps:[00m parsing caps: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)baseline, level=(string)4, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30000/1001, interlace-mode=(string)progressive, colorimetry=(string)bt709
0:05:34.679444373 523 0x71d13950 DEBUG h264parse gsth264parse.c:2716:gst_h264_parse_set_caps:<RecorderH264Parser> have bytestream h264
0:05:34.680368373 523 0x71d13950 DEBUG h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
0:05:34.680629040 523 0x71d13950 DEBUG h264parse gsth264parse.c:381:gst_h264_parse_negotiate:<RecorderH264Parser> negotiating with caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
0:05:34.680830040 523 0x71d13950 DEBUG h264parse gsth264parse.c:334:gst_h264_parse_format_from_caps: parsing caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
0:05:34.681008373 523 0x71d13950 DEBUG h264parse gsth264parse.c:411:gst_h264_parse_negotiate:<RecorderH264Parser> selected format avc, alignment au
0:05:34.681650373 523 0x71d13950 LOG h264parse gsth264parse.c:1102:gst_h264_parse_handle_frame:<RecorderH264Parser> parsing new frame
0:05:34.681862373 523 0x71d13950 DEBUG h264parse gsth264parse.c:186:gst_h264_parse_reset_frame:<RecorderH264Parser> reset frame
Error:
0:05:47.679435708 523 0x6ff35980 DEBUG h264parse gsth264parse.c:334:gst_h264_parse_format_from_caps:[00m parsing caps: video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)baseline, level=(string)4, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30000/1001, interlace-mode=(string)progressive, colorimetry=(string)bt709
0:05:47.680866041 523 0x6ff35980 DEBUG h264parse gsth264parse.c:2716:gst_h264_parse_set_caps:<RecorderH264Parser> have bytestream h264
0:05:47.681662041 523 0x6ff35980 DEBUG h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: (NULL)
0:05:47.682161374 523 0x6ff35980 DEBUG h264parse gsth264parse.c:411:gst_h264_parse_negotiate:<RecorderH264Parser> selected format byte-stream, alignment au
0:05:47.682607374 523 0x6ff35980 DEBUG h264parse gsth264parse.c:1755:gst_h264_parse_update_src_caps:<RecorderH264Parser> sps: (nil)
0:05:47.685859374 523 0x6ff35980 LOG h264parse gsth264parse.c:1102:gst_h264_parse_handle_frame:<RecorderH264Parser> parsing new frame
0:05:47.686587041 523 0x6ff35980 DEBUG h264parse gsth264parse.c:186:gst_h264_parse_reset_frame:<RecorderH264Parser> reset frame
0:05:47.687081708 523 0x6ff35980 DEBUG h264parse gsth264parse.c:1117:gst_h264_parse_handle_frame:<RecorderH264Parser> last parse position 0
Best regards,
Marie
-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:gstreamer-devel-bounces at lists.freedesktop.org] Im Auftrag von Nicolas Dufresne
Gesendet: Donnerstag, 13. Dezember 2018 18:39
An: Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>
Betreff: Re: AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6
Le jeudi 13 décembre 2018 à 14:08 +0000, Maurer, Marie a écrit :
> Hi Nicolas,
>
> many thanks for your answer. I am currently building Gstreamer 1.14.4 (and some more). When it is done, I can retest.
> We don't use Yocto, but I had not yet locally updated to 1.14.4.
>
> The complete pipeline is big, but I think I can reduce it to following overview (queue, caps removed, where not explicitly needed):
>
> appsrc -> tee T1
>
> T1 -> v4l2h264enc -> tee T2
>
> T2#1 -> "Streaming"
>
> T2#2 -> ("Video Recording to a file") h264parse -> CAPS streaming-format=avc -> ...
>
> Video Recording to a file contains the h264parse, which reports these warnings.
>
> We attach v4l2h264enc (to T1) only when streaming and/or video
> recording is enabled to save resources and detach it when both are switched off.
> We attach streaming (to T2) only, when it is enabled and detach it when streaming is disabled.
> We attach "video recording to a file" (also to T2) only when user wants it and detach it when user switches it off.
> At T1 we have more parts connected, which consume frames.
>
> The main problem I have (beside the reported warnings) is that sometimes, h264parse and following caps can't be linked.
>
> !!!Speculation!!!
> This could be, but I am not sure yet, that h264parse reports streaming-format=byte-stream, which does not fit to caps with streaming-format=avc.
> We use h264parse to change streaming-format byte-stream (before) to avc (afterwards).
> Is there a more intelligent way needed after h264parse instead of just forcing caps to solve link problem?
Normally that constraint should come form the next element, like a muxer. If you use a recorder base on ISOMP4 (qtmux), the muxer would force AVC conversion automatically.
>
> Another thing I am currently unsure: When we dynamically add h264parse, is there some synchronization to frame data needed, so we start exactly at the beginning of a GOP?
> But we are already on compressed side, so is it relevant at all?
Normally, h264parse will drop data until it reaches a state where at least one SPS and one PPS has been seen. Everything before that is dropped. I would suggest to review closely you're dynamic linking path, you have have introduce a race.
>
> Many thanks!
>
> Best regards,
>
> Marie
>
> -----Ursprüngliche Nachricht-----
> Von: gstreamer-devel
> [mailto:gstreamer-devel-bounces at lists.freedesktop.org] Im Auftrag von
> Nicolas Dufresne
> Gesendet: Mittwoch, 12. Dezember 2018 18:39
> An: Discussion of the development of and with GStreamer
> <gstreamer-devel at lists.freedesktop.org>
> Betreff: Re: "broken/invalid nal" with h264parse after v4l2h264enc on
> i.MX6
>
> Le mercredi 12 décembre 2018 à 17:12 +0000, Maurer, Marie a écrit :
> > Hello,
> >
> > environment is GStreamer 1.14.2 on Linux with i.MX6 (most of it
> > latest mainlined versions)
>
> Please, try again with 1.14.4, as there is some fixes in the 1.14 stable branch. And if you can, complain to Yocto people who seem not to stay up-to-date with stable.
>
> >
> > I see a lot messages like the following:
> >
> > WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame:<RecorderH264Parser> broken/invalid nal Type: 1 Slice, Size: 81703 will be dropped
> >
> > Size is changing, but always around between 50000 and 80000.
> >
> > I think the relevant part of my pipeline is a v4l2h264enc and after some queues and tees a h264parse.
> >
> > Is it correct that somehow the parser does not understand the data which the v4l2h264enc is generating?
> > Or the data which the encoder is generating gets sometimes somehow corrupted till it is received by h264parse?
>
> Hard to say, though it's better to update first, and see if any of the upstream bugfixes improves the situation.
>
> >
> > Any idea how it can be solved?
>
> You would have to share the pipeline you are using, the settings and all. Ideally if you can reproduce in gst-launch-1.0 form, then I could try reproduce locally.
>
> >
> > Best regards,
> >
> > Marie
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > gstreamer-devel at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
gstreamer-devel at lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
More information about the gstreamer-devel
mailing list