Hello Nicolas,
as far as I know dynamic pipeline attachment and detachment is not possible
via gst-launch-1.0 or similar tools?
The pipeline is part of a bigger layer which controls gstreamer pipeline creating, modification, destruction. It depends on other parts of the software.
So at the moment I think, I am not able to share this source code.
What other steps can I do to analyze the problem?
What I don't yet understand:
When I have
caps stream-format=(string)byte-stream -> h264parse -> caps stream-format=(string)avc
I force h264parse to use a given stream-format.
When I get a negotiation failure, then h264parse seems to not fulfill what I had requested via caps?
I already put printfs into source of h264parse, but not yet able to understand its behavior.
I assume some (wrong?) input data is responsible for stream-format change, which is then not compatible
to my following caps.
Best regards,
Marie
-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:gstreamer-devel-bounces at lists.freedesktop.org] Im Auftrag von Nicolas Dufresne
Gesendet: Freitag, 14. Dezember 2018 19:40
An: Discussion of the development of and with GStreamer <gstreamer-devel at lists.freedesktop.org>
Betreff: Re: AW: AW: "broken/invalid nal" with h264parse after v4l2h264enc on i.MX6
Le vendredi 14 décembre 2018 à 17:18 +0000, Maurer, Marie a écrit :
> Hello Nicolas,
>
> just a short follow-up: Same problem (warnings and error with gst_element_link) with 1.14.4.
>
> Tried these two commits:
>
> https://github.com/GStreamer/gst-plugins-bad/commit/4f7fe897b9be77c983
> f935538d30a5c90cecc947
>
> and
>
> https://lists.freedesktop.org/archives/gstreamer-commits/2018-December
> /109067.html
I don't think any of this is related. Again, can you share the pipeline you are running, I would maybe help by testing on my IMX.6 board here.
>
> -> no difference
>
>
> 25xOK (only 3 shown here), 1x Error:
>
> Line 639452: 0:05:08.615060370 523 0x71d9db50 DEBUG h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
> Line 667789: 0:05:21.600504371 523 0x9e7d20 DEBUG h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
> Line 696382: 0:05:34.680368373 523 0x71d13950 DEBUG h264parse gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser> allowed caps: video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au
> Line 724887: 0:05:47.681662041 523 0x6ff35980 DEBUG
> h264parse
> gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser>
> allowed caps: (NULL)
>
> OK:
>
> 0:05:34.679142373 523 0x71d13950 DEBUG h264parse
> gsth264parse.c:334:gst_h264_parse_format_from_caps:[00m parsing caps:
> video/x-h264, stream-format=(string)byte-stream, alignment=(string)au,
> profile=(string)baseline, level=(string)4, width=(int)1920,
> height=(int)1080, pixel-aspect-ratio=(fraction)1/1,
> framerate=(fraction)30000/1001, interlace-mode=(string)progressive,
> colorimetry=(string)bt709
> 0:05:34.679444373 523 0x71d13950 DEBUG h264parse
> gsth264parse.c:2716:gst_h264_parse_set_caps:<RecorderH264Parser> have
> bytestream h264
> 0:05:34.680368373 523 0x71d13950 DEBUG h264parse
> gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser>
> allowed caps: video/x-h264, parsed=(boolean)true,
> stream-format=(string)avc, alignment=(string)au
> 0:05:34.680629040 523 0x71d13950 DEBUG h264parse
> gsth264parse.c:381:gst_h264_parse_negotiate:<RecorderH264Parser>
> negotiating with caps: video/x-h264, parsed=(boolean)true,
> stream-format=(string)avc, alignment=(string)au
> 0:05:34.680830040 523 0x71d13950 DEBUG h264parse
> gsth264parse.c:334:gst_h264_parse_format_from_caps: parsing caps:
> video/x-h264, parsed=(boolean)true, stream-format=(string)avc,
> alignment=(string)au
> 0:05:34.681008373 523 0x71d13950 DEBUG h264parse gsth264parse.c:411:gst_h264_parse_negotiate:<RecorderH264Parser> selected format avc, alignment au
> 0:05:34.681650373 523 0x71d13950 LOG h264parse gsth264parse.c:1102:gst_h264_parse_handle_frame:<RecorderH264Parser> parsing new frame
> 0:05:34.681862373 523 0x71d13950 DEBUG h264parse
> gsth264parse.c:186:gst_h264_parse_reset_frame:<RecorderH264Parser>
> reset frame
>
> Error:
>
> 0:05:47.679435708 523 0x6ff35980 DEBUG h264parse
> gsth264parse.c:334:gst_h264_parse_format_from_caps:[00m parsing caps:
> video/x-h264, stream-format=(string)byte-stream, alignment=(string)au,
> profile=(string)baseline, level=(string)4, width=(int)1920,
> height=(int)1080, pixel-aspect-ratio=(fraction)1/1,
> framerate=(fraction)30000/1001, interlace-mode=(string)progressive,
> colorimetry=(string)bt709
> 0:05:47.680866041 523 0x6ff35980 DEBUG h264parse
> gsth264parse.c:2716:gst_h264_parse_set_caps:<RecorderH264Parser> have
> bytestream h264
> 0:05:47.681662041 523 0x6ff35980 DEBUG h264parse
> gsth264parse.c:374:gst_h264_parse_negotiate:<RecorderH264Parser>
> allowed caps: (NULL)
> 0:05:47.682161374 523 0x6ff35980 DEBUG h264parse
> gsth264parse.c:411:gst_h264_parse_negotiate:<RecorderH264Parser>
> selected format byte-stream, alignment au
> 0:05:47.682607374 523 0x6ff35980 DEBUG h264parse gsth264parse.c:1755:gst_h264_parse_update_src_caps:<RecorderH264Parser> sps: (nil)
> 0:05:47.685859374 523 0x6ff35980 LOG h264parse gsth264parse.c:1102:gst_h264_parse_handle_frame:<RecorderH264Parser> parsing new frame
> 0:05:47.686587041 523 0x6ff35980 DEBUG h264parse
> gsth264parse.c:186:gst_h264_parse_reset_frame:<RecorderH264Parser>
> reset frame
> 0:05:47.687081708 523 0x6ff35980 DEBUG h264parse
> gsth264parse.c:1117:gst_h264_parse_handle_frame:<RecorderH264Parser>
> last parse position 0
>
> Best regards,
>
> Marie
>
> -----Ursprüngliche Nachricht-----
> Von: gstreamer-devel
> [mailto:gstreamer-devel-bounces at lists.freedesktop.org] Im Auftrag von
> Nicolas Dufresne
> Gesendet: Donnerstag, 13. Dezember 2018 18:39
> An: Discussion of the development of and with GStreamer
> <gstreamer-devel at lists.freedesktop.org>
> Betreff: Re: AW: "broken/invalid nal" with h264parse after v4l2h264enc
> on i.MX6
>
> Le jeudi 13 décembre 2018 à 14:08 +0000, Maurer, Marie a écrit :
> > Hi Nicolas,
> >
> > many thanks for your answer. I am currently building Gstreamer 1.14.4 (and some more). When it is done, I can retest.
> > We don't use Yocto, but I had not yet locally updated to 1.14.4.
> >
> > The complete pipeline is big, but I think I can reduce it to following overview (queue, caps removed, where not explicitly needed):
> >
> > appsrc -> tee T1
> >
> > T1 -> v4l2h264enc -> tee T2
> >
> > T2#1 -> "Streaming"
> >
> > T2#2 -> ("Video Recording to a file") h264parse -> CAPS streaming-format=avc -> ...
> >
> > Video Recording to a file contains the h264parse, which reports these warnings.
> >
> > We attach v4l2h264enc (to T1) only when streaming and/or video
> > recording is enabled to save resources and detach it when both are switched off.
> > We attach streaming (to T2) only, when it is enabled and detach it when streaming is disabled.
> > We attach "video recording to a file" (also to T2) only when user wants it and detach it when user switches it off.
> > At T1 we have more parts connected, which consume frames.
> >
> > The main problem I have (beside the reported warnings) is that sometimes, h264parse and following caps can't be linked.
> >
> > !!!Speculation!!!
> > This could be, but I am not sure yet, that h264parse reports streaming-format=byte-stream, which does not fit to caps with streaming-format=avc.
> > We use h264parse to change streaming-format byte-stream (before) to avc (afterwards).
> > Is there a more intelligent way needed after h264parse instead of just forcing caps to solve link problem?
>
> Normally that constraint should come form the next element, like a muxer. If you use a recorder base on ISOMP4 (qtmux), the muxer would force AVC conversion automatically.
>
> > Another thing I am currently unsure: When we dynamically add h264parse, is there some synchronization to frame data needed, so we start exactly at the beginning of a GOP?
> > But we are already on compressed side, so is it relevant at all?
>
> Normally, h264parse will drop data until it reaches a state where at least one SPS and one PPS has been seen. Everything before that is dropped. I would suggest to review closely you're dynamic linking path, you have have introduce a race.
>
> > Many thanks!
> >
> > Best regards,
> >
> > Marie
> >
> > -----Ursprüngliche Nachricht-----
> > Von: gstreamer-devel
> > [mailto:gstreamer-devel-bounces at lists.freedesktop.org] Im Auftrag
> > von Nicolas Dufresne
> > Gesendet: Mittwoch, 12. Dezember 2018 18:39
> > An: Discussion of the development of and with GStreamer
> > <gstreamer-devel at lists.freedesktop.org>
> > Betreff: Re: "broken/invalid nal" with h264parse after v4l2h264enc
> > on
> > i.MX6
> >
> > Le mercredi 12 décembre 2018 à 17:12 +0000, Maurer, Marie a écrit :
> > > Hello,
> > >
> > > environment is GStreamer 1.14.2 on Linux with i.MX6 (most of it
> > > latest mainlined versions)
> >
> > Please, try again with 1.14.4, as there is some fixes in the 1.14 stable branch. And if you can, complain to Yocto people who seem not to stay up-to-date with stable.
> >
> > >
> > > I see a lot messages like the following:
> > >
> > > WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame:<RecorderH264Parser> broken/invalid nal Type: 1 Slice, Size: 81703 will be dropped
> > >
> > > Size is changing, but always around between 50000 and 80000.
> > >
> > > I think the relevant part of my pipeline is a v4l2h264enc and after some queues and tees a h264parse.
> > >
> > > Is it correct that somehow the parser does not understand the data which the v4l2h264enc is generating?
> > > Or the data which the encoder is generating gets sometimes somehow corrupted till it is received by h264parse?
> >
> > Hard to say, though it's better to update first, and see if any of the upstream bugfixes improves the situation.
> >
> > >
> > > Any idea how it can be solved?
> >
> > You would have to share the pipeline you are using, the settings and all. Ideally if you can reproduce in gst-launch-1.0 form, then I could try reproduce locally.
> >
> > >
> > > Best regards,
> > >
> > > Marie
> > >
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > gstreamer-devel at lists.freedesktop.org
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > gstreamer-devel at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > _______________________________________________
> > gstreamer-devel mailing list
> > gstreamer-devel at lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel