Problem when Stopping then Restarting Pipeline in Python (Video only)
Nicolas Dufresne
nicolas at ndufresne.ca
Wed Jul 8 22:46:31 UTC 2020
Le mercredi 08 juillet 2020 à 15:28 -0700, whoman at gmail.com a écrit :
>
>
> On Wed, Jul 8, 2020 at 3:16 PM Nicolas Dufresne <nicolas at ndufresne.ca> wrote:
> >
> > Le mer. 8 juill. 2020 18 h 00, whoman at gmail.com <whoman at gmail.com> a écrit :
> > > HI Everyone,
> > > I have a simple gstreamer python script that works perfectly on intel with Hardware H264 decoding, however when I
> > > try to port this script to ARM (specifically Rockchip RK3328), it has a strange problem.
> > >
> > > The combination of Rockchip HW decoder and Rockchip Sink causes an error whenever I attempt to change the 'Source' and restart the pipeline
> >
> > These elements based on vendor drivers are maintained by Rockchip on their GitHub repository. Please report the issue there.
> >
>
> They're currently only maintaining for RK3228 and RK3399 chipsets for these GST elements on GITHUB, however I will report the issue there and see if they can assist.
Interesting, I didn't know they had reduce maintenance.
>
>
> > Meanwhile, development release of GStreamer now includes support for rk3288 mainline Linux drivers, feel free to give it a try and report.
> >
>
> I'm using RK3328. Does the development release also support RK3328 mainline drivers?
Ah, sorry for this miss read, numbers are so similar. RK3328 seems to
share are lot with RK3399. Jonas Karlman has been working on enabling
this variant, I can see that MPEG2 support is mainline, but I believe a
lot more is incoming. You'll find him on IRC (usually un libreelec
channel) as Kwiboo.
>
> Thanks so much for your time
>
>
> > > The python script does the following:
> > >
> > > 1. Sets up a pipeline
> > > 2. Listens to keyboard input for either 1 or 2 (number keys)
> > > 3. On Key press,
> > > A. the pipeline is paused (READY)
> > > B. the 'location' for filesrc is changed
> > > C. the pipeline is set back to PLAYING
> > >
> > > When the pipeline is set to PLAYING for the 2nd time, the program crashes, when using ROckchip's HW decoder and sink
> > >
> > > My script for INTEL sets up the following pipeline:
> > >
> > > Source: filesrc (h264 video)
> > > Demux: qtdemux
> > > parse: h264parse
> > > decoder: vaapidec (intel hw h264 decoder)
> > > videoconvert: videoconvert
> > > sink: xvimagesink
> > >
> > > ...with a pad listener to dynamically add the link between Demux and Parse once a Source is selected.
> > >
> > > This pipeline works fine. Changing filesrc and restarting the pipeline multiple times works perfectly.
> > >
> > >
> > > My script for ARM (rockchip) sets up the following pipeline (only changes are Decoder and Sink):
> > >
> > > Source: filesrc (h264 video)
> > > Demux: qtdemux
> > > parse: h264parse
> > > decoder: mppvideodec (rockchip hw h264 decoder)
> > > videoconvert: videoconvert
> > > sink: kmssink or rkximagesink (rockchip video sinks)
> > >
> > > The pipeline works fine when the first video is selected and played, however...
> > >
> > > Whenever I try to change the SOURCE and then restart the pipeline (2nd time), I receive an error when changing the pipeline's state back to PLAYING
> > >
> > > -------
> > > (example.py:1024): GStreamer-CRITICAL **: gst_caps_is_strictly_equal: assertion 'GST_IS_CAPS (caps1)' failed
> > >
> > > ** (example.py:1024): CRITICAL **: gst_video_codec_state_unref: assertion 'state->ref_count > 0' failed
> > >
> > > ** (example.py:1024): CRITICAL **: gst_mpp_allocator_start: assertion 'size != 0' failed
> > > -------
> > >
> > >
> > > with GST_DEBUG=3:
> > > -------
> > > 0:00:08.584026172 1064 0x55932be4f0 ERROR mppvideodec gstmppvideodec.c:693:gst_mpp_video_dec_handle_frame:<decoder> can't process this frame
> > > 0:00:08.589703023 1064 0x55932afe30 WARN basesrc gstbasesrc.c:3491:gst_base_src_start_complete:<source> pad not activated yet
> > > 0:00:08.590945554 1064 0x7fac014540 WARN qtdemux qtdemux.c:7031:qtdemux_parse_container:<demux> length too long (1507328 > 27)
> > > 0:00:08.591528027 1064 0x7fac014540 WARN qtdemux qtdemux.c:2899:qtdemux_parse_trex:<demux> failed to find fragment defaults for stream 1
> > > 0:00:08.592839977 1064 0x7fac014540 WARN qtdemux qtdemux.c:8766:qtdemux_parse_segments:<demux> Segment 0 extends to 0:03:44.891332000 past the end of the file duration 0:03:44.766207000 it will be truncated
> > > 0:00:08.593700415 1064 0x7fac014540 WARN basesrc gstbasesrc.c:2400:gst_base_src_update_length:<source> processing at or past EOS
> > > Received new pad 'video_0' from 'demux'
> > > video/x-h264
> > > Link succeeded (type 'video/x-h264')
> > > 0:00:08.601256230 1064 0x7fac014540 FIXME videodecoder gstvideodecoder.c:945:gst_video_decoder_drain_out:<decoder> Sub-class should implement drain()
> > >
> > > (example.py:1064): GStreamer-CRITICAL **: gst_caps_is_strictly_equal: assertion 'GST_IS_CAPS (caps1)' failed
> > >
> > > ** (example.py:1064): CRITICAL **: gst_video_codec_state_unref: assertion 'state->ref_count > 0' failed
> > > 0:00:08.621745744 1064 0x7fac014540 FIXME videodecoder gstvideodecoder.c:945:gst_video_decoder_drain_out:<decoder> Sub-class should implement drain()
> > > mpp_buf_slot: found info change ready set without internal info change
> > > Segmentation fault
> > > -------
> > >
> > >
> > > Note: If I use the libav SW decoder avdec_h264 instead of mppvideodec, everything works fine and I do not get any errors when changing sources
> > >
> > > I understand that there may be some issues with mppvideodec causing this problem, but since rockchip is mostly focusing on 3228 and 3399 on GITHUB (not RK3328), My best potential option for a solution is to find a workaround within my scripts pipeline code/logic.
> > >
> > >
> > > My Question:
> > > Is there any chance that flushing the buffer/caps between STOP / START of the pipeline could get me past this issue?
> > > Any advice / code to help me make this script function with Rockchip HW decode (mppvideodec) would be greately appreacited.
> > >
> > > FYI, if I use mppvideodec with a standard Gstreamer sink (fbdevsink, xvimagesink, etc) this problem does not occur (however the video is choppy/unusable)
> > > This problem only occurs when using the rockchip decoder (mppvideodec) with a rockchip video sink (kmssink or rkximagesink)
> > >
> > >
> > > I beleive this problem may be solved with proper caps/buffer flushing in-between source changes, but I'm not
> > > sure how to properly acheive this within the Gstreamer environment.
> > >
> > > I've read about what I think may be potential solutions / workarounds....
> > > for example with the older GST (.10) I found this:
> > > https://gstreamer-devel.narkive.com/EpGe0YyN/gst-devel-assertion-gst-is-caps-caps-failed
> > >
> > >
> > > Does anyone know how I might be able to modify this python script to get around this issue?
> > >
> > > Thanks for your time
> > >
> > > ---
> > > example.py:
> > > ---
> > >
> > > import tty
> > > import sys
> > > import termios
> > > import gi
> > > import subprocess
> > > import os, signal
> > > import time
> > > from time import sleep
> > > from gi.repository import GObject
> > > gi.require_version('Gst', '1.0')
> > > from gi.repository import Gst
> > > from gi.repository import GstController
> > > GObject.threads_init()
> > > Gst.init(sys.argv)
> > >
> > >
> > >
> > > #GST PIPELINE
> > > #GST - create the elements
> > > source = Gst.ElementFactory.make("filesrc", "source")
> > > demux = Gst.ElementFactory.make("qtdemux", "demux")
> > > parse = Gst.ElementFactory.make("h264parse", "parse")
> > > decoder = Gst.ElementFactory.make("mppvideodec", "decoder")
> > > video_convert = Gst.ElementFactory.make("videoconvert", "videoconvert")
> > > sink = Gst.ElementFactory.make("kmssink", "sink")
> > >
> > > #GST - create the empty pipeline
> > > p1 = Gst.Pipeline.new("test-pipeline")
> > >
> > > if not p1 or not source or not video_convert or not sink:
> > > print("ERROR: Not all elements could be created")
> > > sys.exit(1)
> > >
> > > #GST - add and link elements in pipeline
> > > p1.add(source)
> > > p1.add(demux)
> > > p1.add(parse)
> > > p1.add(decoder)
> > > p1.add(video_convert)
> > > p1.add(sink)
> > >
> > > if not source.link(demux):
> > > print("ERROR: Could not link source to demux")
> > > sys.exit(1)
> > > #if not demux.link(parse):
> > > # print("ERROR: Could not link demux to parse")
> > > # sys.exit(1)
> > > if not parse.link(decoder):
> > > print("ERROR: Could not link parse to decoder")
> > > sys.exit(1)
> > > if not decoder.link(video_convert):
> > > print("ERROR: Could not link decoder to video_convert")
> > > # sys.exit(1)
> > > if not video_convert.link(sink):
> > > print("ERROR: Could not link scaler to sink")
> > > sys.exit(1)
> > >
> > > # modify the source's location property
> > > source.set_property("location", "/video/g62.mp4")
> > >
> > > ret = p1.get_state(1)
> > > print ret.state
> > >
> > >
> > >
> > > #GST - handler for the pad-added signal
> > > def demux_on_pad_added(src, new_pad):
> > > if new_pad.get_name() == "video_0":
> > > print(
> > > "Received new pad '{0:s}' from '{1:s}'".format(
> > > new_pad.get_name(),
> > > src.get_name()))
> > >
> > > # attempt the link
> > > sink_pad = parse.get_static_pad("sink")
> > > new_pad_caps = new_pad.get_current_caps()
> > > new_pad_struct = new_pad_caps.get_structure(0)
> > > new_pad_type = new_pad_struct.get_name()
> > > print(new_pad_type)
> > > ret = new_pad.link(sink_pad)
> > > if not ret == Gst.PadLinkReturn.OK:
> > > print("Type is '{0:s}' but link failed".format(new_pad_type))
> > > else:
> > > print("Link succeeded (type '{0:s}')".format(new_pad_type))
> > > return
> > > return
> > >
> > > # connect to the pad-added signal
> > > demux.connect("pad-added", demux_on_pad_added)
> > >
> > >
> > >
> > > #GST - Puase playback, set "location", and begin playback
> > > def PlayVideo1():
> > > # GST stop pipeline
> > > p1.set_state(Gst.State.READY)
> > >
> > > # GST set filesrc 'location'
> > > source.set_property('location', '/video/g62s.mp4')
> > >
> > > # GST start pipeline
> > > p1.set_state(Gst.State.PLAYING)
> > >
> > >
> > > def PlayVideo2():
> > > # GST stop pipeline
> > > p1.set_state(Gst.State.READY)
> > >
> > > # GST set filesrc 'location'
> > > source.set_property('location', '/video/drake-hblings.mp4')
> > >
> > > # GST start pipeline
> > > p1.set_state(Gst.State.PLAYING)
> > >
> > >
> > >
> > > #KEYBOARD - detect key press, and launch GST PlayVideo functions
> > > orig_settings = termios.tcgetattr(sys.stdin)
> > >
> > > tty.setcbreak(sys.stdin)
> > > x = 0
> > > while x != chr(27): # ESC
> > > x=sys.stdin.read(1)[0]
> > > print("You pressed", x)
> > > if x == "1":
> > > PlayVideo1()
> > > if x == "2":
> > > PlayVideo2()
> > >
> > >
> > > termios.tcsetattr(sys.stdin, termios.TCSADRAIN, orig_settings)
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > gstreamer-devel at lists.freedesktop.org
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
More information about the gstreamer-devel
mailing list