Problem when Stopping then Restarting Pipeline in Python (Video only)
whoman at gmail.com
whoman at gmail.com
Wed Jul 8 22:54:44 UTC 2020
On Wed, Jul 8, 2020 at 3:46 PM Nicolas Dufresne <nicolas at ndufresne.ca>
wrote:
> Le mercredi 08 juillet 2020 à 15:28 -0700, whoman at gmail.com a écrit :
> >
> >
> > On Wed, Jul 8, 2020 at 3:16 PM Nicolas Dufresne <nicolas at ndufresne.ca>
> wrote:
> > >
> > > Le mer. 8 juill. 2020 18 h 00, whoman at gmail.com <whoman at gmail.com> a
> écrit :
> > > > HI Everyone,
> > > > I have a simple gstreamer python script that works perfectly on
> intel with Hardware H264 decoding, however when I
> > > > try to port this script to ARM (specifically Rockchip RK3328), it
> has a strange problem.
> > > >
> > > > The combination of Rockchip HW decoder and Rockchip Sink causes an
> error whenever I attempt to change the 'Source' and restart the pipeline
> > >
> > > These elements based on vendor drivers are maintained by Rockchip on
> their GitHub repository. Please report the issue there.
> > >
> >
> > They're currently only maintaining for RK3228 and RK3399 chipsets for
> these GST elements on GITHUB, however I will report the issue there and see
> if they can assist.
>
> Interesting, I didn't know they had reduce maintenance.
>
> >
> >
> > > Meanwhile, development release of GStreamer now includes support for
> rk3288 mainline Linux drivers, feel free to give it a try and report.
> > >
> >
> > I'm using RK3328. Does the development release also support RK3328
> mainline drivers?
>
> Ah, sorry for this miss read, numbers are so similar. RK3328 seems to
> share are lot with RK3399. Jonas Karlman has been working on enabling
> this variant, I can see that MPEG2 support is mainline, but I believe a
> lot more is incoming. You'll find him on IRC (usually un libreelec
> channel) as Kwiboo.
>
> Okay, I will first post on Rockchip's Github and if that doesn't work out,
I'll pursue mainline, and see about contacting Kwiboo.
Thanks so much... I really appreciate your pointing me in the right
direction.
> >
> > Thanks so much for your time
> >
> >
> > > > The python script does the following:
> > > >
> > > > 1. Sets up a pipeline
> > > > 2. Listens to keyboard input for either 1 or 2 (number keys)
> > > > 3. On Key press,
> > > > A. the pipeline is paused (READY)
> > > > B. the 'location' for filesrc is changed
> > > > C. the pipeline is set back to PLAYING
> > > >
> > > > When the pipeline is set to PLAYING for the 2nd time, the program
> crashes, when using ROckchip's HW decoder and sink
> > > >
> > > > My script for INTEL sets up the following pipeline:
> > > >
> > > > Source: filesrc (h264 video)
> > > > Demux: qtdemux
> > > > parse: h264parse
> > > > decoder: vaapidec (intel hw h264 decoder)
> > > > videoconvert: videoconvert
> > > > sink: xvimagesink
> > > >
> > > > ...with a pad listener to dynamically add the link between Demux and
> Parse once a Source is selected.
> > > >
> > > > This pipeline works fine. Changing filesrc and restarting the
> pipeline multiple times works perfectly.
> > > >
> > > >
> > > > My script for ARM (rockchip) sets up the following pipeline (only
> changes are Decoder and Sink):
> > > >
> > > > Source: filesrc (h264 video)
> > > > Demux: qtdemux
> > > > parse: h264parse
> > > > decoder: mppvideodec (rockchip hw h264 decoder)
> > > > videoconvert: videoconvert
> > > > sink: kmssink or rkximagesink (rockchip video sinks)
> > > >
> > > > The pipeline works fine when the first video is selected and played,
> however...
> > > >
> > > > Whenever I try to change the SOURCE and then restart the pipeline
> (2nd time), I receive an error when changing the pipeline's state back to
> PLAYING
> > > >
> > > > -------
> > > > (example.py:1024): GStreamer-CRITICAL **:
> gst_caps_is_strictly_equal: assertion 'GST_IS_CAPS (caps1)' failed
> > > >
> > > > ** (example.py:1024): CRITICAL **: gst_video_codec_state_unref:
> assertion 'state->ref_count > 0' failed
> > > >
> > > > ** (example.py:1024): CRITICAL **: gst_mpp_allocator_start:
> assertion 'size != 0' failed
> > > > -------
> > > >
> > > >
> > > > with GST_DEBUG=3:
> > > > -------
> > > > 0:00:08.584026172 1064 0x55932be4f0 ERROR mppvideodec
> gstmppvideodec.c:693:gst_mpp_video_dec_handle_frame:<decoder> can't process
> this frame
> > > > 0:00:08.589703023 1064 0x55932afe30 WARN basesrc
> gstbasesrc.c:3491:gst_base_src_start_complete:<source> pad not activated yet
> > > > 0:00:08.590945554 1064 0x7fac014540 WARN qtdemux
> qtdemux.c:7031:qtdemux_parse_container:<demux> length too long (1507328 >
> 27)
> > > > 0:00:08.591528027 1064 0x7fac014540 WARN qtdemux
> qtdemux.c:2899:qtdemux_parse_trex:<demux> failed to find fragment defaults
> for stream 1
> > > > 0:00:08.592839977 1064 0x7fac014540 WARN qtdemux
> qtdemux.c:8766:qtdemux_parse_segments:<demux> Segment 0 extends to
> 0:03:44.891332000 past the end of the file duration 0:03:44.766207000 it
> will be truncated
> > > > 0:00:08.593700415 1064 0x7fac014540 WARN basesrc
> gstbasesrc.c:2400:gst_base_src_update_length:<source> processing at or past
> EOS
> > > > Received new pad 'video_0' from 'demux'
> > > > video/x-h264
> > > > Link succeeded (type 'video/x-h264')
> > > > 0:00:08.601256230 1064 0x7fac014540 FIXME videodecoder
> gstvideodecoder.c:945:gst_video_decoder_drain_out:<decoder> Sub-class
> should implement drain()
> > > >
> > > > (example.py:1064): GStreamer-CRITICAL **:
> gst_caps_is_strictly_equal: assertion 'GST_IS_CAPS (caps1)' failed
> > > >
> > > > ** (example.py:1064): CRITICAL **: gst_video_codec_state_unref:
> assertion 'state->ref_count > 0' failed
> > > > 0:00:08.621745744 1064 0x7fac014540 FIXME videodecoder
> gstvideodecoder.c:945:gst_video_decoder_drain_out:<decoder> Sub-class
> should implement drain()
> > > > mpp_buf_slot: found info change ready set without internal info
> change
> > > > Segmentation fault
> > > > -------
> > > >
> > > >
> > > > Note: If I use the libav SW decoder avdec_h264 instead of
> mppvideodec, everything works fine and I do not get any errors when
> changing sources
> > > >
> > > > I understand that there may be some issues with mppvideodec causing
> this problem, but since rockchip is mostly focusing on 3228 and 3399 on
> GITHUB (not RK3328), My best potential option for a solution is to find a
> workaround within my scripts pipeline code/logic.
> > > >
> > > >
> > > > My Question:
> > > > Is there any chance that flushing the buffer/caps between STOP /
> START of the pipeline could get me past this issue?
> > > > Any advice / code to help me make this script function with Rockchip
> HW decode (mppvideodec) would be greately appreacited.
> > > >
> > > > FYI, if I use mppvideodec with a standard Gstreamer sink (fbdevsink,
> xvimagesink, etc) this problem does not occur (however the video is
> choppy/unusable)
> > > > This problem only occurs when using the rockchip decoder
> (mppvideodec) with a rockchip video sink (kmssink or rkximagesink)
> > > >
> > > >
> > > > I beleive this problem may be solved with proper caps/buffer
> flushing in-between source changes, but I'm not
> > > > sure how to properly acheive this within the Gstreamer environment.
> > > >
> > > > I've read about what I think may be potential solutions /
> workarounds....
> > > > for example with the older GST (.10) I found this:
> > > >
> https://gstreamer-devel.narkive.com/EpGe0YyN/gst-devel-assertion-gst-is-caps-caps-failed
> > > >
> > > >
> > > > Does anyone know how I might be able to modify this python script to
> get around this issue?
> > > >
> > > > Thanks for your time
> > > >
> > > > ---
> > > > example.py:
> > > > ---
> > > >
> > > > import tty
> > > > import sys
> > > > import termios
> > > > import gi
> > > > import subprocess
> > > > import os, signal
> > > > import time
> > > > from time import sleep
> > > > from gi.repository import GObject
> > > > gi.require_version('Gst', '1.0')
> > > > from gi.repository import Gst
> > > > from gi.repository import GstController
> > > > GObject.threads_init()
> > > > Gst.init(sys.argv)
> > > >
> > > >
> > > >
> > > > #GST PIPELINE
> > > > #GST - create the elements
> > > > source = Gst.ElementFactory.make("filesrc", "source")
> > > > demux = Gst.ElementFactory.make("qtdemux", "demux")
> > > > parse = Gst.ElementFactory.make("h264parse", "parse")
> > > > decoder = Gst.ElementFactory.make("mppvideodec", "decoder")
> > > > video_convert = Gst.ElementFactory.make("videoconvert",
> "videoconvert")
> > > > sink = Gst.ElementFactory.make("kmssink", "sink")
> > > >
> > > > #GST - create the empty pipeline
> > > > p1 = Gst.Pipeline.new("test-pipeline")
> > > >
> > > > if not p1 or not source or not video_convert or not sink:
> > > > print("ERROR: Not all elements could be created")
> > > > sys.exit(1)
> > > >
> > > > #GST - add and link elements in pipeline
> > > > p1.add(source)
> > > > p1.add(demux)
> > > > p1.add(parse)
> > > > p1.add(decoder)
> > > > p1.add(video_convert)
> > > > p1.add(sink)
> > > >
> > > > if not source.link(demux):
> > > > print("ERROR: Could not link source to demux")
> > > > sys.exit(1)
> > > > #if not demux.link(parse):
> > > > # print("ERROR: Could not link demux to parse")
> > > > # sys.exit(1)
> > > > if not parse.link(decoder):
> > > > print("ERROR: Could not link parse to decoder")
> > > > sys.exit(1)
> > > > if not decoder.link(video_convert):
> > > > print("ERROR: Could not link decoder to video_convert")
> > > > # sys.exit(1)
> > > > if not video_convert.link(sink):
> > > > print("ERROR: Could not link scaler to sink")
> > > > sys.exit(1)
> > > >
> > > > # modify the source's location property
> > > > source.set_property("location", "/video/g62.mp4")
> > > >
> > > > ret = p1.get_state(1)
> > > > print ret.state
> > > >
> > > >
> > > >
> > > > #GST - handler for the pad-added signal
> > > > def demux_on_pad_added(src, new_pad):
> > > > if new_pad.get_name() == "video_0":
> > > > print(
> > > > "Received new pad '{0:s}' from '{1:s}'".format(
> > > > new_pad.get_name(),
> > > > src.get_name()))
> > > >
> > > > # attempt the link
> > > > sink_pad = parse.get_static_pad("sink")
> > > > new_pad_caps = new_pad.get_current_caps()
> > > > new_pad_struct = new_pad_caps.get_structure(0)
> > > > new_pad_type = new_pad_struct.get_name()
> > > > print(new_pad_type)
> > > > ret = new_pad.link(sink_pad)
> > > > if not ret == Gst.PadLinkReturn.OK:
> > > > print("Type is '{0:s}' but link failed".format(new_pad_type))
> > > > else:
> > > > print("Link succeeded (type '{0:s}')".format(new_pad_type))
> > > > return
> > > > return
> > > >
> > > > # connect to the pad-added signal
> > > > demux.connect("pad-added", demux_on_pad_added)
> > > >
> > > >
> > > >
> > > > #GST - Puase playback, set "location", and begin playback
> > > > def PlayVideo1():
> > > > # GST stop pipeline
> > > > p1.set_state(Gst.State.READY)
> > > >
> > > > # GST set filesrc 'location'
> > > > source.set_property('location', '/video/g62s.mp4')
> > > >
> > > > # GST start pipeline
> > > > p1.set_state(Gst.State.PLAYING)
> > > >
> > > >
> > > > def PlayVideo2():
> > > > # GST stop pipeline
> > > > p1.set_state(Gst.State.READY)
> > > >
> > > > # GST set filesrc 'location'
> > > > source.set_property('location', '/video/drake-hblings.mp4')
> > > >
> > > > # GST start pipeline
> > > > p1.set_state(Gst.State.PLAYING)
> > > >
> > > >
> > > >
> > > > #KEYBOARD - detect key press, and launch GST PlayVideo functions
> > > > orig_settings = termios.tcgetattr(sys.stdin)
> > > >
> > > > tty.setcbreak(sys.stdin)
> > > > x = 0
> > > > while x != chr(27): # ESC
> > > > x=sys.stdin.read(1)[0]
> > > > print("You pressed", x)
> > > > if x == "1":
> > > > PlayVideo1()
> > > > if x == "2":
> > > > PlayVideo2()
> > > >
> > > >
> > > > termios.tcsetattr(sys.stdin, termios.TCSADRAIN, orig_settings)
> > > > _______________________________________________
> > > > gstreamer-devel mailing list
> > > > gstreamer-devel at lists.freedesktop.org
> > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20200708/d3cdc450/attachment-0001.htm>
More information about the gstreamer-devel
mailing list