Problem when Stopping then Restarting Pipeline in Python (Video only)
whoman at gmail.com
whoman at gmail.com
Wed Jul 8 22:28:32 UTC 2020
On Wed, Jul 8, 2020 at 3:16 PM Nicolas Dufresne <nicolas at ndufresne.ca>
wrote:
>
>
> Le mer. 8 juill. 2020 18 h 00, whoman at gmail.com <whoman at gmail.com> a
> écrit :
>
>> HI Everyone,
>> I have a simple gstreamer python script that works perfectly on intel
>> with Hardware H264 decoding, however when I
>> try to port this script to ARM (specifically Rockchip RK3328), it has a
>> strange problem.
>>
>> The combination of Rockchip HW decoder and Rockchip Sink causes an error
>> whenever I attempt to change the 'Source' and restart the pipeline
>>
>
> These elements based on vendor drivers are maintained by Rockchip on their
> GitHub repository. Please report the issue there.
>
They're currently only maintaining for RK3228 and RK3399 chipsets for these
GST elements on GITHUB, however I will report the issue there and see if
they can assist.
>
> Meanwhile, development release of GStreamer now includes support for
> rk3288 mainline Linux drivers, feel free to give it a try and report.
>
I'm using RK3328. Does the development release also support RK3328
mainline drivers?
Thanks so much for your time
>
>
>> The python script does the following:
>>
>> 1. Sets up a pipeline
>> 2. Listens to keyboard input for either 1 or 2 (number keys)
>> 3. On Key press,
>> A. the pipeline is paused (READY)
>> B. the 'location' for filesrc is changed
>> C. the pipeline is set back to PLAYING
>>
>> When the pipeline is set to PLAYING for the 2nd time, the program
>> crashes, when using ROckchip's HW decoder and sink
>>
>> My script for INTEL sets up the following pipeline:
>>
>> Source: filesrc (h264 video)
>> Demux: qtdemux
>> parse: h264parse
>> decoder: vaapidec (intel hw h264 decoder)
>> videoconvert: videoconvert
>> sink: xvimagesink
>>
>> ...with a pad listener to dynamically add the link between Demux and
>> Parse once a Source is selected.
>>
>> This pipeline works fine. Changing filesrc and restarting the pipeline
>> multiple times works perfectly.
>>
>>
>> My script for ARM (rockchip) sets up the following pipeline (only changes
>> are Decoder and Sink):
>>
>> Source: filesrc (h264 video)
>> Demux: qtdemux
>> parse: h264parse
>> decoder: mppvideodec (rockchip hw h264 decoder)
>> videoconvert: videoconvert
>> sink: kmssink or rkximagesink (rockchip video sinks)
>>
>> The pipeline works fine when the first video is selected and played,
>> however...
>>
>> Whenever I try to change the SOURCE and then restart the pipeline (2nd
>> time), I receive an error when changing the pipeline's state back to PLAYING
>>
>> -------
>> (example.py:1024): GStreamer-CRITICAL **: gst_caps_is_strictly_equal:
>> assertion 'GST_IS_CAPS (caps1)' failed
>>
>> ** (example.py:1024): CRITICAL **: gst_video_codec_state_unref: assertion
>> 'state->ref_count > 0' failed
>>
>> ** (example.py:1024): CRITICAL **: gst_mpp_allocator_start: assertion
>> 'size != 0' failed
>> -------
>>
>>
>> with GST_DEBUG=3:
>> -------
>> 0:00:08.584026172 1064 0x55932be4f0 ERROR mppvideodec
>> gstmppvideodec.c:693:gst_mpp_video_dec_handle_frame:<decoder> can't process
>> this frame
>> 0:00:08.589703023 1064 0x55932afe30 WARN basesrc
>> gstbasesrc.c:3491:gst_base_src_start_complete:<source> pad not activated yet
>> 0:00:08.590945554 1064 0x7fac014540 WARN qtdemux
>> qtdemux.c:7031:qtdemux_parse_container:<demux> length too long (1507328 >
>> 27)
>> 0:00:08.591528027 1064 0x7fac014540 WARN qtdemux
>> qtdemux.c:2899:qtdemux_parse_trex:<demux> failed to find fragment defaults
>> for stream 1
>> 0:00:08.592839977 1064 0x7fac014540 WARN qtdemux
>> qtdemux.c:8766:qtdemux_parse_segments:<demux> Segment 0 extends to
>> 0:03:44.891332000 past the end of the file duration 0:03:44.766207000 it
>> will be truncated
>> 0:00:08.593700415 1064 0x7fac014540 WARN basesrc
>> gstbasesrc.c:2400:gst_base_src_update_length:<source> processing at or past
>> EOS
>> Received new pad 'video_0' from 'demux'
>> video/x-h264
>> Link succeeded (type 'video/x-h264')
>> 0:00:08.601256230 1064 0x7fac014540 FIXME videodecoder
>> gstvideodecoder.c:945:gst_video_decoder_drain_out:<decoder> Sub-class
>> should implement drain()
>>
>> (example.py:1064): GStreamer-CRITICAL **: gst_caps_is_strictly_equal:
>> assertion 'GST_IS_CAPS (caps1)' failed
>>
>> ** (example.py:1064): CRITICAL **: gst_video_codec_state_unref: assertion
>> 'state->ref_count > 0' failed
>> 0:00:08.621745744 1064 0x7fac014540 FIXME videodecoder
>> gstvideodecoder.c:945:gst_video_decoder_drain_out:<decoder> Sub-class
>> should implement drain()
>> mpp_buf_slot: found info change ready set without internal info change
>> Segmentation fault
>> -------
>>
>>
>> Note: If I use the libav SW decoder avdec_h264 instead of mppvideodec,
>> everything works fine and I do not get any errors when changing sources
>>
>> I understand that there may be some issues with mppvideodec causing this
>> problem, but since rockchip is mostly focusing on 3228 and 3399 on GITHUB
>> (not RK3328), My best potential option for a solution is to find a
>> workaround within my scripts pipeline code/logic.
>>
>>
>> My Question:
>> Is there any chance that flushing the buffer/caps between STOP / START of
>> the pipeline could get me past this issue?
>> Any advice / code to help me make this script function with Rockchip HW
>> decode (mppvideodec) would be greately appreacited.
>>
>> FYI, if I use mppvideodec with a standard Gstreamer sink (fbdevsink,
>> xvimagesink, etc) this problem does not occur (however the video is
>> choppy/unusable)
>> This problem only occurs when using the rockchip decoder (mppvideodec)
>> with a rockchip video sink (kmssink or rkximagesink)
>>
>>
>> I beleive this problem may be solved with proper caps/buffer flushing
>> in-between source changes, but I'm not
>> sure how to properly acheive this within the Gstreamer environment.
>>
>> I've read about what I think may be potential solutions / workarounds....
>> for example with the older GST (.10) I found this:
>>
>> https://gstreamer-devel.narkive.com/EpGe0YyN/gst-devel-assertion-gst-is-caps-caps-failed
>>
>>
>> Does anyone know how I might be able to modify this python script to get
>> around this issue?
>>
>> Thanks for your time
>>
>> ---
>> example.py:
>> ---
>>
>> import tty
>> import sys
>> import termios
>> import gi
>> import subprocess
>> import os, signal
>> import time
>> from time import sleep
>> from gi.repository import GObject
>> gi.require_version('Gst', '1.0')
>> from gi.repository import Gst
>> from gi.repository import GstController
>> GObject.threads_init()
>> Gst.init(sys.argv)
>>
>>
>>
>> #GST PIPELINE
>> #GST - create the elements
>> source = Gst.ElementFactory.make("filesrc", "source")
>> demux = Gst.ElementFactory.make("qtdemux", "demux")
>> parse = Gst.ElementFactory.make("h264parse", "parse")
>> decoder = Gst.ElementFactory.make("mppvideodec", "decoder")
>> video_convert = Gst.ElementFactory.make("videoconvert", "videoconvert")
>> sink = Gst.ElementFactory.make("kmssink", "sink")
>>
>> #GST - create the empty pipeline
>> p1 = Gst.Pipeline.new("test-pipeline")
>>
>> if not p1 or not source or not video_convert or not sink:
>> print("ERROR: Not all elements could be created")
>> sys.exit(1)
>>
>> #GST - add and link elements in pipeline
>> p1.add(source)
>> p1.add(demux)
>> p1.add(parse)
>> p1.add(decoder)
>> p1.add(video_convert)
>> p1.add(sink)
>>
>> if not source.link(demux):
>> print("ERROR: Could not link source to demux")
>> sys.exit(1)
>> #if not demux.link(parse):
>> # print("ERROR: Could not link demux to parse")
>> # sys.exit(1)
>> if not parse.link(decoder):
>> print("ERROR: Could not link parse to decoder")
>> sys.exit(1)
>> if not decoder.link(video_convert):
>> print("ERROR: Could not link decoder to video_convert")
>> # sys.exit(1)
>> if not video_convert.link(sink):
>> print("ERROR: Could not link scaler to sink")
>> sys.exit(1)
>>
>> # modify the source's location property
>> source.set_property("location", "/video/g62.mp4")
>>
>> ret = p1.get_state(1)
>> print ret.state
>>
>>
>>
>> #GST - handler for the pad-added signal
>> def demux_on_pad_added(src, new_pad):
>> if new_pad.get_name() == "video_0":
>> print(
>> "Received new pad '{0:s}' from '{1:s}'".format(
>> new_pad.get_name(),
>> src.get_name()))
>>
>> # attempt the link
>> sink_pad = parse.get_static_pad("sink")
>> new_pad_caps = new_pad.get_current_caps()
>> new_pad_struct = new_pad_caps.get_structure(0)
>> new_pad_type = new_pad_struct.get_name()
>> print(new_pad_type)
>> ret = new_pad.link(sink_pad)
>> if not ret == Gst.PadLinkReturn.OK:
>> print("Type is '{0:s}' but link failed".format(new_pad_type))
>> else:
>> print("Link succeeded (type '{0:s}')".format(new_pad_type))
>> return
>> return
>>
>> # connect to the pad-added signal
>> demux.connect("pad-added", demux_on_pad_added)
>>
>>
>>
>> #GST - Puase playback, set "location", and begin playback
>> def PlayVideo1():
>> # GST stop pipeline
>> p1.set_state(Gst.State.READY)
>>
>> # GST set filesrc 'location'
>> source.set_property('location', '/video/g62s.mp4')
>>
>> # GST start pipeline
>> p1.set_state(Gst.State.PLAYING)
>>
>>
>> def PlayVideo2():
>> # GST stop pipeline
>> p1.set_state(Gst.State.READY)
>>
>> # GST set filesrc 'location'
>> source.set_property('location', '/video/drake-hblings.mp4')
>>
>> # GST start pipeline
>> p1.set_state(Gst.State.PLAYING)
>>
>>
>>
>> #KEYBOARD - detect key press, and launch GST PlayVideo functions
>> orig_settings = termios.tcgetattr(sys.stdin)
>>
>> tty.setcbreak(sys.stdin)
>> x = 0
>> while x != chr(27): # ESC
>> x=sys.stdin.read(1)[0]
>> print("You pressed", x)
>> if x == "1":
>> PlayVideo1()
>> if x == "2":
>> PlayVideo2()
>>
>>
>> termios.tcsetattr(sys.stdin, termios.TCSADRAIN, orig_settings)
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20200708/8c669e51/attachment.htm>
More information about the gstreamer-devel
mailing list