Problem when Stopping then Restarting Pipeline in Python (Video only)

Wed Jul 8 20:54:50 UTC 2020

HI Everyone,
I have a simple gstreamer python script that works perfectly on intel with
Hardware H264 decoding, however when I
try to port this script to ARM (specifically Rockchip RK3328), it has a
strange problem.

The combination of Rockchip HW decoder and Rockchip Sink causes an error
whenever I attempt to change the 'Source' and restart the pipeline

The python script does the following:

1.  Sets up a pipeline
2.  Listens to keyboard input for either 1 or 2 (number keys)
3.  On Key press,
      A.  the pipeline is paused (READY)
      B.  the 'location' for filesrc is changed
      C.  the pipeline is set back to PLAYING

When the pipeline is set to PLAYING for the 2nd time, the program crashes,
when using ROckchip's HW decoder and sink

My script for INTEL sets up the following pipeline:

Source: filesrc (h264 video)
Demux: qtdemux
parse: h264parse
decoder: vaapidec (intel hw h264 decoder)
videoconvert: videoconvert
sink: xvimagesink

...with a pad listener to dynamically add the link between Demux and Parse
once a Source is selected.

This pipeline works fine.  Changing filesrc and restarting the pipeline
multiple times works perfectly.

My script for ARM (rockchip) sets up the following pipeline (only changes
are Decoder and Sink):

Source: filesrc (h264 video)
Demux: qtdemux
parse: h264parse
decoder: mppvideodec (rockchip hw h264 decoder)
videoconvert: videoconvert
sink: kmssink or rkximagesink (rockchip video sinks)

The pipeline works fine when the first video is selected and played,
however...

Whenever I try to change the SOURCE and then restart the pipeline (2nd
time), I receive an error when changing the pipeline's state back to PLAYING

-------
(example.py:1024): GStreamer-CRITICAL **: gst_caps_is_strictly_equal:
assertion 'GST_IS_CAPS (caps1)' failed

** (example.py:1024): CRITICAL **: gst_video_codec_state_unref: assertion
'state->ref_count > 0' failed

** (example.py:1024): CRITICAL **: gst_mpp_allocator_start: assertion 'size
!= 0' failed
-------

with GST_DEBUG=3:
-------
0:00:08.584026172  1064   0x55932be4f0 ERROR            mppvideodec
gstmppvideodec.c:693:gst_mpp_video_dec_handle_frame:<decoder> can't process
this frame
0:00:08.589703023  1064   0x55932afe30 WARN                 basesrc
gstbasesrc.c:3491:gst_base_src_start_complete:<source> pad not activated yet
0:00:08.590945554  1064   0x7fac014540 WARN                 qtdemux
qtdemux.c:7031:qtdemux_parse_container:<demux> length too long (1507328 >
27)
0:00:08.591528027  1064   0x7fac014540 WARN                 qtdemux
qtdemux.c:2899:qtdemux_parse_trex:<demux> failed to find fragment defaults
for stream 1
0:00:08.592839977  1064   0x7fac014540 WARN                 qtdemux
qtdemux.c:8766:qtdemux_parse_segments:<demux> Segment 0  extends to
0:03:44.891332000 past the end of the file duration 0:03:44.766207000 it
will be truncated
0:00:08.593700415  1064   0x7fac014540 WARN                 basesrc
gstbasesrc.c:2400:gst_base_src_update_length:<source> processing at or past
EOS
Received new pad 'video_0' from 'demux'
video/x-h264
Link succeeded (type 'video/x-h264')
0:00:08.601256230  1064   0x7fac014540 FIXME           videodecoder
gstvideodecoder.c:945:gst_video_decoder_drain_out:<decoder> Sub-class
should implement drain()

(example.py:1064): GStreamer-CRITICAL **: gst_caps_is_strictly_equal:
assertion 'GST_IS_CAPS (caps1)' failed

** (example.py:1064): CRITICAL **: gst_video_codec_state_unref: assertion
'state->ref_count > 0' failed
0:00:08.621745744  1064   0x7fac014540 FIXME           videodecoder
gstvideodecoder.c:945:gst_video_decoder_drain_out:<decoder> Sub-class
should implement drain()
mpp_buf_slot: found info change ready set without internal info change
Segmentation fault
-------

Note: If I use the libav SW decoder avdec_h264 instead of mppvideodec,
everything works fine and I do not get any errors when changing sources

I understand that there may be some issues with mppvideodec causing this
problem, but since rockchip is mostly focusing on 3228 and 3399 on GITHUB
(not RK3328), My best potential option for a solution is to find a
workaround within my scripts pipeline code/logic.

My Question:
Is there any chance that flushing the buffer/caps between STOP / START of
the pipeline could get me past this issue?
Any advice / code to help me make this script function with Rockchip HW
decode (mppvideodec) would be greately appreacited.

FYI, if I use mppvideodec with a standard Gstreamer sink (fbdevsink,
xvimagesink, etc) this problem does not occur (however the video is
choppy/unusable)
This problem only occurs when using the rockchip decoder (mppvideodec) with
a rockchip video sink (kmssink or rkximagesink)

I beleive this problem may be solved with proper caps/buffer flushing
in-between source changes, but I'm not
sure how to properly acheive this within the Gstreamer environment.

I've read about what I think may be potential solutions / workarounds....
for example with the older GST (.10) I found this:
https://gstreamer-devel.narkive.com/EpGe0YyN/gst-devel-assertion-gst-is-caps-caps-failed

Does anyone know how I might be able to modify this python script to get
around this issue?

Thanks for your time

---
example.py:
---

import tty
import sys
import termios
import gi
import subprocess
import os, signal
import time
from time import sleep
from gi.repository import GObject
gi.require_version('Gst', '1.0')
from gi.repository import Gst
from gi.repository import GstController
GObject.threads_init()
Gst.init(sys.argv)

#GST PIPELINE
#GST - create the elements
source = Gst.ElementFactory.make("filesrc", "source")
demux = Gst.ElementFactory.make("qtdemux", "demux")
parse = Gst.ElementFactory.make("h264parse", "parse")
decoder = Gst.ElementFactory.make("mppvideodec", "decoder")
video_convert = Gst.ElementFactory.make("videoconvert", "videoconvert")
sink = Gst.ElementFactory.make("kmssink", "sink")

#GST - create the empty pipeline
p1 = Gst.Pipeline.new("test-pipeline")

if not p1 or not source or not  video_convert or not sink:
print("ERROR: Not all elements could be created")
sys.exit(1)

#GST - add and link elements in pipeline
p1.add(source)
p1.add(demux)
p1.add(parse)
p1.add(decoder)
p1.add(video_convert)
p1.add(sink)

if not source.link(demux):
print("ERROR: Could not link source to demux")
sys.exit(1)
#if not demux.link(parse):
# print("ERROR: Could not link demux to parse")
# sys.exit(1)
if not parse.link(decoder):
print("ERROR: Could not link parse to decoder")
sys.exit(1)
if not decoder.link(video_convert):
print("ERROR: Could not link decoder to video_convert")
# sys.exit(1)
if not video_convert.link(sink):
print("ERROR: Could not link scaler to sink")
sys.exit(1)

# modify the source's location property
source.set_property("location", "/video/g62.mp4")

ret = p1.get_state(1)
print ret.state

#GST - handler for the pad-added signal
def demux_on_pad_added(src, new_pad):
if new_pad.get_name() == "video_0":
print(
"Received new pad '{0:s}' from '{1:s}'".format(
new_pad.get_name(),
src.get_name()))

# attempt the link
sink_pad = parse.get_static_pad("sink")
new_pad_caps = new_pad.get_current_caps()
new_pad_struct = new_pad_caps.get_structure(0)
new_pad_type = new_pad_struct.get_name()
print(new_pad_type)
ret = new_pad.link(sink_pad)
if not ret == Gst.PadLinkReturn.OK:
print("Type is '{0:s}' but link failed".format(new_pad_type))
else:
print("Link succeeded (type '{0:s}')".format(new_pad_type))
return
return

# connect to the pad-added signal
demux.connect("pad-added", demux_on_pad_added)

#GST - Puase playback, set "location", and begin playback
def PlayVideo1():
# GST stop pipeline
p1.set_state(Gst.State.READY)

# GST set filesrc 'location'
source.set_property('location', '/video/g62s.mp4')

# GST start pipeline
p1.set_state(Gst.State.PLAYING)

def PlayVideo2():
# GST stop pipeline
p1.set_state(Gst.State.READY)

# GST set filesrc 'location'
source.set_property('location', '/video/drake-hblings.mp4')

# GST start pipeline
p1.set_state(Gst.State.PLAYING)

#KEYBOARD - detect key press, and launch GST PlayVideo functions
orig_settings = termios.tcgetattr(sys.stdin)

tty.setcbreak(sys.stdin)
x = 0
while x != chr(27): # ESC
    x=sys.stdin.read(1)[0]
    print("You pressed", x)
    if x == "1":
PlayVideo1()
    if x == "2":
PlayVideo2()

termios.tcsetattr(sys.stdin, termios.TCSADRAIN, orig_settings)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20200708/1c6a01de/attachment-0001.htm>