How do I correctly play video with gstreamer in real-time without delay? (problem using queue)

[michaelhe]

Hello,

I am working on OpenCV with GStreamer, using VideoCapture and appsink to
acquire frame from H265 videos. Here is my pipeline:

filesrc location=<path/to/mp4> ! qtdemux ! h265parse ! omxh265dec
skip-frames=1 ! queue ! nvvidconv ! video/x-raw, format=(string)I420 ! queue
leaky=upstream max-size-buffers=25 ! videoconvert ! video/x-raw,format=BGR !
appsink sync=True

My hardware in Jetson nano and I know it has a weak CPU, so videoconvert
should be the bottleneck in this pipeline. I put a queue before videoconvert
so I wish gstreamer can abandon frames when the processing speed can't catch
up the video speed, so I can receive images and display it in real-time. 
Without queue, the pipeline can process images with ~8FPS, thus can play it
in about 1/3 speed. 
With queue, I expected it can drop frames when queue is full, and can still
process frames in 8 FPS. But I got very slow processing speed, and only
first few (depending on max-size-buffer) frames can be received, than
getting slower and slower, and than hang on there.I have tried setting
max-size-buffers to larger number, but it can only plays slightly more
images before getting hanged but not solving the problem. 
I guess something wrong happened when the queue is full, but I can't figure
out why?

Interesting, when I am reading rtsp stream using following pipeline:
rtspsrc location=<path/to/rtsp/address>  latency=0 drop-on-latency=true !
rtph265depay ! h265parse !  omxh265dec skip-frames=1 ! nvvidconv !
video/x-raw, format=(string)I420  ! queue leaky=upstream max-size-buffers=25
! videoconvert ! video/x-raw,format=BGR ! appsink sync=True

Almost the same except rtph265depay and qtdemux, but rtsp stream works very
smoothly without delay? I wonder why queue works well only on second
pipeline but not first?

Please tell me if my question is not clear, or I need to provide more
informations.
Thanks

Michael





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/