High CPU consumption

[Michael Gruner]

Hello Matteo

OpenCV video capture will, sadly, copy the frame when taken out of the appsink. This is likely one of the causes of the high CPU consumption. I have proposed a PR to OpenCV in order to fix this, which they reviewed but I haven't gotten the time to continue. You may find the PR here:

https://github.com/opencv/opencv/pull/18377 <https://github.com/opencv/opencv/pull/18377>

If you'd like to try this out, you'll need to build OpenCV from scratch. Here are some instructions:

https://developer.ridgerun.com/wiki/index.php?title=Compiling_OpenCV_from_Source <https://developer.ridgerun.com/wiki/index.php?title=Compiling_OpenCV_from_Source>

Make sure you choose the correct branch and that you enabled GStreamer before launching the build, because it takes a few hours. This branch is now a bit behind master, but should be easy enough to update. I may do it myself in the next few days.

Please find some extra answers inline.

Michael
www.ridgerun.com

> On 15 Oct 2021, at 08:44, Matteo Foglio via gstreamer-devel <gstreamer-devel at lists.freedesktop.org> wrote:
> 
> Good morning,
> 
> I am trying to decode several video streams using GStreamer and Python. I can't tell you how much I will appreciate your help!
> 
> Here's my code:
> 
> self.video_capture = cv2.VideoCapture(self.pipeline, cv2.CAP_GSTREAMER)
> if self.video_capture.isOpened() is False:
>     raise Error
> while True:
>     status, image = self.video_capture.read()
>     # do slow stuff
> 
> I am encountering the following issue:
> 
> Option 1
> If I use the pipeline:
> self.pipeline = f"rtspsrc location={self.url} latency=10 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink max-buffers=1 drop=true"
> 
> I get extremely high CPU consumption. Also, I am probably decoding more frames than the number my application can actually consumes.
> 

Sadly, you are receiving a live compressed feed, which means you cannot drop frames before decoding. If you do so your stream may become momentarily corrupted. You can only drop after decoding. Hopefully you can lower the CPU consumption avoiding the memory copy mentioned above.

> 
> Option 2
> If I use this pipeline
> self.pipeline = f"rtspsrc location={self.url} latency=10 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink max-buffers=1 drop=false"
> 
> After a few hours the application is returning me old frames. E.g. at 9am my application was receiving frames at nighttime (from the night before).
> 

Yeah, likely everything is being queued somewhere. As you said, sounds like your app is consuming at a lower rate than you are receiving RTSP frames. You need to drop somewhere in your pipe. Maybe you can try setting "qos=true"  in the appsink, which will monitor late buffers and ask the decoder to start dropping buffers as the appsink can't keep up with the pace.

> 
> Is this the expected behavior? How can I solve this problem?
> 
> Thank you a lot,
> 
> Matteo
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20211015/8a165d64/attachment-0001.htm>