GStreamer RTSP Server pipeline using nvh264enc, Windows 10, multiple threads

[Yurkanin, David J CTR USN NAWCAD NJ (USA)]

HELLO,

I have created a Windows 10 application that acquires camera video feeds, overlays them with textual information, then outputs them using the GStreamer RTSP Server API.  There is an acquisition thread for each camera feed (OpenCV/GStreamer), main thread for textual overlay (OpenGL), and separate threads to stream out each overlayed video each running its own RTSP Server. Here is the GStreamer factory pipeline used to stream outward:

"( appsrc name=mysrc ! videoconvert ! nvh264enc ! video/x-h264,profile=high ! rtph264pay name=pay0 pt=96 )"

Now when handling 5 streams, and NO streams demanded, no streams are going out and this pipeline is not processed.  The display of the overlayed video streams (using GLFW) are showing at 30 frames per second (FPS), all overlayed feeds look good.  As each stream is demanded (using Milestone VMS) over the network the RTSP threads begin processing, CPU usage goes up incrementally and the FPS goes down, depending on the computer and its installed NVidia GPU (e.g., for a Intel i-7 w/ Quadro RTX 4000 the FPS only goes down after the 3rd stream is demanded, while on a Dual Xeon w/ A4000 the FPS goes down with the first stream demanded).

So I am trying to understand how nvh264 behaves with multiple threads calling into the GPU hardware encoder, and perhaps setting of its properties to help with this problem.  Not sure if the other pipeline elements (rtph264pay?) are involved in the problem.  Placing of queue2 after the nvh264enc library element for example increased the FPS by about 5 units (on the Dual Xeon, no CPU increase):

"( appsrc name=mysrc ! videoconvert ! nvh264enc ! queue2 max-size buffers=400 ! video/x-h264,profile=high ! rtph264pay name=pay0 pt=96 )"

I also noticed the CPU usage goes up with each stream demanded on some computers, along with the GPU usage as shown in Task Manager.  Does the RTSP server use Windows resources (dlls) that run on the CPU, or does nvh264 or some of the other pipeline elements?

Thanks for any help

Dave Y.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230912/e01e2b40/attachment-0001.htm>