How to provide data to RTSPClient wihout latency? [appsrc, buffering]

[Thanh Do]

Hi Andrey,

How are you pushing data to the appsrc?  From the doc, it seems you should only push when it's requesting more data via the need-data signal.  You can maintain a buffer of your camera data separately and use that to feed into appsrc only when it's requesting through that signal.  Just my thoughts, might be worth a try.
________________________________
From: gstreamer-devel <gstreamer-devel-bounces at lists.freedesktop.org> on behalf of Andrey Sotnikov via gstreamer-devel <gstreamer-devel at lists.freedesktop.org>
Sent: Thursday, August 24, 2023 12:10 AM
To: gstreamer-devel at lists.freedesktop.org <gstreamer-devel at lists.freedesktop.org>
Cc: Andrey Sotnikov <usaonmonday at gmail.com>
Subject: How to provide data to RTSPClient wihout latency? [appsrc, buffering]

Hi, dear GStreamer community, I am tired of parsing GStreamer's source code to understand how everything works and how to solve my problem. My company manufactures cameras. I am trying to create an ap
External (gstreamer-devel at lists.freedesktop.org<mailto:gstreamer-devel at lists.freedesktop.org>)
  Report This Email<https://protection.inkyphishfence.com/report?id=dml2aW50L3RoYW5oLmRvQHZpdmludC5jb20vYmJjOGY4MjdiNjI2M2UxZDMzY2ZmOWFlMzA3YmI5ZTMvMTY5Mjg2NjAyNS40Ng==#key=24f6990f44a416f12414e98d46e78fd5>  FAQ<https://www.inky.com/banner-faq/>

Hi, dear GStreamer community,

I am tired of parsing GStreamer's source code to understand how everything works and how to solve my problem.

My company manufactures cameras. I am trying to create an application that shares the data from these cameras over RTSP. Here is the launch string for GstRTSPMediaFactory: "( appsrc name=ourcamera ! queue ! x265enc speed-preset=5 tune=4 option-string=colormatrix=gbr:lossless=true ! rtph265pay name=pay0 pt=96 )". When I receive a frame from my camera, I push it to appsrc. The problem is, in the beginning, appsrc buffers all the data pushed, while the connection is being established. When the real data transfer starts, this buffer is not being discarded, leading to a latency of dozens of seconds. I would love to push only after RTSPClient is ready to pop data, but how?

I was trying to figure what was going on, and here are my discoveries. When the pipeline created for an RTSPCLient is in the play mode, gst_base_src_loop checks if reconfigure is required. For some reason, it finds out it is, and calls gst_base_src_negotiate_unlocked. The latter hangs on gst_base_src_prepare_allocation. This function hangs on gst_pad_query called on queue:sink and query is a new allocation. It hangs only because, for some reason, gst_queue_loop is not being called. While gst_queue_loop is postponed, my application continues pushing data. When finally gst_queue_loop is called, appsrc has a buffer of up to a hundred frames accumulated. So what gst_queue_loop is waiting for?

I tried removing gst_queue_loop all together, but not only the latency problem remains, the pipeline reports it is not configured properly and asks to add a queue.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230824/3c22ffd9/attachment.htm>