How to provide data to RTSPClient wihout latency? [appsrc, buffering]

[Michael Gruner]

Hi Andrey, 

You can use the max-bytes, max-buffers or max-time along with the leaky-type properties in the appsrc to avoid queueing data. For example `appsrc max-buffers=3 leaky-type=downstream` will only keep the latest 3 camera buffers and discard older ones. Also, if your buffers come from a camera set the `is-live=true` property as well. If I recall correctly, that'll let the pipeline know you're pushing data from a live source and will not perform this delay (don't quote me on that :) ). An initial PTS of 1000 hours is expected in video encoders, and is to allow reordering of frames where the first DTS can be lower than the first PTS.

Michael

> On 25 Aug 2023, at 15:37, Andrey Sotnikov via gstreamer-devel <gstreamer-devel at lists.freedesktop.org> wrote:
> 
> I found out the cause of the delay, and it's very bizarre. The delay is caused by appsink0 that was added to the media pipeline by RTSPClient. I guess this AppSink sends RTP packets. When the first data buffer arrives to appsink0, it has the timestamp equal to 1000 hours. GstBaseSink::segment at the same time has a timestamp of seemingly random few seconds earlier than 1000 hours. appsink0 blocks the thread for this number of seconds. This blocks gst_queue_lock. In turn, this blocks gst_base_src_loop, when the latter calls gst_pad_peer_query.
> 
> Can somebody explain the logic of what is going on and why?
> 
> 23.08.2023 23:10, Andrey Sotnikov пишет:
>> Hi, dear GStreamer community,
>> 
>> I am tired of parsing GStreamer's source code to understand how everything works and how to solve my problem.
>> 
>> My company manufactures cameras. I am trying to create an application that shares the data from these cameras over RTSP. Here is the launch string for GstRTSPMediaFactory: "( appsrc name=ourcamera ! queue ! x265enc speed-preset=5 tune=4 option-string=colormatrix=gbr:lossless=true ! rtph265pay name=pay0 pt=96 )". When I receive a frame from my camera, I push it to appsrc. The problem is, in the beginning, appsrc buffers all the data pushed, while the connection is being established. When the real data transfer starts, this buffer is not being discarded, leading to a latency of dozens of seconds. I would love to push only after RTSPClient is ready to pop data, but how?
>> 
>> I was trying to figure what was going on, and here are my discoveries. When the pipeline created for an RTSPCLient is in the play mode, gst_base_src_loop checks if reconfigure is required. For some reason, it finds out it is, and calls gst_base_src_negotiate_unlocked. The latter hangs on gst_base_src_prepare_allocation. This function hangs on gst_pad_query called on queue:sink and query is a new allocation. It hangs only because, for some reason, gst_queue_loop is not being called. While gst_queue_loop is postponed, my application continues pushing data. When finally gst_queue_loop is called, appsrc has a buffer of up to a hundred frames accumulated. So what gst_queue_loop is waiting for?
>> 
>> I tried removing gst_queue_loop all together, but not only the latency problem remains, the pipeline reports it is not configured properly and asks to add a queue.
>