Correct way of building RTSP pipelines

[valmirjunior0088]

Hello!

We are trying to use GStreamer (with the Rust bindings) to manage the media
feed coming from IP cameras. We are having some problems getting our
solution to work consistently, and with this thread we expect to have some
feedback as to how we can make it better.

Our application consists of opening a connection to the RTSP feed that the
camera has available. We open this connection early on and maintain it open
for as long as the application lives. This is because our application deals
with events coming from multiple sources, and we need the RTP packets to be
readily available as soon as events are triggered. The connection is only
interrupted when the camera is no longer available for any reason, and in
this case, the application will continuously try to reconnect.

The initial state of the application consists of a `rtspsrc` continuously
reading the RTSP feed of the IP camera. If there are no active events, then
the packets are just discarded. Then, when events are triggered by other
parts of the system, this RTSP feed is used to 1) write videos to disk,
starting a few seconds before the event is triggered and 2) forward the raw
RTP packets to a target UDP host. It is worth mentioning that at any given
time, there may be multiple active events. Because of that, while there will
always be a single `rtspsrc` producer for each feed, there may be multiple
`mp4mux` and `udpsink` consumers.

We have attempted to implement this behaviour using, at first, a dynamic
approach. We have a single central pipeline with `rtspsrc` and `tee`, and
then we dynamically add and remove `tee` branches and elements based on the
task that needs to be accomplished at that time. We have been facing
problems though, as our application hangs after it has been running for some
time, and we haven't yet been able to understand why.

Another approach we have been experimenting with involves three distinct
pipelines, a producer and two consumers. The producer pipeline, backed by
`appsink`, allows direct access to `Sample`s. The consumer pipelines have a
`appsrc` that is used for introducing these `Sample`s back into GStreamer.
That makes it really easy for us to store them (for example, in a
`Vec<Sample>`). It also makes it really easy to know when to feed a `Sample`
to a consumer pipeline by simply looking at the information it holds. The
problem with this approach is that `Sample`s are produced with presentation
times based on the producer pipeline, which don't always make sense with the
consumer pipelines, since the base time in the consumer pipelines can begin
long after the producer’s base time began.

Included here are detailed explanations of each of our attempts to implement
the functionality:
https://gist.github.com/valmirjunior0088/99657cbaf05d69921f80a9be21beba26

The question is: is there a more straightforward way of implementing this
functionality? We have hit roadblocks in both of our attempts, and while
they don't signify that our approaches have been incorrect, they may hint at
the fact that there's perhaps an easier, more robust solution.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/