Seeing huge volume of buffers with HLS/TS source compared to RTSP sources

[Matt Feury]

Hey Tim, thanks for the context and great questions.

I had a similar thought. RTSP streams itself "live" whereas the HLS stream
is served using .m3u8 files which are playlists of ts files. So I do think
perhaps the souphttpsrc is internally fetching the ts files as they are
added to the playlist in realtime, but those files are likely processed
"all at once" since it's then just a download of a .ts file. I'm not sure
how this could lead to what I'm seeing, but it is a good direction I should
keep chasing down. To answer your question, the HLS stream is a
"live" stream as well (it's supposed to be real time and i have even set
is-live on the souphttpsrc), but to my knowledge that's still served in a
playlist of .ts fragments, so will naturally be more latent.

I put an identity directly after the source (either souphttpsrc or
rtspsrc), and I am seeing buffers of size 500-1000 (roughly) with
souphttpsrc, whereas rtspsrc buffers are of size 50,000-100,000. so it
"makes sense" that the same h264 data would require *more *buffers if
they're smaller (hls) but less if they're bigger. But I just need to figure
out how to "combine them" so to speak.

I am going to play with clocksync and other sync attributes as well.

Feel free to correct any of my misunderstandings above. I'm fairly new to
the gst world but really enjoying digging in :).

thanks
Matt


On Wed, Jan 18, 2023 at 7:06 PM Tim-Philipp Müller <t.i.m at zen.co.uk> wrote:

> On Wed, 2023-01-18 at 17:30 -0500, Matt Feury via gstreamer-devel wrote:
>
> Hi Matt,
>
> I'm currently running a pipeline that can support either rtspsrc or HLS
> sources (via souphttpsrc). The pipeline generally looks something like this:
>
> > rtspsrc/souphttpsrc (depending on url structure) ! parsebin ! decodebin
> ! videoconvert ! identity ! tee ! queue ! multiple sinks (e.g. appsink,
> webrtc, etc, all from tee).
>
> Note this has been scrubbed of detailed pipeline info like specific sinks
> and extra queues/identities for simplicity's sake.
>
> I am using the identity element to calculate the number of buffers per
> second passing through. When running the pipeline with an RTSPsrc, i am
> seeing a BPS value that corresponds heavily to the FPS (e.g. 5). However,
> when I run with an HLS source, i am seeing BPS values 10-100x as large as
> FPS (e.g. 250).
>
> Weirdly, in my appsink, if i just do a simple cv2.imwrite to the local
> machine, I can confirm that each buffer coming through corresponds to a
> unique frame of the stream.
>
> So I guess my question is: what could be causing these increased buffer
> counts for the same pipeline but only when HLS/TS data is flowing through
> it? While the stream sources are different, the FPS of each is roughly
> similar. I would presume that after the decodebin and videoconvert, I am
> left with raw video of the same general media type / caps regardless of the
> input source, so I am surprised to see such variation.
>
>
> Only thing I can think of is that rtspsrc is live and naturally receives
> an input stream at the playback rate, whereas with your hls setup maybe
> you're downloading fragments as fast as your bandwidth allows?
>
> is your HLS stream a live stream or a VOD stream? Do you get the higher
> fps only at the beginning or continuously over a longer period?
>
> For non-live inputs synchronisation only happens at the sink(s) typically,
> and it will depend on the sink in use whether it's live or not by default
> (sync=true on the sink).
>
> For appsink you might alaso want to limit the internal queue which by
> default is unlimited.
>
> You can play with identity sync=true (or replace with a clocksync element)
> to see if that makes a difference (I don't think that should be necessary
> though).
>
> Cheers
>  Tim
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230118/ddab9e55/attachment.htm>