Help with pulling data from appsink, but earlier

[Tamas]

@vmjl

Thanks for the suggestion.

Yes, I think sync=false is also a possible approach as I could just pull
frames as needed. However, changing the pipeline into a "pull" driven
system might cause a few issues that I don't have experience with. For
example, (how) would such a system work with parallel video and audio
streams and two corresponding appsinks?

My current approach works, as far as I can tell, and it doesn't change the
basic flow of how a media file is played. I'm consuming data at the rate as
it's produced, just trying to get it a bit earlier than it would be
presented by default.

The current solution is to either pull buffers from a data probe on the
sink pad, or, alternatively, change the PTS of incoming buffers. I believe
this should be OK, as long as appsink doesn't do anything special (such as
combining or reorderding) with the buffers. Does it?

I'd like to get confirmation that what I'm doing isn't the wrong way to go
about it. I'm also OK with being told that my questions don't make sense,
because I could also learn from that.

Tamas


> ---------- Forwarded message ----------
> From: Tamas <km212121 at gmail.com>
> To: gstreamer-devel at lists.freedesktop.org
> Cc:
> Bcc:
> Date: Fri, 10 Mar 2023 16:04:13 +0800
> Subject: Help with pulling data from appsink, but earlier
> Hi,
>
> In short, I'd like to pull samples from the appsink as soon as possible
> (after the previous sample, before their PTS, without setting sync to
> false). However, the appsink serves them "around" their respective PTS in a
> seemingly best-effort way, sometimes before, sometimes significantly later.
> How to extract the same samples earlier?
>
> Longer: I would like to pull video and audio data from appsink elements
> and serve them to an external application that processes said frames, and
> displays them using Gstreamer's timing (i.e. current position on appsink).
> What I mean is that, for example, video frames would be processed and then
> displayed by an external program, but at their PTS according to Gstreamer's
> clock. I've already posted about this earlier (How to reliably get
> samples from appsink before PTS (lists.freedesktop.org)
> <https://lists.freedesktop.org/archives/gstreamer-devel/2023-January/080926.html>),
> but I was probably not clear about my questions and that post received no
> response.
>
> My current solution for video data is that I extract buffers from a data
> probe on the appsink's sink. I can show you some code if necessary, but
> it's essentially the same as the example code in "sdlshare.c", only in my
> case it's on a data probe. I also have a callback on "new sample", which
> unreferences samples to clear up buffer space in the appsink. The sink
> probe receives buffers very soon after a previous sample is discarded,
> which is exactly what I need. However, is this a reliable solution? Is
> there a 1:1 relationship between buffers on the input and samples on the
> output? It seems to be the case for the few video files I've tested it on,
> but my knowledge about such things is limited. Would this work for audio?
> Should I need to be concerned about buffer lists (there doesn't seem to be
> much documentation on those)? Does the appsink sometimes combine separate
> buffers (as seen on the sink probe) into a sample?
>
> As an alternative, I could just modify PTS values for buffers (e.g. set
> them to the previous frame's PTS + 1), and then use "new sample" to get the
> data, which also seems to be working. Is this solution better?
>
> Best regards to you all, and I hope that this time I was able to make my
> point clearer.
>
>
>
>
> ---------- Forwarded message ----------
> From: Victor Jaquez <vjaquez at igalia.com>
> To: Discussion of the development of and with GStreamer <
> gstreamer-devel at lists.freedesktop.org>
> Cc:
> Bcc:
> Date: Fri, 10 Mar 2023 09:50:17 +0100
> Subject: Re: Help with pulling data from appsink, but earlier
> On Fri, 10 Mar 2023 at 16:04, Tamas via gstreamer-devel wrote:
> > Hi,
> >
> > In short, I'd like to pull samples from the appsink as soon as possible
> > (after the previous sample, before their PTS, without setting sync to
> > false). However, the appsink serves them "around" their respective PTS
> in a
> > seemingly best-effort way, sometimes before, sometimes significantly
> later.
> > How to extract the same samples earlier?
>
> Have you tried with sync=false property set in appsink?
>
> vmjl
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230310/c31b375a/attachment.htm>