Help with pulling data from appsink, but earlier

[Tamas]

Hi,

In short, I'd like to pull samples from the appsink as soon as possible
(after the previous sample, before their PTS, without setting sync to
false). However, the appsink serves them "around" their respective PTS in a
seemingly best-effort way, sometimes before, sometimes significantly later.
How to extract the same samples earlier?

Longer: I would like to pull video and audio data from appsink elements and
serve them to an external application that processes said frames, and
displays them using Gstreamer's timing (i.e. current position on appsink).
What I mean is that, for example, video frames would be processed and then
displayed by an external program, but at their PTS according to Gstreamer's
clock. I've already posted about this earlier (How to reliably get samples
from appsink before PTS (lists.freedesktop.org)
<https://lists.freedesktop.org/archives/gstreamer-devel/2023-January/080926.html>),
but I was probably not clear about my questions and that post received no
response.

My current solution for video data is that I extract buffers from a data
probe on the appsink's sink. I can show you some code if necessary, but
it's essentially the same as the example code in "sdlshare.c", only in my
case it's on a data probe. I also have a callback on "new sample", which
unreferences samples to clear up buffer space in the appsink. The sink
probe receives buffers very soon after a previous sample is discarded,
which is exactly what I need. However, is this a reliable solution? Is
there a 1:1 relationship between buffers on the input and samples on the
output? It seems to be the case for the few video files I've tested it on,
but my knowledge about such things is limited. Would this work for audio?
Should I need to be concerned about buffer lists (there doesn't seem to be
much documentation on those)? Does the appsink sometimes combine separate
buffers (as seen on the sink probe) into a sample?

As an alternative, I could just modify PTS values for buffers (e.g. set
them to the previous frame's PTS + 1), and then use "new sample" to get the
data, which also seems to be working. Is this solution better?

Best regards to you all, and I hope that this time I was able to make my
point clearer.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230310/acdcfe67/attachment.htm>