Correlating video to Jpeg frame in appsink reliably

[Nirbheek Chauhan]

Hi Dwight,

This sounds like an ideal case for a custom meta:

https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c#gst_buffer_add_custom_meta

Metas are per-buffer structures that are copied as-is in 1-1
transforms such as video encode, but will probably get dropped by
muxers (you'll have to deal with that manually with pad probes). You
can add an incrementing integer id to the meta for each buffer and use
it to correlate the encoded H264 and JPEG frames.

Cheers,
Nirbheek

On Wed, Dec 7, 2022 at 2:47 AM Dwight Kulkarni via gstreamer-devel
<gstreamer-devel at lists.freedesktop.org> wrote:
>
> Hi all,
>
> I am reposting this as my previous post didn't get any responses. Your help is greatly appreciated.
>
> I have a pipeline as follows:
> videosrc  > Tee  > Tee 1:  Splitmuxsink  >  Tee 2: jpegenc > app sink
>
> 1) A splitmuxsink is saving the video files and I am maintaining a counter of the current sequence used to name the file.
> 2) When I get a callback for format-location-full, I increment the sequence counter
> 3) When I get a "new-sample", I associate the Jpeg frame with the current counter to infer which video this frame is coming from.
>
> The problem is that in some cases, the format-location-full call back occurs a few seconds after the next video has started. So I get a new-sample and it is associated with the previous video. But it actually is present in the current video.
>
> Are there any suggestions of how best to stay in synch with the current video in splitmuxsink ?
>
> --
> Sincerely,
>
> Dwight Kulkarni
> B.ASC, M.ASC, PMP
>
>