Read frame by frame mp4 file, need example.

[Mathieu Duponchelle]

As Nicolas said, frame-stepping doesn't need to enter the picture here,
just set up appsink to be sync=false, pull preroll / samples (wait for
signals only if you don't want to block), do your processing, rinse /
repeat. 

On Thu, 2023-07-27 at 00:31 +0300, Антон Шаров via gstreamer-devel
wrote:
> Thank you for reply, Nicolas!
>  
> > The question is ambiguous, since MP4 containers can contains
> > multiple streams and metadata. Do you want to read each >video
> > frame of the first video frame ? Do you want these to be coded or
> > decoded ? All this is important, because before >pulling this data
> > into your application (I would suggesting using appsink element),
> > you will have to design the right pipeline.
>  
> I don’t quite understand your clarification because I’m quite new to
> this domain. I split rtsp stream into mp4 files via
> rtsp ! h264depay ! h264parse ! splitmuxsink  location=… max-time-
> size=5 seconds .
> Given camera with 15 fps and ~5 sec. video I suppose that I need to
> iterate over 75 frames. So it is unclear to me what you mean by
> «video frame of the first frame». Keyframe and it’s subframes  or I-
> frame and sequential p-frames or all GOP’s in a file.  I need to
> extract  timestamp for each frame and find the most accurate frame
> for a given timestamp.
> My pipe for this is filesrc ! decodebin ! appsink, where I hope to
> handle new-sample or preroll event for each frame (buffer contents ==
> frame).
>  
> > Frame stepping is unlikely what you are a looking for.
>  
> I supposed that it means just manual iteration over frames in video
> file — step by step. Like loop  with opencv. 
>  
> > Среда, 26 июля 2023, 22:42 +03:00 от Nicolas Dufresne
> > <nicolas at ndufresne.ca>:
> >  
> > Hi,
> >  
> > Le mercredi 26 juillet 2023 à 18:48 +0300, Антон Шаров via
> > gstreamer-devel a écrit :
> > > Hi.
> > >  
> > > Where I can find example of reading mp4 file frame by frame, so
> > > it was equivalent to OpenCv code like:
> > 
> >  
> > The question is ambiguous, since MP4 containers can contains
> > multiple streams and metadata. Do you want to read each video frame
> > of the first video frame ? Do you want these to be coded or decoded
> > ? All this is important, because before pulling this data into your
> > application (I would suggesting using appsink element), you will
> > have to design the right pipeline.
> >  
> > Nicolas
> >  
> > >  
> > > while(cap.isOpened()):
> > >   # Capture frame-by-frame
> > >    ret, frame = cap.read()
> > >  
> > > I saw this frame stepping
> > > example https://fossies.org/linux/gstreamer/tests/examples/steppi
> > > ng/framestep1.c
> > > and .  I’ve tried to recreate this example on gstreamer-sharp and
> > > here is my approach:
> > > https://gist.github.com/sharov-am/78949b8866235ca14112818385893152
> > 
> >  
> > Frame stepping is unlikely what you are a looking for.
> >  
> > >  
> > > And of course it doesn’t work. I’ve read this doc
> > > (https://gstreamer.freedesktop.org/documentation/additional/desig
> > > n/framestep.html?gi-language=c#frame-stepping)
> > > and got from it that there are two main very important
> > > parameters like flush and intermediate and sequnce of events:
> > > 2 step start and 1 step done. Also I expected that appsink
> > > preroll or new-sample event would be raised, but nothing
> > > happens.
> > >  
> > > My expectations are: I raise step event, some machinery happens
> > > and eventually I handle appsink preroll or new-sample
> > > events with needed frame buffer with actual frame. And it
> > > continues till EOS (end of file).
> > > So, what I’m doing wrong?
> > >  
> > > Also, I’ve noticed that my pipe has WARN messages like
> > >  
> > > WARN                 basesrc
> > > gstbasesrc.c:3132:gst_base_src_loop:<filesrc> error: Internal
> > > data stream error.
> > > WARN                 basesrc
> > > gstbasesrc.c:3132:gst_base_src_loop:<filesrc> error: streaming
> > > stopped, reason not-linked (-1)
> > > but I’m not sure that this is the cause…
> > > 
> > > Thanks in advance.
> > >  
> > >  
> > >  
> > 
> >  
> >  
>  
>  
> --
> С Уважением,
> Шаров Антон
>