Read frame by frame mp4 file, need example.

Thu Jul 27 15:57:03 UTC 2023

You seem to be confusing concepts a bit, I can't answer your question.
You can safely ignore the preroll sample in your case, and simply block
on PullSample in a loop, you also don't need to connect to the signals
(NewSample, NewPreroll) at all.

Frame-stepping is for going for one sample to the next with any sink,
but the level of control you get from appsink makes this feature
irrelevant.

On Thu, 2023-07-27 at 18:40 +0300, Антон Шаров via gstreamer-devel
wrote:
> Hi!
>  
> Thank you for reply!
>  
> > just set up appsink to be sync=false, pull preroll / samples (wait
> > for
> > signals only if you don't want to block), do your processing, rinse
> > /
> > repeat.
>  
> I did as you advised (add sync=false) and call PullPreroll in a loops
> — 
> https://gist.github.com/sharov-am/e895575c8d21753b77f2fa73d9b7ba1e an
> d it blocks on appsink.PullPreroll or NewSample.
> I’ve tried to set pipe into playing or paused state — nothing, not a
> single event raised…
>  
> What is the difference between PullPreroll and NewSample in this
> specific approach?
>  
> > frame-stepping doesn't need to enter the picture here,
>  
> Then what for it is usefull? I thought it is manual frame iterating,
> step by step. Than what it is useful
> for?
>  
> Thanks in advance.
> > Четверг, 27 июля 2023, 14:49 +03:00 от Mathieu Duponchelle
> > <mathieu at centricular.com>:
> >  
> > As Nicolas said, frame-stepping doesn't need to enter the picture
> > here,
> > just set up appsink to be sync=false, pull preroll / samples (wait
> > for
> > signals only if you don't want to block), do your processing, rinse
> > /
> > repeat.
> > 
> > On Thu, 2023-07-27 at 00:31 +0300, Антон Шаров via gstreamer-devel
> > wrote:
> > > Thank you for reply, Nicolas!
> > >  
> > > > The question is ambiguous, since MP4 containers can contains
> > > > multiple streams and metadata. Do you want to read each >video
> > > > frame of the first video frame ? Do you want these to be coded
> > > > or
> > > > decoded ? All this is important, because before >pulling this
> > > > data
> > > > into your application (I would suggesting using appsink
> > > > element),
> > > > you will have to design the right pipeline.
> > >  
> > > I don’t quite understand your clarification because I’m quite new
> > > to
> > > this domain. I split rtsp stream into mp4 files via
> > > rtsp ! h264depay ! h264parse ! splitmuxsink  location=… max-time-
> > > size=5 seconds .
> > > Given camera with 15 fps and ~5 sec. video I suppose that I need
> > > to
> > > iterate over 75 frames. So it is unclear to me what you mean by
> > > «video frame of the first frame». Keyframe and it’s subframes 
> > > or I-
> > > frame and sequential p-frames or all GOP’s in a file.  I need to
> > > extract  timestamp for each frame and find the most accurate
> > > frame
> > > for a given timestamp.
> > > My pipe for this is filesrc ! decodebin ! appsink, where I hope
> > > to
> > > handle new-sample or preroll event for each frame (buffer
> > > contents ==
> > > frame).
> > >  
> > > > Frame stepping is unlikely what you are a looking for.
> > >  
> > > I supposed that it means just manual iteration over frames in
> > > video
> > > file — step by step. Like loop  with opencv. 
> > >  
> > > > Среда, 26 июля 2023, 22:42 +03:00 от Nicolas Dufresne
> > > > <nicolas at ndufresne.ca>:
> > > >  
> > > > Hi,
> > > >  
> > > > Le mercredi 26 juillet 2023 à 18:48 +0300, Антон Шаров via
> > > > gstreamer-devel a écrit :
> > > > > Hi.
> > > > >  
> > > > > Where I can find example of reading mp4 file frame by frame,
> > > > > so
> > > > > it was equivalent to OpenCv code like:
> > > > 
> > > >  
> > > > The question is ambiguous, since MP4 containers can contains
> > > > multiple streams and metadata. Do you want to read each video
> > > > frame
> > > > of the first video frame ? Do you want these to be coded or
> > > > decoded
> > > > ? All this is important, because before pulling this data into
> > > > your
> > > > application (I would suggesting using appsink element), you
> > > > will
> > > > have to design the right pipeline.
> > > >  
> > > > Nicolas
> > > >  
> > > > >  
> > > > > while(cap.isOpened()):
> > > > >   # Capture frame-by-frame
> > > > >    ret, frame = cap.read()
> > > > >  
> > > > > I saw this frame stepping
> > > > > example 
> > > > > https://fossies.org/linux/gstreamer/tests/examples/steppi
> > > > > ng/framestep1.c
> > > > > and .  I’ve tried to recreate this example on gstreamer-sharp
> > > > > and
> > > > > here is my approach:
> > > > > https://gist.github.com/sharov-am/78949b8866235ca14112818385893152
> > > > 
> > > >  
> > > > Frame stepping is unlikely what you are a looking for.
> > > >  
> > > > >  
> > > > > And of course it doesn’t work. I’ve read this doc
> > > > > (
> > > > > https://gstreamer.freedesktop.org/documentation/additional/des
> > > > > ig
> > > > > n/framestep.html?gi-language=c#frame-stepping)
> > > > > and got from it that there are two main very important
> > > > > parameters like flush and intermediate and sequnce of events:
> > > > > 2 step start and 1 step done. Also I expected that appsink
> > > > > preroll or new-sample event would be raised, but nothing
> > > > > happens.
> > > > >  
> > > > > My expectations are: I raise step event, some machinery
> > > > > happens
> > > > > and eventually I handle appsink preroll or new-sample
> > > > > events with needed frame buffer with actual frame. And it
> > > > > continues till EOS (end of file).
> > > > > So, what I’m doing wrong?
> > > > >  
> > > > > Also, I’ve noticed that my pipe has WARN messages like
> > > > >  
> > > > > WARN                 basesrc
> > > > > gstbasesrc.c:3132:gst_base_src_loop:<filesrc> error: Internal
> > > > > data stream error.
> > > > > WARN                 basesrc
> > > > > gstbasesrc.c:3132:gst_base_src_loop:<filesrc> error:
> > > > > streaming
> > > > > stopped, reason not-linked (-1)
> > > > > but I’m not sure that this is the cause…
> > > > > 
> > > > > Thanks in advance.
> > > > >  
> > > > >  
> > > > >  
> > > > 
> > > >  
> > > >  
> > >  
> > >  
> > > --
> > > С Уважением,
> > > Шаров Антон
> > >  
>  
>  
> --
> С Уважением,
> Шаров Антон
>  