Re: Read frame by frame mp4 file, need example.

[Антон Шаров]

Hi!
 
Thank you for reply!
 
>just set up appsink to be sync=false, pull preroll / samples (wait for
>signals only if you don't want to block), do your processing, rinse /
>repeat.
 
I did as you advised (add sync=false) and call PullPreroll in a loops —  https://gist.github.com/sharov-am/e895575c8d21753b77f2fa73d9b7ba1e  and it blocks on appsink.PullPreroll or NewSample.
I’ve tried to set pipe into playing or paused state — nothing, not a single event raised…
 
What is the difference between PullPreroll and NewSample in this specific approach?
 
>frame-stepping doesn't need to enter the picture here,
 
Then what for it is usefull? I thought it is manual frame iterating, step by step. Than what it is useful
for?
 
Thanks in advance. 
>Четверг, 27 июля 2023, 14:49 +03:00 от Mathieu Duponchelle <mathieu at centricular.com>:
> 
>As Nicolas said, frame-stepping doesn't need to enter the picture here,
>just set up appsink to be sync=false, pull preroll / samples (wait for
>signals only if you don't want to block), do your processing, rinse /
>repeat.
>
>On Thu, 2023-07-27 at 00:31 +0300, Антон Шаров via gstreamer-devel
>wrote:
>> Thank you for reply, Nicolas!
>>  
>> > The question is ambiguous, since MP4 containers can contains
>> > multiple streams and metadata. Do you want to read each >video
>> > frame of the first video frame ? Do you want these to be coded or
>> > decoded ? All this is important, because before >pulling this data
>> > into your application (I would suggesting using appsink element),
>> > you will have to design the right pipeline.
>>  
>> I don’t quite understand your clarification because I’m quite new to
>> this domain. I split rtsp stream into mp4 files via
>> rtsp ! h264depay ! h264parse ! splitmuxsink  location=… max-time-
>> size=5 seconds .
>> Given camera with 15 fps and ~5 sec. video I suppose that I need to
>> iterate over 75 frames. So it is unclear to me what you mean by
>> «video frame of the first frame». Keyframe and it’s subframes  or I-
>> frame and sequential p-frames or all GOP’s in a file.  I need to
>> extract  timestamp for each frame and find the most accurate frame
>> for a given timestamp.
>> My pipe for this is filesrc ! decodebin ! appsink, where I hope to
>> handle new-sample or preroll event for each frame (buffer contents ==
>> frame).
>>  
>> > Frame stepping is unlikely what you are a looking for.
>>  
>> I supposed that it means just manual iteration over frames in video
>> file — step by step. Like loop  with opencv. 
>>  
>> > Среда, 26 июля 2023, 22:42 +03:00 от Nicolas Dufresne
>> > < nicolas at ndufresne.ca >:
>> >  
>> > Hi,
>> >  
>> > Le mercredi 26 juillet 2023 à 18:48 +0300, Антон Шаров via
>> > gstreamer-devel a écrit :
>> > > Hi.
>> > >  
>> > > Where I can find example of reading mp4 file frame by frame, so
>> > > it was equivalent to OpenCv code like:
>> >
>> >  
>> > The question is ambiguous, since MP4 containers can contains
>> > multiple streams and metadata. Do you want to read each video frame
>> > of the first video frame ? Do you want these to be coded or decoded
>> > ? All this is important, because before pulling this data into your
>> > application (I would suggesting using appsink element), you will
>> > have to design the right pipeline.
>> >  
>> > Nicolas
>> >  
>> > >  
>> > > while(cap.isOpened()):
>> > >   # Capture frame-by-frame
>> > >    ret, frame = cap.read()
>> > >  
>> > > I saw this frame stepping
>> > > example https://fossies.org/linux/gstreamer/tests/examples/steppi
>> > > ng/framestep1.c
>> > > and .  I’ve tried to recreate this example on gstreamer-sharp and
>> > > here is my approach:
>> > >  https://gist.github.com/sharov-am/78949b8866235ca14112818385893152
>> >
>> >  
>> > Frame stepping is unlikely what you are a looking for.
>> >  
>> > >  
>> > > And of course it doesn’t work. I’ve read this doc
>> > > ( https://gstreamer.freedesktop.org/documentation/additional/desig
>> > > n/framestep.html?gi-language=c#frame-stepping)
>> > > and got from it that there are two main very important
>> > > parameters like flush and intermediate and sequnce of events:
>> > > 2 step start and 1 step done. Also I expected that appsink
>> > > preroll or new-sample event would be raised, but nothing
>> > > happens.
>> > >  
>> > > My expectations are: I raise step event, some machinery happens
>> > > and eventually I handle appsink preroll or new-sample
>> > > events with needed frame buffer with actual frame. And it
>> > > continues till EOS (end of file).
>> > > So, what I’m doing wrong?
>> > >  
>> > > Also, I’ve noticed that my pipe has WARN messages like
>> > >  
>> > > WARN                 basesrc
>> > > gstbasesrc.c:3132:gst_base_src_loop:<filesrc> error: Internal
>> > > data stream error.
>> > > WARN                 basesrc
>> > > gstbasesrc.c:3132:gst_base_src_loop:<filesrc> error: streaming
>> > > stopped, reason not-linked (-1)
>> > > but I’m not sure that this is the cause…
>> > >
>> > > Thanks in advance.
>> > >  
>> > >  
>> > >  
>> >
>> >  
>> >  
>>  
>>  
>> --
>> С Уважением,
>> Шаров Антон
>>  
 
 
--
С Уважением,
Шаров Антон
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20230727/cc2aa1a4/attachment.htm>