Reading frames one-by-one to python/numpy

[Tim Müller]

On Mon, 2015-12-07 at 13:30 -0300, Hjalmar Turesson wrote:

Hi Hjalmar,

This (C) example shows how to retrieve frames from appsink, perhaps you
find it useful:

http://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/tests/examp
les/snapshot/snapshot.c

It uses the pull-preroll signal/API, that only works once, if you want
to retrieve one frame, and then another one, then just use the pull-
sample API N times.

There are two ways to use appsink: one is to have it emit a signal
(new-sample) whenever a buffer arrives, and then you pull it in the
callback). The other is to simply set the pipeline to PLAYING and then
do pull-sample, which will block and wait for a simple if none is
available yet then.

Note that from the "new-sample" signal callback you can also return
GST_FLOW_EOS if you don't want to be called again.

Cheers
 -Tim

> Hi,
> 
> I would like read individual frames to python/numpy for analysis.
> That is, I want call a function that reads and returns 1 frame. I
> thought I could do this with appsink, but I have 2 problems. 1) I
> cannot return a frame from the callback function 2) It doesn't just
> read 1 frame.
> 
> Simple example that does not do what I want:
> 
>    import gi
>    gi.require_version('Gst', '1.0')
>    from gi.repository import GObject, Gst, Gtk
>    GObject.threads_init()
>    Gst.init(None)
>    ##########
> 
>     CLI='filesrc location=testVideo.mp4 ! decodebin ! appsink
> name=sink' 
>     pipeline=Gst.parse_launch(CLI)
>     appsink=pipeline.get_by_name("sink")
>     appsink.set_property("max-buffers",1)
>     appsink.set_property('emit-signals',True)
>     appsink.set_property('sync',False) 
>     
>     def on_new_buffer(appsink):
>         sample = appsink.emit('pull-sample')
>         #get the buffer
>         buf=sample.get_buffer()
>         data=buf.extract_dup(0,buf.get_size())
>         return False  # Here I would like to return the frame
> 
>     appsink.connect('new-sample', on_new_buffer)
>     pipeline.set_state(Gst.State.PLAYING)
>     pipeline.set_state(Gst.State.NULL) 
> 
> Is it possible to do what I want to do using appsink?
> 
> Another strategy I tried is to use fdsink and read the data from that
> location:
> 
>     import os
> 
>     readfd, writefd = os.pipe() 
>     frame_size_bytes = 460800
> 
>     CLI='filesrc location=testVideo.mp4 ! decodebin ! fdsink
> name=sink'    
>     pipeline=Gst.parse_launch(CLI)
>     fdsink=pipeline.get_by_name("sink")
>     fdsink.set_property("fd", readfd)
> 
>     pipeline.set_state(Gst.State.PLAYING)
>     data = os.read(readfd, frame_size_bytes)
>     pipeline.set_state(Gst.State.PAUSED)
> 
> This does not work either. The code hang at the data = os.read...
> line, maybe trying to play the entire file before moving on.
> 
> How can I read only one frame?
> 
> Thanks,
> Hjalmar
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-- 
Tim Müller, Centricular Ltd - http://www.centricular.com