Reading frames one-by-one to python/numpy

[Hjalmar Turesson]

Hi,

In general, it seems that things fail with Format.BUFFERS while it works
with Format.TIME.

For example:

In [13]: pipeline.query_duration(Gst.Format.BUFFERS)
Out[13]: (False, 0)

While this works fine:
In [14]: pipeline.query_duration(Gst.Format.TIME)
Out[14]: (True, 889267000000)

Format.BUFFERS looks otherwise ok:

In [15]: Gst.Format.BUFFERS
Out[15]: <enum GST_FORMAT_BUFFERS of type GstFormat>

Maybe I messed up something obvious. Using version 1.6.0.0

Thanks,
Hjalmar










On Mon, Dec 7, 2015 at 6:01 PM, Hjalmar Turesson <hturesson at gmail.com>
wrote:

> Hi,
>
> The seek works when the format is TIME, e.g.:
>
> pipeline.seek_simple(Gst.Format.TIME, Gst.SeekFlags.FLUSH, 0.3*Gst.SECOND)
>
> but not when the format is BUFFERS, e.g.:
>
> pipeline.seek_simple(Gst.Format.BUFFERS, Gst.SeekFlags.FLUSH, 1000)
>
> Does this make any sense?
>
> Thanks,
> Hjalmar
>
> On Mon, Dec 7, 2015 at 4:07 PM, Hjalmar Turesson <hturesson at gmail.com>
> wrote:
>
>> Perfect example. It does exactly what I wanted.
>>
>> But... The seek returns false (the other part works perfectly)
>>
>>     CLI='filesrc location=%s ! decodebin ! appsink name=sink' % filename
>>     pipeline=Gst.parse_launch(CLI)
>>     # get sink
>>     appsink=pipeline.get_by_name("sink")
>>     # set to PAUSED to make the first frame arrive in the sink
>>     pipeline.set_state(Gst.State.PAUSED)
>> *    pipeline.seek_simple(Gst.Format.BUFFERS, Gst.SeekFlags.FLUSH,
>> frame_number)*
>>     smp = appsink.emit('pull-preroll')
>>     buf = smp.get_buffer()
>>
>>     data=buf.extract_dup(0, buf.get_size())[:307200] # Only 1/2 of
>> expected RGB size
>>     frame = np.fromstring(data,dtype='uint8').reshape((480,640,1))
>>
>>
>> Thanks a lot,
>>
>> Hjalmar
>>
>> On Mon, Dec 7, 2015 at 1:47 PM, Tim Müller <tim at centricular.com> wrote:
>>
>>> On Mon, 2015-12-07 at 13:30 -0300, Hjalmar Turesson wrote:
>>>
>>> Hi Hjalmar,
>>>
>>> This (C) example shows how to retrieve frames from appsink, perhaps you
>>> find it useful:
>>>
>>> http://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/tests/examp
>>> les/snapshot/snapshot.c
>>> <http://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/tests/examples/snapshot/snapshot.c>
>>>
>>> It uses the pull-preroll signal/API, that only works once, if you want
>>> to retrieve one frame, and then another one, then just use the pull-
>>> sample API N times.
>>>
>>> There are two ways to use appsink: one is to have it emit a signal
>>> (new-sample) whenever a buffer arrives, and then you pull it in the
>>> callback). The other is to simply set the pipeline to PLAYING and then
>>> do pull-sample, which will block and wait for a simple if none is
>>> available yet then.
>>>
>>> Note that from the "new-sample" signal callback you can also return
>>> GST_FLOW_EOS if you don't want to be called again.
>>>
>>> Cheers
>>>  -Tim
>>>
>>> > Hi,
>>> >
>>> > I would like read individual frames to python/numpy for analysis.
>>> > That is, I want call a function that reads and returns 1 frame. I
>>> > thought I could do this with appsink, but I have 2 problems. 1) I
>>> > cannot return a frame from the callback function 2) It doesn't just
>>> > read 1 frame.
>>> >
>>> > Simple example that does not do what I want:
>>> >
>>> >    import gi
>>> >    gi.require_version('Gst', '1.0')
>>> >    from gi.repository import GObject, Gst, Gtk
>>> >    GObject.threads_init()
>>> >    Gst.init(None)
>>> >    ##########
>>> >
>>> >     CLI='filesrc location=testVideo.mp4 ! decodebin ! appsink
>>> > name=sink'
>>> >     pipeline=Gst.parse_launch(CLI)
>>> >     appsink=pipeline.get_by_name("sink")
>>> >     appsink.set_property("max-buffers",1)
>>> >     appsink.set_property('emit-signals',True)
>>> >     appsink.set_property('sync',False)
>>> >
>>> >     def on_new_buffer(appsink):
>>> >         sample = appsink.emit('pull-sample')
>>> >         #get the buffer
>>> >         buf=sample.get_buffer()
>>> >         data=buf.extract_dup(0,buf.get_size())
>>> >         return False  # Here I would like to return the frame
>>> >
>>> >     appsink.connect('new-sample', on_new_buffer)
>>> >     pipeline.set_state(Gst.State.PLAYING)
>>> >     pipeline.set_state(Gst.State.NULL)
>>> >
>>> > Is it possible to do what I want to do using appsink?
>>> >
>>> > Another strategy I tried is to use fdsink and read the data from that
>>> > location:
>>> >
>>> >     import os
>>> >
>>> >     readfd, writefd = os.pipe()
>>> >     frame_size_bytes = 460800
>>> >
>>> >     CLI='filesrc location=testVideo.mp4 ! decodebin ! fdsink
>>> > name=sink'
>>> >     pipeline=Gst.parse_launch(CLI)
>>> >     fdsink=pipeline.get_by_name("sink")
>>> >     fdsink.set_property("fd", readfd)
>>> >
>>> >     pipeline.set_state(Gst.State.PLAYING)
>>> >     data = os.read(readfd, frame_size_bytes)
>>> >     pipeline.set_state(Gst.State.PAUSED)
>>> >
>>> > This does not work either. The code hang at the data = os.read...
>>> > line, maybe trying to play the entire file before moving on.
>>> >
>>> > How can I read only one frame?
>>> >
>>> > Thanks,
>>> > Hjalmar
>>> > _______________________________________________
>>> > gstreamer-devel mailing list
>>> > gstreamer-devel at lists.freedesktop.org
>>> > http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>> --
>>> Tim Müller, Centricular Ltd - http://www.centricular.com
>>>
>>>
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20151207/bacf3643/attachment-0001.html>