FrameStepping gstreamer1.x python and general

[Turmel, Frederic]

Thanks Luis, I was able to confirm correct frame stepping with xvideosink but only for a specific behavior when the pipe is never set to playing. I used a test video sequence that has frame number overlay on each frame so I could confirm the stepping result.

Stepping worked correctly with this scenario:

1.       Start pipe in paused state

2.       Send the step event
Result: For each step event, frames were played 1 by 1. Frame were incremented correctly on the video display.

Where it gets messy (this is the case that I was dealing with so far):

1.       Start pipe in paused state

2.       Change pipe state to playing for a small amount of frames to be played.

3.       Paused the pipe again
Result: Now when stepping, for each step event with step value of 1 buffer, the display will skip 10 or more frames after the pipe has been set to playing and then paused. It looks like it is flushing the entire pipe/queue buffer??.

Here’s the code that show the behavior in the second scenario and the input file used: http://www.fredt.org/colorbars.mpg
Is this a bug or the expected behavior? Is there a better way to do this?

from gi.repository import Gst, GObject, GLib
from time import sleep

Gst.init(None)

def on_pad_added(element, pad):
                print "new pad!"
                pad.link(convert.get_static_pad("sink"))

pipe = Gst.Pipeline.new('pipe')
src = Gst.ElementFactory.make('filesrc')
decode = Gst.ElementFactory.make('decodebin')
convert = Gst.ElementFactory.make('videoconvert')
sink = Gst.ElementFactory.make('ximagesink')

src.set_property('location', ‘/home/test/colorbars.mpg')

pipe.add(src)
pipe.add(decode)
pipe.add(convert)
pipe.add(sink)

src.link(decode)
decode.link(convert)
convert.link(sink)

decode.connect('pad-added', on_pad_added)

#pipe.set_state(Gst.State.PLAYING)
pipe.set_state(Gst.State.PAUSED)
loop = 1
while loop:
                sleep(2)
                for x in range(0, 5):
                                sink.send_event(Gst.Event.new_step(Gst.Format.BUFFERS, 1, 1, True, False))
                                print 'step'
                                sleep(1)
                print "pause"
                sleep(2)
                print "playing"
                pipe.set_state(Gst.State.PLAYING)
                sleep(.5)
                pipe.set_state(Gst.State.PAUSED)
                sleep(.5)
                for x in range(0, 5):
                                sink.send_event(Gst.Event.new_step(Gst.Format.BUFFERS, 1, 1, True, False))
                                print 'step'
                                sleep(1)
                loop = loop + 1
                print 'loop: ' + str(loop)
                if loop > 5:
                                exit()


From: gstreamer-devel [mailto:gstreamer-devel-bounces at lists.freedesktop.org] On Behalf Of Luis de Bethencourt
Sent: Saturday, May 30, 2015 4:58 AM
To: Discussion of the development of and with GStreamer
Subject: Re: FrameStepping gstreamer1.x python and general

On 30 May 2015 at 05:19, Turmel, Frederic <Frederic.Turmel at arris.com<mailto:Frederic.Turmel at arris.com>> wrote:
More on this, I was reading the design for the step event. (http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-framestep.txt)

If I understand the step event correctly, a step event will skip the amount of frame specified in the event but those buffers/frames will be drop/flush. Then upon putting the pipeline to playing, it will start at the new position.

If I'm correct, this may not be what I'm looking for.  What I'm trying to achieve is pause the pipeline and play frame by frame with a trigger, my app control when the next frame is sent to the sink.

I did a simple test using filesink as the sink with uncompressed video. Nothing is written when I trigger the frame stepping event. I tried with and without the flush flag with same result.

Any ideas/suggestions?

Thanks


-----Original Message-----
From: gstreamer-devel [mailto:gstreamer-devel-bounces at lists.freedesktop.org<mailto:gstreamer-devel-bounces at lists.freedesktop.org>] On Behalf Of Tim Müller
Sent: Friday, May 29, 2015 2:21 AM
To: Discussion of the development of and with GStreamer
Subject: Re: FrameStepping gstreamer1.x python and general
On Fri, 2015-05-29 at 10:58 +0200, Thibault Saunier wrote:

> As for frame stepping support, I  do not think it is properly
> implemented everywhere, best is to try it out and see/fix it/open a
> bug if needed.

Frame stepping is handled entirely by the sink base class and shouldn't require support from upstream elements (unless you want to step back, then it requires reverse playback support), so hopefully it to mostly just works whatever the source/demuxer/decoder (minus bugs of course, as Thibault said).

 Cheers
  -Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com

Here you have an example:
-------------------------------------------------------------------------------------------------
#!/usr/bin/env python

from gi.repository import Gst, GObject, GLib

Gst.init(None)

pipe = Gst.Pipeline.new('pipe')
src = Gst.ElementFactory.make('videotestsrc')
sink = Gst.ElementFactory.make("autovideosink")

pipe.add(src)
pipe.add(sink)
src.link(sink)

# pipe.set_state(Gst.State.PLAYING)
pipe.set_state(Gst.State.PAUSED)

def step():
    print 'step'
    sink.send_event(Gst.Event.new_step(Gst.Format.BUFFERS, 1, 1, True, False))
    GLib.timeout_add(1000, step)

GLib.timeout_add(1000, step)

ml = GObject.MainLoop()
ml.run()
-------------------------------------------------------------------------------------------------
Run it and you can see that each frame will be displayed in the sink while staying in PAUSED state.
Not sure if/how filesink handles step events. Use the above code to play with frame stepping.
Thanks,
Luis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20150530/a2d93718/attachment-0001.html>