Multithreaded fast-forward possible?
Peter Rennert
p.rennert at cs.ucl.ac.uk
Thu Dec 13 05:35:42 PST 2012
Although in my code it does not change anything of the behaviour, I just
found the theoretical solution to my idea of skipping frames for faster
playback:
"When doing fast forward (rate > 1.0) or fast reverse (rate < -1.0)
trickmode playback, the GST_SEEK_FLAG_SKIP flag can be used to instruct
decoders and demuxers to adjust the playback rate by skipping frames.
This can improve performance and decrease CPU usage because not all
frames need to be decoded." [1]
[1]
http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstEvent.html#GstSeekFlags
On 12/13/2012 01:08 PM, Peter Rennert wrote:
> Thanks Juraj these were good tips.
>
> I needed to add a queue before ffdec_h264, but there must not be one
> after ffdec_264. Furthermore, I needed to set sync=false at the image
> sink. However, now it seems as the filesink (or qtdemux/h264parse?) is
> not feeding the data quickly enough. (I am just at x2 speed, my disk
> I/O maximum should far ahead). My ffdec_264 sends now the following
> warning message when I try to increase the speed:
>
> 0:00:06.193160955 3058 0x27f08f0 WARN videodecoder
> gstvideodecoder.c:2847:gst_video_decoder_alloc_output_frame:<dec>
> failed to get buffer wrong-state
> 0:00:06.193239481 3058 0x27f08f0 WARN ffmpeg
> gstffmpegviddec.c:1218:gst_ffmpegviddec_frame:<dec> ffdec_h264:
> decoding error (len: -1, have_data: 0)
>
> I got rid of those warnings when I add a queue after the filesink.
>
> Still there are problems with my pipeline:
>
> First problem is that the standard speed seems to be twice as normal.
>
> Second, although I see that the computations are now spread over
> several cores and I do not get anymore warnings from ffdec_h264 after
> keyPress (and respective the seek event), the playback speed does not
> change at all anymore.
>
> I put my test code below. Its in python (my first attempt of python
> and gstreamer), but I think it should be readable for people who use C
> or C++, too. I suspect I might use wrong flags in gst.event_new_seek().
>
> P.S.: just to make it complete for documentation purposes, playing a
> bit around with it and using
>
> self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
> gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE,
> gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE))
>
> instead of
>
> self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
> gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0,
> gst.SEEK_TYPE_NONE, 0))
>
> gives me a python error:
>
> Traceback (most recent call last):
> File "pyGstViewer.py", line 61, in keyPress
> self.increasePlaybackSpeed()
> File "pyGstViewer.py", line 70, in increasePlaybackSpeed
> gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE))
>
>
> ===== CODE =====
>
> import sys, os
> import pygst
> pygst.require("0.10")
> import gst
> import pygtk, gobject
> import gtk
>
> class GTK_Main:
> def __init__(self):
> self.window = gtk.Window(gtk.WINDOW_TOPLEVEL)
> self.window.connect("destroy", gtk.main_quit, "WM destroy")
> vbox = gtk.VBox()
> self.window.add(vbox)
> vbox.pack_start(gtk.Label("Please type text"))
> entry = gtk.Entry()
> vbox.pack_start(entry)
> self.window.connect("key-press-event", self.keyPress)
>
> self.window.show_all()
>
> self.pipeline = gst.parse_launch("filesrc " +
> "location=/home/peter/vid/20121207/00/2012-12-07.00-00-00.mp4 ! " +
> "queue ! qtdemux ! h264parse ! queue ! ffdec_h264 name=dec
> ! " +
> "ffmpegcolorspace ! deinterlace ! xvimagesink sync=false")
>
> self.dec = self.pipeline.get_by_name("dec")
>
> self.dec.set_property("max-threads", 8);
>
> self.pbRate = 1
>
> self.pipeline.set_state(gst.STATE_PLAYING)
>
> bus = self.pipeline.get_bus()
> bus.add_signal_watch()
> bus.connect("message", self.onMessage)
>
>
> def keyPress(self, widget, event):
> print "keypress event!!"
> key = gtk.gdk.keyval_name(event.keyval)
> if key == "d":
> self.increasePlaybackSpeed()
> else:
> self.decreasePlaybackSpeed()
>
> def increasePlaybackSpeed(self):
> print str(self.pbRate) + " --> " + str(self.pbRate + 4)
> self.pbRate += 4
> self.pipeline.send_event(gst.event_new_seek(self.pbRate,
> gst.FORMAT_TIME,
> gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0,
> gst.SEEK_TYPE_NONE, 0))
> print "increased playback speed!!!"
>
> def decreasePlaybackSpeed(self):
> self.pbRate -= 2
> self.pipeline.send_event(gst.event_new_seek(self.pbRate,
> gst.FORMAT_TIME,
> gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0,
> gst.SEEK_TYPE_NONE, 0))
> print "decreased playback speed!!!"
>
> def onMessage(self, bus, message):
> t = message.type
> if t == gst.MESSAGE_EOS:
> self.pipeline.set_state(gst.STATE_NULL)
> elif t == gst.MESSAGE_ERROR:
> self.pipeline.set_state(gst.STATE_NULL)
> err, debug = message.parse_error()
> print "Error: %s" % err, debug
>
> GTK_Main()
> gtk.gdk.threads_init()
> gtk.main()
>
>
>
> On 12/13/2012 11:18 AM, Juraj Holtak wrote:
>>
>> And maybe the xvimagesink should have sync=false set too... maybe...
>>
>> On Dec 13, 2012 12:15 PM, "Juraj Holtak" <juraj.holtak at gmail.com
>> <mailto:juraj.holtak at gmail.com>> wrote:
>>
>> Hi,
>>
>> Maybe worth a try:
>>
>> Put an "queue" element before and after ffdec_h264 and use
>> max-threads=<your_cpu_count>. I imagine it worked like this for
>> me but maybe I just had luck...
>>
>> Juraj
>>
>> On Dec 13, 2012 2:03 AM, "Peter Rennert" <p.rennert at cs.ucl.ac.uk
>> <mailto:p.rennert at cs.ucl.ac.uk>> wrote:
>>
>> Dear all,
>>
>> I want to increase the playback of a video quite drastically.
>> At the moment I am playing a test video sequence with
>>
>> gst-launch-0.10 filesrc location=/path/to/my.mp4 ! qtdemux !
>> h264parse ! ffdec_h264 name=dec ! ffmpegcolorspace !
>> deinterlace ! xvimagesink
>>
>> and at some point the video speed cannot get increased
>> further, because it uses only one CPU core to decode the
>> video. Gstreamer starts complaining of getting behind the
>> timestamps of the stream and crashes.
>>
>> I could think of two solutions:
>> - Is there a "native" way of making use of more than just
>> a single core to decode the video frames? I tried to set
>> "max-threads" of ffdec_h264 to 4, but it still only uses a
>> single core.
>>
>> or,
>> - Is there a way of skipping frames? As I need only a
>> "effective" framerate on the screen of about 25fps I could
>> just decode the frames I need and skip the other frames. Then
>> I would not need more CPU power than for realtime playback.
>>
>> or.
>> - Is there a way to split the stream after the qtdemux
>> and use several decoders in a kind of decoder pool
>> (distribute the frames between them) and unify the stream after?
>>
>> My videos are in H264 format. I am not sure if the
>> non-keyframes are encoded with respect to the previous frame
>> or with respect to the last key-frame. In that case I could
>> try to filter the key-frames and send them to every decoder
>> in the pool and decode the any frame with that as reference
>> until the next key-frame. With some management I could try to
>> merge the output of the pool decoder with a input-selector.
>> But that sounds bulky :\ So if one knows how to get to one of
>> the first two solutions, I would be very happy...
>>
>>
>> Cheers,
>>
>> Peter
>>
>> PS I am happy to provide an example video. Maybe something is
>> wrong with the encoding there that prevents ffdec_h264 to use
>> multiple cores
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> <mailto:gstreamer-devel at lists.freedesktop.org>
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>>
>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20121213/1aa116de/attachment.html>
More information about the gstreamer-devel
mailing list