<html> <head> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <div class="moz-cite-prefix">I rewrote my script in C++ to make sure that I am not messing up things with python, but it did not change anything. But I solved the problem. If I compile the playback speed tutorial [1] that uses a playbin2 on my file, I can do some kind of fast forward. If I test it with a file [2] with low resolution, I can do fast-forward till about 90x (using GST_SEEK_FLAG_SKIP does not change anything in my case). As I increase the playback speed, the CPU use changes only slightly. But then suddenly it seems that the threading overhead in ffdec_h264 becomes too big and and a single thread goes to full load and blocks the playback. Fast forward at about 50x starts to become instable and gets blocked some time. I have the impression that flushing the pipeline helps in that case. I will try tomorrow to install a handler that flushes the pipeline in such cases. Using my HD file, in the same setup allows me to speed the playback up only until 4x speed. Even though the cores are under relative low use (below 50%) one of the threads starts to block the playback. Anyway it works somehow. One remark to setting xvimagesink to sync=false. In some cases it makes the pipeline play at a high speed (seems to depend on the speed that ffdec_h264 decodes), but the speed is not controllable. In a reduced pipeline (see below) everything works. Therefore for me the multithreading problem is more or less solved by using this pipeline: filesrc location=/path/to/my.mp4 ! qtdemux ! h264parse ! ffdec_h264 name=dec ! autovideosink sync=false It might have been the deinterlace or ffmpegcolorspace that caused ffdec_h264 to decode only with a single thread... [1] <a class="moz-txt-link-freetext" href="http://docs.gstreamer.com/display/GstSDK/Basic+tutorial+13%3A+Playback+speed">http://docs.gstreamer.com/display/GstSDK/Basic+tutorial+13%3A+Playback+speed</a> [2] <a class="moz-txt-link-freetext" href="http://archive.org/details/ThePhantomoftheOpera">http://archive.org/details/ThePhantomoftheOpera</a> On 12/13/2012 01:35 PM, Peter Rennert wrote: </div> <blockquote cite="mid:50C9D9AE.3030503@cs.ucl.ac.uk" type="cite"> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> <div class="moz-cite-prefix">Although in my code it does not change anything of the behaviour, I just found the theoretical solution to my idea of skipping frames for faster playback: "When doing fast forward (rate > 1.0) or fast reverse (rate < -1.0) trickmode playback, the GST_SEEK_FLAG_SKIP flag can be used to instruct decoders and demuxers to adjust the playback rate by skipping frames. This can improve performance and decrease CPU usage because not all frames need to be decoded." [1] [1] <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstEvent.html#GstSeekFlags">http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstEvent.html#GstSeekFlags</a> On 12/13/2012 01:08 PM, Peter Rennert wrote: </div> <blockquote cite="mid:50C9D339.5040406@cs.ucl.ac.uk" type="cite"> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> <div class="moz-cite-prefix">Thanks Juraj these were good tips. I needed to add a queue before ffdec_h264, but there must not be one after ffdec_264. Furthermore, I needed to set sync=false at the image sink. However, now it seems as the filesink (or qtdemux/h264parse?) is not feeding the data quickly enough. (I am just at x2 speed, my disk I/O maximum should far ahead). My ffdec_264 sends now the following warning message when I try to increase the speed: 0:00:06.193160955 3058 0x27f08f0 WARN videodecoder gstvideodecoder.c:2847:gst_video_decoder_alloc_output_frame:<dec> failed to get buffer wrong-state 0:00:06.193239481 3058 0x27f08f0 WARN ffmpeg gstffmpegviddec.c:1218:gst_ffmpegviddec_frame:<dec> ffdec_h264: decoding error (len: -1, have_data: 0) I got rid of those warnings when I add a queue after the filesink. Still there are problems with my pipeline: First problem is that the standard speed seems to be twice as normal. Second, although I see that the computations are now spread over several cores and I do not get anymore warnings from ffdec_h264 after keyPress (and respective the seek event), the playback speed does not change at all anymore. I put my test code below. Its in python (my first attempt of python and gstreamer), but I think it should be readable for people who use C or C++, too. I suspect I might use wrong flags in gst.event_new_seek(). P.S.: just to make it complete for documentation purposes, playing a bit around with it and using self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME, gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE, gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE)) instead of self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME, gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0, gst.SEEK_TYPE_NONE, 0)) gives me a python error: Traceback (most recent call last): File "pyGstViewer.py", line 61, in keyPress self.increasePlaybackSpeed() File "pyGstViewer.py", line 70, in increasePlaybackSpeed gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE)) ===== CODE ===== import sys, os import pygst pygst.require("0.10") import gst import pygtk, gobject import gtk class GTK_Main: def __init__(self): self.window = gtk.Window(gtk.WINDOW_TOPLEVEL) self.window.connect("destroy", gtk.main_quit, "WM destroy") vbox = gtk.VBox() self.window.add(vbox) vbox.pack_start(gtk.Label("Please type text")) entry = gtk.Entry() vbox.pack_start(entry) self.window.connect("key-press-event", self.keyPress) self.window.show_all() self.pipeline = gst.parse_launch("filesrc " + "location=/home/peter/vid/20121207/00/2012-12-07.00-00-00.mp4 ! " + "queue ! qtdemux ! h264parse ! queue ! ffdec_h264 name=dec ! " + "ffmpegcolorspace ! deinterlace ! xvimagesink sync=false") self.dec = self.pipeline.get_by_name("dec") self.dec.set_property("max-threads", 8); self.pbRate = 1 self.pipeline.set_state(gst.STATE_PLAYING) bus = self.pipeline.get_bus() bus.add_signal_watch() bus.connect("message", self.onMessage) def keyPress(self, widget, event): print "keypress event!!" key = gtk.gdk.keyval_name(event.keyval) if key == "d": self.increasePlaybackSpeed() else: self.decreasePlaybackSpeed() def increasePlaybackSpeed(self): print str(self.pbRate) + " --> " + str(self.pbRate + 4) self.pbRate += 4 self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME, gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0, gst.SEEK_TYPE_NONE, 0)) print "increased playback speed!!!" def decreasePlaybackSpeed(self): self.pbRate -= 2 self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME, gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0, gst.SEEK_TYPE_NONE, 0)) print "decreased playback speed!!!" def onMessage(self, bus, message): t = message.type if t == gst.MESSAGE_EOS: self.pipeline.set_state(gst.STATE_NULL) elif t == gst.MESSAGE_ERROR: self.pipeline.set_state(gst.STATE_NULL) err, debug = message.parse_error() print "Error: %s" % err, debug GTK_Main() gtk.gdk.threads_init() gtk.main() On 12/13/2012 11:18 AM, Juraj Holtak wrote: </div> <blockquote cite="mid:CALnZJ1tzK5+pF8JbGrfcVUiGXz1omUhfwST-EesfZ1TjCjw9Ww@mail.gmail.com" type="cite"> And maybe the xvimagesink should have sync=false set too... maybe... <div class="gmail_quote">On Dec 13, 2012 12:15 PM, "Juraj Holtak" <<a moz-do-not-send="true" href="mailto:juraj.holtak@gmail.com">juraj.holtak@gmail.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Hi, Maybe worth a try: Put an "queue" element before and after ffdec_h264 and use max-threads=<your_cpu_count>. I imagine it worked like this for me but maybe I just had luck... Juraj <div class="gmail_quote">On Dec 13, 2012 2:03 AM, "Peter Rennert" <<a moz-do-not-send="true" href="mailto:p.rennert@cs.ucl.ac.uk" target="_blank">p.rennert@cs.ucl.ac.uk</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Dear all, I want to increase the playback of a video quite drastically. At the moment I am playing a test video sequence with gst-launch-0.10 filesrc location=/path/to/my.mp4 ! qtdemux ! h264parse ! ffdec_h264 name=dec ! ffmpegcolorspace ! deinterlace ! xvimagesink and at some point the video speed cannot get increased further, because it uses only one CPU core to decode the video. Gstreamer starts complaining of getting behind the timestamps of the stream and crashes. I could think of two solutions: - Is there a "native" way of making use of more than just a single core to decode the video frames? I tried to set "max-threads" of ffdec_h264 to 4, but it still only uses a single core. or, - Is there a way of skipping frames? As I need only a "effective" framerate on the screen of about 25fps I could just decode the frames I need and skip the other frames. Then I would not need more CPU power than for realtime playback. or. - Is there a way to split the stream after the qtdemux and use several decoders in a kind of decoder pool (distribute the frames between them) and unify the stream after? My videos are in H264 format. I am not sure if the non-keyframes are encoded with respect to the previous frame or with respect to the last key-frame. In that case I could try to filter the key-frames and send them to every decoder in the pool and decode the any frame with that as reference until the next key-frame. With some management I could try to merge the output of the pool decoder with a input-selector. But that sounds bulky :\ So if one knows how to get to one of the first two solutions, I would be very happy... Cheers, Peter PS I am happy to provide an example video. Maybe something is wrong with the encoding there that prevents ffdec_h264 to use multiple cores _______________________________________________ gstreamer-devel mailing list <a moz-do-not-send="true" href="mailto:gstreamer-devel@lists.freedesktop.org" target="_blank">gstreamer-devel@lists.freedesktop.org</a> <a moz-do-not-send="true" href="http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel" target="_blank">http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a> </blockquote> </div> </blockquote> </div> <fieldset class="mimeAttachmentHeader"></fieldset> <pre wrap="">_______________________________________________ gstreamer-devel mailing list <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a> <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel">http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a> </pre> </blockquote> <fieldset class="mimeAttachmentHeader"></fieldset> <pre wrap="">_______________________________________________ gstreamer-devel mailing list <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a> <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel">http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a> </pre> </blockquote> <fieldset class="mimeAttachmentHeader"></fieldset> <pre wrap="">_______________________________________________ gstreamer-devel mailing list <a class="moz-txt-link-abbreviated" href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a> <a class="moz-txt-link-freetext" href="http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel">http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a> </pre> </blockquote> </body> </html>