[gst-devel] gnonlin composition - filling queues
Olivier Aubert
olivier.aubert at liris.cnrs.fr
Fri Jul 30 18:39:20 CEST 2010
Hello again
Now that I know how to extract a single A+V segment from a file, I want
to extract multiple ones. The context is a video annotation application
(http://www.advene.org/), where I can define annotations (start
timecode, end timecode, some content) on video files. From the
annotations, I can get multiple visualisations, one of them being a
virtual montage of interesting annotations. I already have an
event-based interactive version, which does not use gnonlin.
Now, I want to be able to save it to a file, and that is where I am
stuck. The following code takes a list of annotations (segments
basically, position and duration information) and generates a
gnlcomposition from them: two gnlfilesources (audio, video) for each
annotation.
The problem is that the pipeline blocks because the vmuxqueue is full
(the current-level-time is higher than max-size-time (1s) ). Setting the
queues to leaky=1 is not satisfactory, it just moves the issue
elsewhere.
What triggers the queue filling is the change of gnlfilesource
(incidentally, the code works with 1 annotation only), so my current
analysis is that the timestamp discontinuity bothers the queue, even
though I have put identity + videorate elements to try to fix the
timestamp issues.
I join the code + gst-debug messages below. Thanks in advance to anyone
who has some advice to share here.
Regards,
Olivier
--- montage renderer code ---
CAPS_VIDEO_STRING = 'video/x-raw-yuv'
CAPS_VIDEO = gst.caps_from_string(CAPS_VIDEO_STRING)
CAPS_AUDIO_STRING = 'audio/x-raw-int;audio/x-raw-float'
CAPS_AUDIO = gst.caps_from_string(CAPS_AUDIO_STRING)
class MontageRenderer(object):
def __init__(self, controller, elements=None):
self.controller = controller
# self.elements is a list of annotations
if elements is None:
elements = []
self.elements = elements
self.encoding_pipe = None
def render(self, outputfile, progress_callback = None):
sourcefile = self.controller.get_default_media()
if not sourcefile:
return
pipedef = "gnlcomposition name=videocomp caps=%s ! queue ! progressreport name=progress silent=true update-freq=1 ! identity single-segment=true ! ffmpegcolorspace ! videorate ! theoraenc ! queue name=vmuxqueue ! oggmux name=mux ! filesink name=sink gnlcomposition name=audiocomp caps=%s ! queue ! identity single-segment=true ! audioconvert ! audiorate ! vorbisenc ! queue name=amuxqueue ! mux." % (CAPS_VIDEO_STRING, CAPS_AUDIO_STRING)
pipe = gst.parse_launch(pipedef)
videocomp = pipe.get_by_name('videocomp')
audiocomp = pipe.get_by_name('audiocomp')
sink = pipe.get_by_name('sink')
bus = pipe.get_bus()
bus.enable_sync_message_emission()
def filesource(a, pos, caps):
"""Create a filesource.
"""
e = gst.element_factory_make('gnlfilesource')
e.set_property("location", sourcefile)
e.set_property("caps", caps)
e.set_property("start", pos * gst.MSECOND)
e.set_property("duration", a.fragment.duration * gst.MSECOND)
e.set_property("media-start", a.fragment.begin * gst.MSECOND)
e.set_property("media-duration", a.fragment.duration * gst.MSECOND)
return e
pos = 0
for a in self.elements:
e = filesource(a, pos, CAPS_VIDEO)
videocomp.add(e)
e = filesource(a, pos, CAPS_AUDIO)
audiocomp.add(e)
pos += a.fragment.duration
sink.set_property("location", outputfile)
def on_bus_message(bus, message):
if message.type == gst.MESSAGE_STATE_CHANGED:
old, new, pending = message.parse_state_changed()
print "STATE", old.value_nick, new.value_nick, pending.value_nick
elif message.type == gst.MESSAGE_EOS:
print " EOS"
pipe.set_state(gst.STATE_NULL)
progress_callback(None)
elif message.structure:
s=message.structure
print "MSG " + bus.get_name() + ": " + s.to_string()
if s.get_name() == 'progress' and progress_callback is not None:
progress_callback(s['percent-double'] / 100)
return True
bus.add_signal_watch()
bus.connect('message', on_bus_message)
pipe.set_state(gst.STATE_PLAYING)
self.encoding_pipe = pipe
return True
def finalize(self):
if self.encoding_pipe is not None:
self.encoding_pipe.set_state(gst.STATE_NULL)
self.encoding_pipe = None
--- GST_DEBUG output ---
0:00:30.433191931 1064 0x9d883fc0 DEBUG ffmpeg gstffmpegdec.c:1764:gst_ffmpegdec_video_frame:<ffdec_mpeg45> picture: display 0
0:00:30.433206807 1064 0xb238738 DEBUG multiqueue gstmultiqueue.c:1589:single_queue_check_full:<multiqueue5> queue 1: visible 4/5, bytes 3749/2097152, time 182583334/0
0:00:30.433208833 1064 0x9d883fc0 DEBUG ffmpeg gstffmpegdec.c:1766:gst_ffmpegdec_video_frame:<ffdec_mpeg45> picture: opaque 0xb18acb8
0:00:30.433225944 1064 0xb238738 DEBUG multiqueue gstmultiqueue.c:874:apply_buffer:<multiqueue5> queue 1, last_stop updated to 0:00:23.072541666
0:00:30.433228528 1064 0x9d883fc0 DEBUG ffmpeg gstffmpegdec.c:1768:gst_ffmpegdec_video_frame:<ffdec_mpeg45> repeat_pict:0
0:00:30.433245710 1064 0xb238738 DEBUG multiqueue gstmultiqueue.c:797:update_time_level:<multiqueue5> queue 1, sink 0:00:01.872541666, src 0:00:01.637166666
0:00:30.433096246 1064 0xacaede0 DEBUG basetransform gstbasetransform.c:1399:gst_base_transform_prepare_output_buffer:<ffmpegcsp0> discard buffer, reuse input buffer
0:00:30.433248853 1064 0x9d883fc0 DEBUG ffmpeg gstffmpegdec.c:1770:gst_ffmpegdec_video_frame:<ffdec_mpeg45> interlaced_frame:0
0:00:30.434205135 1064 0xb238738 DEBUG avidemux gstavidemux.c:4522:gst_avi_demux_advance:<avidemux5> moved forwards from 576 to 577
0:00:30.434210443 1064 0x9d883fc0 DEBUG ffmpeg gstffmpegdec.c:1792:gst_ffmpegdec_video_frame:<ffdec_mpeg45> timestamp discont, we have DTS as timestamps
0:00:30.434232373 1064 0x9d883fc0 DEBUG ffmpeg gstffmpegdec.c:1815:gst_ffmpegdec_video_frame:<ffdec_mpeg45> DTS as timestamps, interpolate
0:00:30.434242011 1064 0xb238738 DEBUG basesrc gstbasesrc.c:2031:gst_base_src_update_length:<source> reading offset 6688422, length 6, size 79857152, segment.stop -1, maxsize 79857152
0:00:30.434259262 1064 0xb238738 DEBUG basesrc gstbasesrc.c:2121:gst_base_src_get_range:<source> calling create offset 6688422 length 6, time 0
0:00:30.434259961 1064 0x9d883fc0 DEBUG query gstquery.c:359:gst_query_new: creating new query 0xadc34e0 8
0:00:30.434282799 1064 0xb238738 DEBUG basesrc gstbasesrc.c:1995:gst_base_src_do_sync:<source> no sync needed
0:00:30.434281612 1064 0x9d883fc0 DEBUG GST_PADS gstpad.c:3674:gst_pad_query:<multiqueue5:src0> sending query 0xadc34e0
0:00:30.434298095 1064 0xb238738 DEBUG basesrc gstbasesrc.c:2162:gst_base_src_get_range:<source> buffer ok
0:00:30.434302844 1064 0x9d883fc0 DEBUG GST_PADS gstpad.c:3674:gst_pad_query:<avidemux5:video_00> sending query 0xadc34e0
0:00:30.434314578 1064 0xb238738 DEBUG avidemux gstavidemux.c:4702:gst_avi_demux_loop_data:<avidemux5> Pushing buffer of size 6, ts 0:00:23.040000000, dur 0:00:00.040000000, off 576, off_end 577
0:00:30.434324076 1064 0x9d883fc0 DEBUG ffmpeg gstffmpegdec.c:1952:gst_ffmpegdec_video_frame:<ffdec_mpeg45> return flow 0, out 0xb18acb8, len 24927
0:00:30.434332038 1064 0xb238738 DEBUG multiqueue gstmultiqueue.c:1589:single_queue_check_full:<multiqueue5> queue 0: visible 4/209, bytes 35683/2097152, time 160000000/0
0:00:30.434344610 1064 0x9d883fc0 DEBUG queue_dataflow gstqueue.c:963:gst_queue_chain:<queue0> queue is full, waiting for free space
0:00:30.434347473 1064 0xb238738 DEBUG multiqueue gstmultiqueue.c:874:apply_buffer:<multiqueue5> queue 0, last_stop updated to 0:00:23.080000000
0:00:30.434275396 1064 0xacaede0 DEBUG basetransform gstbasetransform.c:2036:gst_base_transform_handle_buffer:<ffmpegcsp0> element is in passthrough
0:00:30.434361582 1064 0xb238738 DEBUG multiqueue gstmultiqueue.c:797:update_time_level:<multiqueue5> queue 0, sink 0:00:01.880000000, src 0:00:01.680000000
0:00:30.434376318 1064 0xb238738 DEBUG avidemux gstavidemux.c:4522:gst_avi_demux_advance:<avidemux5> moved forwards from 576 to 577
0:00:30.434372826 1064 0xacaede0 DEBUG videorate gstvideorate.c:696:gst_video_rate_chain:<videorate0> got buffer with timestamp 0:00:07.880000000
0:00:30.434394477 1064 0xb238738 DEBUG basesrc gstbasesrc.c:2031:gst_base_src_update_length:<source> reading offset 6711828, length 622, size 79857152, segment.stop -1, maxsize 79857152
0:00:30.435127334 1064 0xb238738 DEBUG basesrc gstbasesrc.c:2121:gst_base_src_get_range:<source> calling create offset 6711828 length 622, time 0
0:00:30.435148216 1064 0xb238738 DEBUG basesrc gstbasesrc.c:1995:gst_base_src_do_sync:<source> no sync needed
0:00:30.435161626 1064 0xb238738 DEBUG basesrc gstbasesrc.c:2162:gst_base_src_get_range:<source> buffer ok
0:00:30.435176293 1064 0xb238738 DEBUG avidemux gstavidemux.c:4702:gst_avi_demux_loop_data:<avidemux5> Pushing buffer of size 622, ts 0:00:23.072541666, dur 0:00:00.025916667, off 18446744073709551615, off_end 18446744073709551615
0:00:30.435195290 1064 0xb238738 DEBUG multiqueue gstmultiqueue.c:1589:single_queue_check_full:<multiqueue5> queue 1: visible 5/5, bytes 5016/2097152, time 235375000/0
0:00:30.435210935 1064 0xb238738 DEBUG multiqueue gstmultiqueue.c:1512:single_queue_overrun_cb:<multiqueue5> queue 0: visible 5/209, bytes 35689/2097152, time 200000000/0
0:00:30.435225253 1064 0xb238738 DEBUG multiqueue gstmultiqueue.c:1512:single_queue_overrun_cb:<multiqueue5> queue 1: visible 5/5, bytes 5016/2097152, time 235375000/0
0:00:30.435239081 1064 0xb238738 DEBUG multiqueue gstmultiqueue.c:1589:single_queue_check_full:<multiqueue5> queue 1: visible 5/5, bytes 5016/2097152, time 235375000/0
0:00:30.478657663 1064 0xacaede0 DEBUG queue_dataflow gstqueue.c:963:gst_queue_chain:<vmuxqueue> queue is full, waiting for free space
More information about the gstreamer-devel
mailing list