playbin2 with MJPEG streams over http: long delay before playback starts, and buffering delays on each frame

Wed Jun 29 15:52:53 PDT 2011

I'm trying to use GStreamer (0.10.28 and 0.10.35 tested, Linux) to play
MJPEG streams delivered over HTTP from an Axis video camera.  The
network or the camera tend to be somewhat slow.

If I create the following pipeline, everything works great:

gst-launch-0.10 souphttpsrc
location=http://ccd1611-403/mjpg/video.mjpg ! multipartdemux ! jpegdec !
autovideosink

However, the final outcome is to use gstreamer inside of Qt's
QMultimediaKit, which automatically uses a playbin or playbin2 element,
and doesn't offer any capability to manually construct the pipeline.  In
this situation:

gst-launch-0.10 playbin2 uri=http://ccd1611-403/mjpg/video.mjpg

I'm experiencing two problems:

1)  Playback doesn't start for about 35 seconds.  During this time,
frames are received but I see "Buffering... 0%".  After about 30
seconds, the output window appears and all the frames that piled up
during this time are displayed very quickly.   

[Some people have suggested elsewhere on this list that the
multipartdemux or multiqueue might be waiting to find out if there is an
audio stream, and then blocking until the audio stream is up (or
confirmed non-existent).

2)  Once the output window is created and playback starts,  playback
pauses and buffers for about 3s between each set of frames... making the
real-time experience basically unusable.   It will pause and buffer for
3s, and then quickly show all the frames received during those 3s, and
then pause and buffer again [repeat].

I can get around this by setting playbin2's buffer-duration=10000, and
then each frame is displayed as soon as it is received.  However, the
stream now alternates between playing and paused on every single frame.
[Unfortunately, I have no way of specifying the buffer-duration property
when using QMultimediaKit's gst backend.]

This makes the camera essentially useless as a real-time video monitor.
I've tested the same camera(s) using VLC, and it starts playing
immediately and without 3s pauses.

Any suggestions for what could be going on here?  Any suggestions for
how to work around it?  If I could set up the pipeline manually like the
first example, all would be well.  However, I'm stuck using whatever
playbin2 creates because I need to use this inside the multimedia module
that comes with Qt/QtMobility.

Thanks a lot!
-Mark

================
Mark Boots

Controls Analyst
Canadian Light Source
University of Saskatchewan

Mark.Boots at lightsource.ca

101 Perimeter Rd
Saskatoon, SK
S7N 0X4
================

Verbose log:  No limit on buffering-duration: (Shows problem 1 and 2)

gst-launch-0.10 -v --gst-plugin-spew playbin2
uri=http://ccd1611-403/mjpg/video.mjpg
Setting pipeline to PAUSED ...
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0: connection-speed =
0
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0: download = FALSE
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0: uri =
"http://ccd1611-403/mjpg/video.mjpg"
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0: use-buffering =
FALSE
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0: buffer-duration =
-1
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0: buffer-size = -1
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0: source =
(GstSoupHTTPSrc) source
Pipeline is PREROLLING ...
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstTypeFindElement:typefindelement0.GstPad:src: caps = multipart/x-mixed-replace
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/GstTypeFindElement:typefind: force-caps = multipart/x-mixed-replace
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20: sink-caps = multipart/x-mixed-replace
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/GstTypeFindElement:typefind.GstPad:src: caps = multipart/x-mixed-replace
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstQueue2:queue20.GstPad:sink: caps = multipart/x-mixed-replace
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstQueue2:queue20.GstPad:src: caps = multipart/x-mixed-replace
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/GstTypeFindElement:typefind.GstPad:sink: caps = multipart/x-mixed-replace
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20.GstGhostPad:sink: caps = multipart/x-mixed-replace
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20.GstGhostPad:sink.GstProxyPad:proxypad0: caps = multipart/x-mixed-replace
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/GstMultipartDemux:multipartdemux0.GstPad:sink: caps = multipart/x-mixed-replace
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/GstMultiQueue:multiqueue0.GstPad:sink0: caps = image/jpeg
buffering... 0%

#[35 seconds go by right here]

/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPlayBin2:playbin20/GstPlaybin2InputSelector:playbin2inputselector0.GstPlaybin2SelectorPad:sink0: always-ok = FALSE
/GstPlayBin2:playbin20/GstPlaybin2InputSelector:playbin2inputselector0:
active-pad = (GstPlaybin2SelectorPad) sink0
/GstPlayBin2:playbin20/GstPlaybin2InputSelector:playbin2inputselector0.GstPlaybin2SelectorPad:sink0: tags = ((GstTagList*) 0xb6500aa8)
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/GstMultiQueue:multiqueue0.GstPad:src0: caps = image/jpeg
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/GstJpegDec:jpegdec0.GstPad:sink: caps = image/jpeg
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0.GstGhostPad:src0:
caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640,
height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20.GstDecodePad:src0: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/GstJpegDec:jpegdec0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaybin2InputSelector:playbin2inputselector0.GstPlaybin2SelectorPad:sink0: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0.GstGhostPad:src0.GstProxyPad:proxypad4: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20.GstDecodePad:src0.GstProxyPad:proxypad3: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaybin2InputSelector:playbin2inputselector0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstQueue:vqueue.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin.GstGhostPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaySink:playsink0.GstGhostPad:video_raw_sink:
caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640,
height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaySink:playsink0.GstGhostPad:video_raw_sink.GstProxyPad:proxypad5: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin.GstGhostPad:sink.GstProxyPad:proxypad7: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstQueue:vqueue.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstFFMpegCsp:vconv.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstFFMpegCsp:vconv.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstVideoScale:vscale.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstVideoScale:vscale.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstAutoVideoSink:videosink/GstXvImageSink:videosink-actual-sink-xvimage.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstAutoVideoSink:videosink.GstGhostPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
/GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstAutoVideoSink:videosink.GstGhostPad:sink.GstProxyPad:proxypad6: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)640, height=(int)480, framerate=(fraction)0/1
Prerolled, waiting for buffering to finish...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

# [Lots of frames... all piled up during those 30 seconds... shown very
quickly here]

Buffering, setting pipeline to PAUSED ...
Prerolled, waiting for buffering to finish...  # [3 seconds go by]
Done buffering, setting pipeline to PLAYING ...  # [All frames received
during those 3 seconds now shown quickly]
Buffering, setting pipeline to PAUSED ...
Prerolled, waiting for buffering to finish... # [3 seconds go by]
Done buffering, setting pipeline to PLAYING ... #[All frames received
during those 3 seconds now shown quickly]

# [and so on:]

Buffering, setting pipeline to PAUSED ...
Prerolled, waiting for buffering to finish...
Done buffering, setting pipeline to PLAYING ...
Buffering, setting pipeline to PAUSED ...
Prerolled, waiting for buffering to finish...
Done buffering, setting pipeline to PLAYING ...
Buffering, setting pipeline to PAUSED ...
Prerolled, waiting for buffering to finish...
Done buffering, setting pipeline to PLAYING ...
# [...]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20110629/6087a2a8/attachment.html>