[gst-devel] Use videorate in MJPEG pipeline

[Ron McOuat]

I would like to use videorate in a MJPEG pipeline in order to get 1 feed 
from for example an Axis camera, then use a tee connection to split the 
image/jpeg stream and deliver to two different destination sinks but at 
different frame rates. For example deliver image/jpeg at 8 fps to one 
sink and at 2 fps to another sink by using videorate in the 2 fps tee 
branch.

I tested and got videorate running correctly in the following gst-launch 
to illustrate (no tee to 2 sinks for simplicity now) and sending the 
output to a window to observe the output

gst-launch -v gnomevfssrc 
location="http://root:pass@192.168.2.90/axis-cgi/mjpg/video.cgi?fps=8&resolution=640x480" 
do-timestamp=true ! multipartdemux ! 
image/jpeg,width=640,height=480,framerate=8/1 ! jpegdec ! videorate ! 
video/x-raw-yuv,framerate=1/5 ! xvimagesink

where the video rate change occurs in the pipeline section where the 
mime type is video/x-raw-yuv. The pipeline refused to run until 
framerate=8/1 was added to the caps entry after multipartdemux. Watching 
the XWindow image, the camera time updates once every 5 seconds as 
requested for the output framerate of videorate.


Knowing the pad templates on videorate allow only video/x-raw-yuv and 
video/x-raw-rgb, I altered the source to add in image/jpeg as a third 
mime type in the source and sink pad templates

line 112 and 119 in gstvideorate.c changed from

    GST_STATIC_CAPS ("video/x-raw-yuv; video/x-raw-rgb")

to

    GST_STATIC_CAPS ("video/x-raw-yuv; video/x-raw-rgb; image/jpeg")

The reasoning I have is raw video or jpeg images should both stream by 
as a full frame per buffer and since every buffer contains a full frame 
the videorate component should be equally valid for changing frame rates 
on an image/jpeg stream as it is for the raw video stream types. In the 
actual application I don't want to decode and re-encode for performance 
and image quality degradation reasons.

After rebuild, gst-inspect videorate command shows the additional mime 
type in the source and sink pad descriptions (using a local library 
uninstall environment).

So now I build a pipeline to demonstrate if it works like this:

gst-launch -v gnomevfssrc 
location="http://root:pass@192.168.2.90/axis-cgi/mjpg/video.cgi?fps=8&resolution=640x480" 
do-timestamp=true ! multipartdemux ! 
image/jpeg,width=640,height=480,framerate=8/1 ! videorate ! 
image/jpeg,framerate=1/5 ! jpegdec ! xvimagesink

where the jpegdec ! xvimagesink is not where I actually want to consume 
the stream, it is only for debugging purposes.

The pipeline goes into preroll and sits, then if interrupted I get the 
did not want to preroll message - output as follows:

[gst-0.10.21] [ronm at localhost 0.10.21]$ gst-launch -v gnomevfssrc 
location="http://root:pass@192.168.2.90/axis-cgi/mjpg/video.cgi?fps=8&resolution=640x480" 
do-timestamp=true ! multipartdemux ! 
image/jpeg,width=640,height=480,framerate=8/1 ! videorate ! 
image/jpeg,framerate=1/5 ! jpegdec ! xvimagesink

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2: caps = image/jpeg, 
width=(int)640, height=(int)480, framerate=(fraction)8/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:src: caps = 
image/jpeg, width=(int)640, height=(int)480, framerate=(fraction)8/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:sink: caps = 
image/jpeg
/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:sink: caps = 
image/jpeg, width=(int)640, height=(int)480, framerate=(fraction)8/1
/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:src: caps = 
image/jpeg, width=(int)640, height=(int)480, framerate=(fraction)1/5
/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:sink: caps = 
image/jpeg, width=(int)640, height=(int)480, framerate=(fraction)8/1

wait for minutes here, nothing happening
then type ^C

^CCaught interrupt -- handling interrupt.
Interrupt: Stopping pipeline ...
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:src: caps = NULL
/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:sink: caps = NULL
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:src: caps = NULL
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:sink: caps = NULL
/GstPipeline:pipeline0/GstMultipartDemux:multipartdemux0.GstPad:src_0: 
caps = NULL
FREEING pipeline ...


Any comments on what I have missed or should do different. Should I 
consider writing a custom component? To have this work long term the 
videorate component would need to have the additional mime type added to 
the pad templates as an approved enhancement.

Thanks,
Ron