Why is videobox/videomixer dropping frames?

[wally bkg]

I need binocular video for an image processing application, so it seemed
logical to "stack" cameraR and cameraL into a single frame using videobox
and videomixer and then pass that into my appsink for processing as a single
buffer.

Unfortunately performance is terrible with jerky video and occasional
messages about dropped frames.  Using Ubuntu 10.04 64-bit, all patches as of
this morning and its "stock" gstreamer-0.10.28 setup on an AMD quad core
processor with 8GB RAM.  (I had tried the PPA version a while back, but it
hosed my system, recovering was a PITA so I'm not amenable to trying it
again!)

Sample pipeline with gst-launch:

gst-launch v4l2src device=/dev/video1 ! ffmpegcolorspace !
video/x-raw-yuv,format=\(fourcc\)AYUV,width=640,height=480 !\
 videobox border-alpha=0 top=-480 bottom=-48 ! videomixer name=mix
sink_0::alpha=1.0 sink_1::alpha=1.0 ! ffmpegcolorspace !\
 xvimagesink  v4l2src device=/dev/video0 ! ffmpegcolorspace !
video/x-raw-yuv,format=\(fourcc\)AYUV,width=640,height=480 !\
 ffmpegcolorspace ! mix.


But running two instances gives smooth video despite using significantly
more CPU according to top:

gst-launch v4l2src device=/dev/video1 ! ffmpegcolorspace !
video/x-raw-yuv,format=\(fourcc\)AYUV,width=640,height=480 !\
 ffmpegcolorspace ! xvimagesink  v4l2src device=/dev/video0 !
ffmpegcolorspace !\
 video/x-raw-yuv,format=\(fourcc\)AYUV,width=640,height=480 !
ffmpegcolorspace ! xvimagesink

I've 5 different v4l2 capture devices (two PCI and 3 USB) and all pairs have
the perfromance problem including one USB device that has two full frame
rate capture inputs (Sensoray 2255S).  All pairings give smooth video with
two instances.

The video has perfect "genlock" as its one VCR output fed to the capture
devices with a distribution amplifier.  The 48 extra scan lines at the
bottom of the stacked frames I planned to fill with barcoded metadata,
removing them didn't change anything.  In fact I can run both gst-launch
commands from separate terminal windows and its very obvious when the
videobox/videomixer drops frames.

Is there something I'm missing that can fix this?  The two cameras in a
single buffer would seem to make the downstream code much more
understandable, but I can use the two instance solution.

Any rational reason why videobox supports video/x-raw-gray and videomixer
does not?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20110811/42ad14ad/attachment.htm>