[gst-devel] Problem Trying To Use v4l2src

Wed Aug 19 18:40:23 CEST 2009

>>It seems I need to  write a gstreamer plugin to get access to the actual data being passed
>> through the gstreamer pipeline.
>
>Not necessarily, but if you want to manipulate the data in the middle of
>a pipeline writing an element is usually the right thing to do.
>You can get data out of a pipeline using the appsink element, or the
>fakesink element with a "handoff" signal handler, or the identity
>element with a "handoff" signal handler, or by setting up a pad probe.

Thanks so much for this and the quick response Tim, I'll look into it.   

It could make for a much easier to maintain project if I simply have a gstreamer application instead of a gstreamer application and plugin.  But if the documentation or sample programs were better I can clearly see the advantage of using a plugin.

>> My question is how do I initialize the v4l2src plugin?  I need to set NTSC mode
>> (some of our cards default to PAL on power up), image size to 640x480
>> (although it seems I could use one of the resize or cropping
>> plugins),  select the composite or S-Video input (most cards seem to
>> default to tuner), and maybe set 8-bit grayscale video mode (our
>> cameras are monochrome).
>  
>The desired pixel format/layout and resolution can be set by putting a
>capsfilter with suitable filter caps after the v4l2src element. This
>usually assumes the camera/drivers/libv4l support the exact format and
>resolution desired. If you just want to get data in a particular
>format/size no matter what you can put some converters like
>ffmpegcolorspace and videoscale between v4l2src and your capsfilter, and
>those elements will try their best to convert the data from the camera
>into the desired format if the format isn't supported.

The devil is in the details of exactly how I put a "capsfilter with suitable filter caps after the v4l2src element".

I'd prefer to avoid format conversions if possible beyond what it takes for ximagesink to display on a 16-bit or 24-bit or 32-bit X server.

Playing around with this pipeline: 
        v4l2src ! video/x-raw-yuv, framerate=\(fraction\)30000/1001 ! gamma ! wally ! xvimagesink

where wally is my gst-template made gstfilter toy plugin, it seems I'm getting I420 video.  I've not had a lot of luck Googling for for the details of the data layouts for the various formats, but poking around the SMTPE plugin,  I420 seems to be something I can use as the buffer seems to consist of width*height bytes of intensity (essentially grayscale) data followed by width*height/2 arrays of half resolution color channel data which I could ignore.

Since my application needs to run a pair of genlocked cameras into two capture cards, I prefer not to be passing around data I intend to ignore.  Although with performance of today's machines its probably not an issue.  Enhancing the system for higher frame rates would be much higher priority over trying to incorporate any color data into the analysis.

I'm finding the details of specifying a colorspace in ffmpegcolorspace or videoscale lacking.  I'd rather set the capture to be width=640 height=480 from the start instead of cropping something line 704x480 to 640x480.  I'm all in favor of having things auto-negotiate formats, but how to I tell them my choices are when it matters?

>NTSC/PAL selection is done using the GstTuner interface (see
>gst-plugins-base libraries docs), I believe. The device might need to be
>
There seems to be a lot of poorly if at all documented "common knowledge" about the gstreamer structure and conventions.
At: http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-plugins/html/ I don't see any gstTuner or tuner anywhere in the list of base Elements or  base Plugins.

I did find this: http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-gsttunernorm.html but the "Details" leaves out the actual strings that sets the norm and what "property" I'd be setting or method to invoke to change it, since gst-inspect tuner or gsttuner returns "No such element".  I'm not clairvoyant.  I'm left without a clue as to how I'd have "gst-launch v4l2src" set the norm to NTSC, short of perhaps grepping the gstreamer tree in hopes of finding NTSC or ntsc somewhere.  

Automatically generated documentation I find generally voluminous, repetitive, and mostly useless except for looking up the details you've forgotten once you've already become an "expert".

A big factor in my evaluation is the fact that retirement is on the horizon, and its very unlikely I'll be here past three more years, so choosing a methodology for the next version of my application (which I inherited and ported from the Mac in 2005) that has an enormous learning curve would be a disservice to my employer and my replacement.  On the plus side, learning and using gstreamer might facilitate porting to Windows or back to Mac;  but a major plus of Linux is I can "clone" the system and give it to colleagues to use without having to invoke our bureaucracy to buy or sell anything.  So on balance, facilitating a port away from Linux could make the system less useful overall.

>> If I run an application like TVtime first, the card then works with
>> v4l2src in a simple gst-launch pipeline to test my plugin.  But it
>> seems pretty fragile with respect to video errors often requiring
>> another run of TVtime to reset the card so v4l2src will work.
>
>What errors do you get? What does TVtime do to reset it?
>
I can't cause the problem on command except after a reboot, but I suspect bad spots on the SVHS tapes I play back for development/testing might have something to do with it.  I've not dissected TVtime yet to figure out what its doing (that is Plan B, if I choose not to use gstreamer), but I  know it just works from a cold boot and recovers whatever got messed up while I was playing with gstreamer every time.

On boot up or after my screw-up or error, either my gst-launch pipeline just dies in about a second:
$ gst-launch v4l2src ! video/x-raw-yuv, framerate=$fraction$30000/1001 ! xvimagesink

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
WARNING: from element /pipeline0/v4l2src0: Could not set parameters on device '/dev/video0'
Additional debug info:
v4l2src_calls.c(1165): gst_v4l2src_set_capture (): /pipeline0/v4l2src0:
system error: Invalid argument
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /pipeline0/v4l2src0: Failed trying to get video frames from device '/dev/video0'.
Additional debug info:
v4l2src_calls.c(1069): gst_v4l2src_grab_frame (): /pipeline0/v4l2src0:The buffer type is not supported, or the index is out of bounds, or no buffers have been allocated yet, or the userptr or length are invalid.    device /dev/video0
Execution ended after 998910962 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
FREEING pipeline ...

or it displays a messed up window with a few mostly correct scanlines at the top with a bunch of green tinged distorted lines filling the window.

Then if I start and stop TVtime, the gstreamer pipeline will work fine afterwards:
$ gst-launch v4l2src ! video/x-raw-yuv, framerate=$fraction$30000/1001 ! xvimagesink

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
WARNING: from element /pipeline0/v4l2src0: Could not set parameters on device '/dev/video0'
Additional debug info:
v4l2src_calls.c(1165): gst_v4l2src_set_capture (): /pipeline0/v4l2src0:
system error: Invalid argument
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /pipeline0/xvimagesink0: Output window was closed
Additional debug info:
xvimagesink.c(1223): gst_xvimagesink_handle_xevents (): /pipeline0/xvimagesink0
Execution ended after 9357423821 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
FREEING pipeline ...

I haven't been able to get a clue as to what:

WARNING: from element /pipeline0/v4l2src0: Could not set parameters on device '/dev/video0'
Additional debug info:
v4l2src_calls.c(1165): gst_v4l2src_set_capture (): /pipeline0/v4l2src0:
system error: Invalid argument

is trying to tell me, other than its clearly not a problem in terms of seeing the video playing in the ximagesink window and is there independent of if the pipeline actually works or not.