[gst-devel] Slight audio/video sync problem - any suggestions?
Jason L. Buberel
jason at buberel.org
Fri Mar 7 10:04:56 CET 2008
There must be something a bit off with that command.
When issued, I get one still frame of video onscreen and the output:
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstAudioSrcClock
I then press Ctrl-C and I see:
Caught interrupt -- handling interrupt.
Interrupt: Setting pipeline to PAUSED ...
Execution ended after 35390780000 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
I then press Ctrl-C again to exit. The resulting file '1.ogg' is empty.
Using the debug level 4 (instead of 3) flag with
'framerate=30/1,width=800,height=600' I get:
> gst-launch-0.10 v4l2src num-buffers=1 ! ffmpegcolorspace ! fakesink
0:00:00.338550000 9822 0x804e050 DEBUG v4l2src
gstv4l2src.c:398:gst_v4l2src_fixate:<v4l2src0> fixating caps
video/x-raw-yuv, format=(fourcc)YUY2, width=(int)160, height=(int)120,
framerate=(fraction){ 30/1, 25/1, 20/1, 15/1, 10/1, 5/1 }
0:00:00.339356000 9822 0x804e050 DEBUG v4l2src
gstv4l2src.c:854:gst_v4l2src_set_caps:<v4l2src0> trying to set_capture
160x120 at 30/1 fps, format YUV 4:2:2 (YUYV)
But this command give a different result:
> gst-launch-0.10 v4l2src num-buffers=1 ! fakesink
0:00:00.203129000 9849 0x804e050 DEBUG v4l2src
gstv4l2src.c:398:gst_v4l2src_fixate:<v4l2src0> fixating caps image/jpeg,
width=(int)160, height=(int)120, framerate=(fraction){ 30/1, 25/1, 20/1,
15/1, 10/1, 5/1 }
0:00:00.203615000 9849 0x804e050 DEBUG v4l2src
gstv4l2src.c:854:gst_v4l2src_set_caps:<v4l2src0> trying to set_capture
160x120 at 30/1 fps, format MJPEG
What video/x-raw-? parameter would I have to use in order to force v4l2
to use what the 'luvcview' command indicates it can handle: 30fps @
960x720 in MJPEG format?
-jason
Edgard Lima wrote:
> try exactly this one:
>
> gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv,
> format=\(fourcc\)I420, width=800, height=600 ! tee name=t ! queue2 !
> timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg
> alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 !
> videoscale ! xvimagesink sync=false
>
> what happens?
>
> 1- works fine, ok done
>
> 2- 100% cpu ... not good result
>
> then decrease the resolution
>
> to see the possible resolution try
>
> GST_DEBUG=*v4l2*:3 gst-launch-0.10 v4l2src num-buffers=1 ! fakesink
>
> * I can't see how to solve the problem of 100% cpu without some QoS
> smartness to decrease the video framerate
>
> BR,
> Edgard
>
> ext Jason L. Buberel wrote:
>
>> Some additional information on my device and it's capabilities.
>>
>> According to 'v4l-info':
>>
>> video capture
>> VIDIOC_ENUM_FMT(0,VIDEO_CAPTURE)
>> index : 0
>> type : VIDEO_CAPTURE
>> flags : 1
>> description : "MJPEG"
>> pixelformat : 0x47504a4d [MJPG]
>> VIDIOC_ENUM_FMT(1,VIDEO_CAPTURE)
>> index : 1
>> type : VIDEO_CAPTURE
>> flags : 0
>> description : "YUV 4:2:2 (YUYV)"
>> pixelformat : 0x56595559 [YUYV]
>>
>> I am assuming that the second format corresponds to 'video/x-raw-yuv' in
>> the gstreamer pipeline. As far as I can tell, there is no gstreamer
>> support for reading the camera data using this format.
>>
>> Starting with the basic - the following command works:
>>
>>
>>> gst-launch-0.10 v4l2src ! ffmpegcolorspace !
>>>
>> video/x-raw-yuv,width=800,height=600 ! xvimagesink sync=false
>>
>> Adding the timeoverlay also works:
>>
>>
>>> gst-launch-0.10 v4l2src ! ffmpegcolorspace !
>>>
>> video/x-raw-yuv,width=800,height=600 ! timeoverlay ! xvimagesink sync=false
>>
>> Adding the format=I420 also works when 'ffmpegcolorspace' comes early in
>> the pipline:
>>
>>
>>> gst-launch-0.10 -v v4l2src ! ffmpegcolorspace !
>>>
>> video/x-raw-yuv,width=800,height=600,format=\(fourcc\)I420 ! timeoverlay
>> ! xvimagesink sync=false
>>
>> But when I then attempt to remove the ffmpegcolorspace:
>>
>>
>>> gst-launch-0.10 -v v4l2src !
>>>
>> video/x-raw-yuv,width=800,height=600,format=\(fourcc\)I420 ! timeoverlay
>> ! xvimagesink sync=false
>>
>> Setting pipeline to PAUSED ...
>> ERROR: Pipeline doesn't want to pause.
>> ERROR: from element /pipeline0/v4l2src0: Could not negotiate format
>> Additional debug info:
>> gstbasesrc.c(2062): gst_base_src_start (): /pipeline0/v4l2src0:
>> Check your filtered caps, if any
>> Setting pipeline to NULL ...
>> FREEING pipeline ...
>>
>> However, if I change the format to YUV2, it works:
>>
>>
>>> gst-launch-0.10 -v v4l2src !
>>>
>> video/x-raw-yuv,width=800,height=600,format=\(fourcc\)YUY2 !
>> ffmpegcolorspace ! timeoverlay ! xvimagesink sync=false
>>
>> This tells me that the camera simply does not support the I420 pixel
>> format - only YUY. And because the 'timeoverlay' always expects an input
>> format of 'video/x-raw-yuv,format=I420', I have to put the
>> ffmpegcolorspace element after the video capture element.
>>
>> I'm going to work a bit more on your command to see if I can get the
>> 'tee' element working properly.
>>
>> -jason
>>
>>
>>
>>
>> Edgard Lima wrote:
>>
>>> Hi,
>>>
>>> nice your blog
>>>
>>> http://www.buberel.org/serendipity/
>>>
>>> and nice
>>>
>>> http://gentrans.sourceforge.net/
>>>
>>> project
>>>
>>> But I think, unfortunately, it doesn't help for something more robust
>>> and professional.
>>>
>>> In case of your pipeline:
>>>
>>> gst-launch-0.10 v4l2src queue-size=16 ! stamp sync-margin=1
>>> sync-interval=1
>>> ! video/x-raw-yuv,width=800,height=600,framerate=15/1
>>> ! queue2 max-size-buffers=1000 max-size-bytes=0 max-size-time=0 !
>>> ffmpegcolorspace
>>> ! theoraenc quality=60 name=venc alsasrc device="hw:1,0"
>>> ! audio/x-raw-int,rate=16000,channels=1,depth=16
>>> ! audioconvert ! queue2 max-size-buffers=1000 max-size-bytes=0
>>> max-size-time=0
>>> ! vorbisenc quality=0.9 name=aenc oggmux name=mux ! filesink
>>> location=test.ogg aenc. ! mux. venc. ! mux
>>>
>>> Considering you have 100% CPU usage, what happens is that you are
>>> delaying the problem a bit by using a large queue (1000) and again
>>> delaying the problem a bit with "stamp" element
>>>
>>> Please try with the following pipeline:
>>>
>>> gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv,
>>> format=\(fourcc\)I420, width=640, height=480 ! tee name=t ! queue2 !
>>> timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg
>>> alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 !
>>> videoscale ! xvimagesink sync=false
>>>
>>> Then try change the resolution to the one your camera accept
>>> (width=800,height=600,framerate=15/1) and see if you can remove
>>> ffmpegcolorspace and videoscale
>>>
>>> if you don't have 100% you will have best result.
>>>
>>> if you have 100% then try to increase the queue size (or also use stamp)
>>> so it would be ok to make short videos (as you already did).
>>>
>>> - - - - - - - - - - - - - - - - - --
>>>
>>> Now I would like discuss what would be the best way to record audio and
>>> video in 100% cpu case.
>>>
>>> if I use
>>>
>>> gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv,
>>> format=\(fourcc\)I420, width=1280, height=960 ! tee name=t ! queue2 !
>>> timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg
>>> alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 !
>>> videoscale ! xvimagesink sync=false
>>>
>>> then my file isn't recorded properly (Can't record audio fast
>>> enough...this discontinuity breaks the recording)
>>>
>>> BUT
>>>
>>> gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv,
>>> format=\(fourcc\)I420, width=1280, height=960 ! tee name=t ! queue2 !
>>> timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg
>>> alsasrc ! queue2 ! audioconvert ! vorbisenc ! oggmux ! filesink
>>> location=2.ogg mux. t. ! queue leaky=2 ! videoscale ! xvimagesink
>>> sync=false
>>>
>>> Both audio and video file are recorded fine (individually). In addition,
>>> the audio file has duration greater than video file. I think it happens
>>> because of video dropping.
>>>
>>> But I think it is close to the best thing that could be done. So what I
>>> would like to do is to give priority to audio as much as possible, and
>>> drop videos, but still having audio and video sync. So, How to do it?
>>>
>>> Best Regards,
>>> Edgard
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20080307/31c6f831/attachment.htm>
More information about the gstreamer-devel
mailing list