[gst-devel] Slight audio/video sync problem - any suggestions?

[Edgard Lima]

try exactly this one:

gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv,
format=\(fourcc\)I420, width=800, height=600 ! tee name=t ! queue2 !
timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg
alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 !
videoscale ! xvimagesink sync=false

what happens?

1- works fine, ok done

2- 100% cpu ... not good result

then decrease the resolution

to see the possible resolution try

GST_DEBUG=*v4l2*:3 gst-launch-0.10 v4l2src num-buffers=1 ! fakesink

* I can't see how to solve the problem of 100% cpu without some QoS
smartness to decrease the video framerate

BR,
Edgard

ext Jason L. Buberel wrote:
> Some additional information on my device and it's capabilities.
> 
> According to 'v4l-info':
> 
> video capture
>    VIDIOC_ENUM_FMT(0,VIDEO_CAPTURE)
>        index                   : 0
>        type                    : VIDEO_CAPTURE
>        flags                   : 1
>        description             : "MJPEG"
>        pixelformat             : 0x47504a4d [MJPG]
>    VIDIOC_ENUM_FMT(1,VIDEO_CAPTURE)
>        index                   : 1
>        type                    : VIDEO_CAPTURE
>        flags                   : 0
>        description             : "YUV 4:2:2 (YUYV)"
>        pixelformat             : 0x56595559 [YUYV]
> 
> I am assuming that the second format corresponds to 'video/x-raw-yuv' in
> the gstreamer pipeline. As far as I can tell, there is no gstreamer
> support for reading the camera data using this format.
> 
> Starting with the basic - the following command works:
> 
>> gst-launch-0.10 v4l2src ! ffmpegcolorspace !
> video/x-raw-yuv,width=800,height=600 ! xvimagesink sync=false
> 
> Adding the timeoverlay also works:
> 
>> gst-launch-0.10 v4l2src ! ffmpegcolorspace !
> video/x-raw-yuv,width=800,height=600 ! timeoverlay ! xvimagesink sync=false
> 
> Adding the format=I420 also works when 'ffmpegcolorspace' comes early in
> the pipline:
> 
>> gst-launch-0.10 -v v4l2src ! ffmpegcolorspace !
> video/x-raw-yuv,width=800,height=600,format=\(fourcc\)I420 ! timeoverlay
> ! xvimagesink sync=false
> 
> But when I then attempt to remove the ffmpegcolorspace:
> 
>> gst-launch-0.10 -v v4l2src !
> video/x-raw-yuv,width=800,height=600,format=\(fourcc\)I420 ! timeoverlay
> ! xvimagesink sync=false
> 
> Setting pipeline to PAUSED ...
> ERROR: Pipeline doesn't want to pause.
> ERROR: from element /pipeline0/v4l2src0: Could not negotiate format
> Additional debug info:
> gstbasesrc.c(2062): gst_base_src_start (): /pipeline0/v4l2src0:
> Check your filtered caps, if any
> Setting pipeline to NULL ...
> FREEING pipeline ...
> 
> However, if I change the format to YUV2, it works:
> 
>> gst-launch-0.10 -v v4l2src !
> video/x-raw-yuv,width=800,height=600,format=\(fourcc\)YUY2 !
> ffmpegcolorspace ! timeoverlay ! xvimagesink sync=false
> 
> This tells me that the camera simply does not support the I420 pixel
> format - only YUY. And because the 'timeoverlay' always expects an input
> format of 'video/x-raw-yuv,format=I420', I have to put the
> ffmpegcolorspace element after the video capture element.
> 
> I'm going to work a bit more on your command to see if I can get the
> 'tee' element working properly.
> 
> -jason
> 
> 
> 
> 
> Edgard Lima wrote:
>> Hi,
>>
>> nice your blog
>>
>> http://www.buberel.org/serendipity/
>>
>> and nice
>>
>> http://gentrans.sourceforge.net/
>>
>> project
>>
>> But I think, unfortunately, it doesn't help for something more robust
>> and professional.
>>
>> In case of your pipeline:
>>
>> gst-launch-0.10  v4l2src queue-size=16 !  stamp sync-margin=1
>> sync-interval=1
>>     ! video/x-raw-yuv,width=800,height=600,framerate=15/1
>>     ! queue2 max-size-buffers=1000 max-size-bytes=0 max-size-time=0  !
>> ffmpegcolorspace
>>     ! theoraenc quality=60 name=venc  alsasrc device="hw:1,0"
>>     !  audio/x-raw-int,rate=16000,channels=1,depth=16
>>     !  audioconvert ! queue2 max-size-buffers=1000 max-size-bytes=0
>> max-size-time=0
>>     !  vorbisenc quality=0.9 name=aenc oggmux name=mux ! filesink
>> location=test.ogg aenc. ! mux. venc. ! mux
>>
>> Considering you have 100% CPU usage, what happens is that you are
>> delaying the problem a bit by using a large queue (1000) and again
>> delaying the problem a bit with "stamp" element
>>
>> Please try with the following pipeline:
>>
>> gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv,
>> format=\(fourcc\)I420, width=640, height=480 ! tee name=t ! queue2 !
>> timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg
>> alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 !
>> videoscale ! xvimagesink sync=false
>>
>> Then try change the resolution to the one your camera accept
>> (width=800,height=600,framerate=15/1) and see if you can remove
>> ffmpegcolorspace and videoscale
>>
>> if you don't have 100% you will have best result.
>>
>> if you have 100% then try to increase the queue size (or also use stamp)
>> so it would be ok to make short videos (as you already did).
>>
>> - - - - - - - - - - - - - - - - - --
>>
>> Now I would like discuss what would be the best way to record audio and
>> video in 100% cpu case.
>>
>> if I use
>>
>> gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv,
>> format=\(fourcc\)I420, width=1280, height=960 ! tee name=t ! queue2 !
>> timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg
>> alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 !
>> videoscale ! xvimagesink sync=false
>>
>> then my file isn't recorded properly (Can't record audio fast
>> enough...this discontinuity breaks the recording)
>>
>> BUT
>>
>> gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv,
>> format=\(fourcc\)I420, width=1280, height=960 ! tee name=t ! queue2 !
>> timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg
>> alsasrc ! queue2 ! audioconvert ! vorbisenc ! oggmux ! filesink
>> location=2.ogg  mux. t. ! queue leaky=2 ! videoscale ! xvimagesink
>> sync=false
>>
>> Both audio and video file are recorded fine (individually). In addition,
>> the audio file has duration greater than video file. I think it happens
>> because of video dropping.
>>
>> But I think it is close to the best thing that could be done. So what I
>> would like to do is to give priority to audio as much as possible, and
>> drop videos, but still having audio and video sync. So, How to do it?
>>
>> Best Regards,
>> Edgard
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>   
>