Dropping Audio Samples, Downstream Can't Keep Up

Mon Jan 16 08:52:05 UTC 2017

Maybe the video frame timestamps are not being timestamped correctly, or
you need to wait until the first frame is available. You can try changing
the behavior of videorate to start from the first received frame using
skip-to-first, but the default behavior should be doing what you need for
the pipeline to work.

Another minor thing is since the camera has a variant rate you can try
setting it before the videorate element to explicitly state that the rate
is variant. I don't think it will have much of a difference but it is worth
taking a shot. you can do that this way: "ksvideosrc device-index=0 !
image/jpeg,framerate=0/1 ! videorate !
image/jpeg,width=1024,height=768,framerate=30/1".
Again, it's just a wild guess.

Also if you haven't done this yet, try debugging the demuxer using the
"--gst-debug=flvmux:5" option on gst-launch. The last thing I can think of
is that in some cases encoders/muxers need a certain amount of frames to
start producing results. Try increasing the size of your queues in the
audio pipeline to 5-10 seconds so that the video pipe has enough time to
produce enough data.

If none of those work then I am kinda stumped. Maybe someone else can give
a bit of insight on what we may be missing.

On Sat, Jan 14, 2017 at 10:22 AM, Brendan Lockhart <somedude114 at gmail.com>
wrote:

> Thanks for the reply,
>
> >The videorate element doesn't change the rate at which the hardware is
> capturing, so my guess is that it has something to do with that.
>
> I know that it wouldn't change the hardware capture rate, but wouldn't it
> duplicate frames as needed to make the framerate constant? Does it not
> correctly set the timestamps?
>
> >One thing you may want to test is adding an audio rate in the pipeline
> with the videorate.
>
> I tried that, along with a few permutations of audioresample &
> audioconvert. Unfortunately that never fixed the issue.
>
>  >Another would be to check what the actual framerate with the higher
> exposure is and setting the pipeline with that framerate rather than 30.
>
> That's a good idea, but it wouldn't work in the case that the camera is on
> autoexposure, as the framerate varies with lighting.
>
> >The muxer may be expecting frames at the target rate but the source
> can't report that it can capture at a lower rate, leading to the audio
> pipeline being stalled.
>
> I agree that that's the most likely cause, but I still don't fully
> understand why videorate doesn't fix that.
>
> On Fri, Jan 13, 2017 at 12:54 AM, Dimitrios Katsaros <
> patcherwork at gmail.com> wrote:
>
>> The videorate element doesn't change the rate at which the hardware is
>> capturing, so my guess is that it has something to do with that. It is
>> possible that the muxer was being stalled which in turn would fill the
>> audio pipeline and would eventually lead to sample loss on a live source.
>> However I am not experienced with the windows elements so I can only
>> speculate. One thing you may want to test is adding an audio rate in the
>> pipeline with the videorate. Another would be to check what the actual
>> framerate with the higher exposure is and setting the pipeline with that
>> framerate rather than 30. The muxer may be expecting frames at the target
>> rate but the source can't report that it can capture at a lower rate,
>> leading to the audio piepline being stalled.
>>
>> Dimitrios
>>
>> On Fri, Jan 13, 2017 at 9:14 AM, Brendan Lockhart <somedude114 at gmail.com>
>> wrote:
>>
>>> Turns out what was happening is that the exposure on the webcam was too
>>> high. Lowering the exposure increased the framerate and fixed the warning.
>>> That's strange, because it was happening even if I put a videorate
>>> element after the ksvideosrc.
>>>
>>> Is this because ksvideosrc was producing bad timestamps? Is there a way
>>> that I can check that hypothesis?
>>>
>>> On Thu, Jan 12, 2017 at 2:29 AM, Brendan Lockhart <somedude114 at gmail.com
>>> > wrote:
>>>
>>>> Hello all,
>>>>
>>>> I'm having an issue where when streaming video/audio with the pipeline:
>>>>
>>>> gst-launch-1.0 ksvideosrc device-index=0 !
>>>>> image/jpeg,width=1024,height=768,framerate=30/1 ! queue ! jpegparse !
>>>>> jpegdec ! queue ! x264enc tune=zerolatency speed-preset=veryfast
>>>>> bitrate=4096 ! h264parse ! video/x-h264, profile=high ! queue ! flvmux
>>>>> streamable=true name=mux ! queue ! rtmpsink location=rtmp://<my-url>
>>>>> autoaudiosrc ! audio/x-raw ! queue ! audioconvert ! avenc_aac ! aacparse !
>>>>> queue ! mux.
>>>>
>>>>
>>>> I eventually start generating errors:
>>>>
>>>>   gst_audio_base_src_create (): /GstPipeline:pipeline0/GstAuto
>>>>> AudioSrc:autoaudiosrc0/GstDirectSoundSrc:autoaudiosrc0-actua
>>>>> l-src-directsoun:
>>>>> Dropped 1420020 samples. This is most likely because downstream can't
>>>>> keep up and is consuming samples too slowly.
>>>>
>>>>
>>>>  With this pipeline, my PC is at ~8% CPU utilization so I don't think
>>>> it's because my PC isn't powerful enough.
>>>>
>>>> I know this error has been brought up in the past, but unfortunately
>>>> none of the proposed solutions seem to be fixing this problem. Does anyone
>>>> have any insight as to what might be causing this error?
>>>>
>>>
>>>
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>> gstreamer-devel at lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>
>>>
>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>>
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20170116/42b418db/attachment.html>