<div dir="ltr">Hi Nirbheek,<div><br></div><div>That's good to know that opus doesn't have timestamps, why then would adding the latency to the appsrc make a difference? The incoming OPUS data is in format 16000 Hz sample rate, 1 frame per packet, 60 ms frame size</div><div><br></div><div>Also good to know that I need to push data continuously, I will work on that. What is the recommended way to push silence?</div><div><br></div><div>I am happy to share the code, it's just really basic so I wasn't sure it was necessary.</div><div><br></div><div><b>This is the current pipeline:</b></div><div><br></div><div>appsrc min-latency=4000000000 is-live=true do-timestamp=true name=src ! queue2 ! opusparse ! opusdec ! audioconvert ! audioresample ! queue2 ! ! pulsesink volume=2<br></div><div><br></div><div>I've been messing around with different min-latencies, queue types, and timestamp properties</div><div><br></div><div><b>Here is the Go code that feeds the audio coming in over WebRTC</b></div><div><br></div><div>sinkPipelineStr := fmt.Sprintf("%s", viper.GetString("sinkPipeline"))<br><br> sinkPipeline := gst.CreatePipeline(sinkPipelineStr, "src", "", nil)<br><br> m.log.Infof("Sink pipeline: %s", sinkPipelineStr)<br><br> sinkPipeline.Start()<br> defer sinkPipeline.Stop()<br><br> localAudioSrc := make(chan []byte)<br><br> fmt.Println("Waiting for audio...")<br> for {<br> select {<br> case data := <-m.audioChans.Out:<br><br> fmt.Println("audio OUT")<br> // fmt.Println(data)<br> sinkPipeline.Push(data)<br> //f.Write(data)<br> case <-ctx.Done():<br> return<br> }<br> }<br></div><div><br></div><div><b>Here is the code on the CreatePipeline function:</b></div><div><br></div><div>func CreatePipeline(pipelineStr string, srcName string, sinkName string, sink chan []byte) *Pipeline {<br><br> pipelineStrUnsafe := C.CString(pipelineStr)<br> defer C.free(unsafe.Pointer(pipelineStrUnsafe))<br><br> pipelinesLock.Lock()<br> defer pipelinesLock.Unlock()<br><br> pipeline := &Pipeline{<br> id: len(pipelines),<br> pipeline: C.gstreamer_create_pipeline(pipelineStrUnsafe),<br> sink: sink,<br> srcName: srcName,<br> sinkName: sinkName,<br> }<br><br> pipelines[<a href="http://pipeline.id">pipeline.id</a>] = pipeline<br><br> return pipeline<br>}<br></div><div><br></div><div><b>Here is the code for the start function:</b></div><div><br></div><div>func (p *Pipeline) Start() {<br> C.gstreamer_start_pipeline(p.pipeline, C.int(<a href="http://p.id">p.id</a>), C.CString(p.srcName), C.CString(p.sinkName))<br>}<br></div><div><br></div><div><b>Here is the push function:</b></div><div><br></div><div>func (p *Pipeline) Push(buffer []byte) {<br> b := C.CBytes(buffer)<br> defer C.free(b)<br> C.gstreamer_receive_push_buffer(p.pipeline, b, C.int(len(buffer)), C.CString(p.srcName))<br>}<br></div><div><br></div><div><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">All the best,<br>Kyle Gibbons<br><br></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Nov 25, 2021 at 7:15 AM Nirbheek Chauhan <<a href="mailto:nirbheek.chauhan@gmail.com">nirbheek.chauhan@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Kyle,<br>
<br>
Opus frames do not contain timestamps. It sounds like you're not<br>
pushing buffers correctly or maybe the opus frames are being<br>
misdetected (wrong sample rate / channels, maybe). You haven't shared<br>
your code so we can only guess. You definitely need to push data<br>
continuously, though, since this is a live pipeline.<br>
<br>
I recommend using the "need-data" signal to know when to push data<br>
into the pipeline, and if you do not have data ready to push, the<br>
simplest thing would be to push an opus frame containing silence.<br>
<br>
There's other things you can do, like using audiomixer to ensure that<br>
pulsesink gets a continuous stream, etc.<br>
<br>
Cheers,<br>
Nirbheek<br>
<br>
On Wed, Nov 24, 2021 at 8:15 PM Kyle Gibbons via gstreamer-devel<br>
<<a href="mailto:gstreamer-devel@lists.freedesktop.org" target="_blank">gstreamer-devel@lists.freedesktop.org</a>> wrote:<br>
><br>
> I am finally making some progress! I set the min-latency to 8000000000 which obviously causes a huge delay, but does allow audio to play. When I stop sending audio I get a "Got Underflow" error from pulsesink and then audio does not play again until I restart the application. Also, the audio does not sound great. It's almost like it's playing under speed, sounds a bit lower than expected. I have to set the volume to at least 2 to be able to hear the audio well.<br>
><br>
> Is there a way to compensate for the timestamps coming in from the source without introducing a large delay? I am guessing that since I am basically just passing the opus from Zello through my application that the origins opus timestamp is being used, which of course would be well past when my app starts playing.<br>
><br>
> All the best,<br>
> Kyle Gibbons<br>
><br>
><br>
><br>
> On Wed, Nov 24, 2021 at 8:02 AM Kyle Gibbons <<a href="mailto:kyle@kylegibbons.com" target="_blank">kyle@kylegibbons.com</a>> wrote:<br>
>><br>
>> I wanted to add that when there is data coming in the samples and buffers should be consistent, but because the ultimate source is a walkie-talkie like interface, there is not always audio coming in. We only send data to gstreamer when there is audio coming into the system over the network, we do not send silence. I did try starting the stream before the application so there was essentially always audio flowing in, but that made no difference.<br>
>><br>
>> All the best,<br>
>> Kyle Gibbons<br>
>><br>
>><br>
>><br>
>> On Wed, Nov 24, 2021 at 7:00 AM Kyle Gibbons <<a href="mailto:kyle@kylegibbons.com" target="_blank">kyle@kylegibbons.com</a>> wrote:<br>
>>><br>
>>> Tim,<br>
>>><br>
>>> Thanks for the reply. I tried adding min-latency of 40000000, 60000000, 100000000, and 1000000000 to no avail.<br>
>>><br>
>>> The buffers and number of samples should be consistent. The audio comes from another service I wrote using Go and Pion which gets its audio from the Zello API (<a href="http://zello.com" rel="noreferrer" target="_blank">zello.com</a>)<br>
>>><br>
>>> All the best,<br>
>>> Kyle Gibbons<br>
>>><br>
>>><br>
>>><br>
>>> On Wed, Nov 24, 2021 at 6:48 AM Tim-Philipp Müller via gstreamer-devel <<a href="mailto:gstreamer-devel@lists.freedesktop.org" target="_blank">gstreamer-devel@lists.freedesktop.org</a>> wrote:<br>
>>>><br>
>>>> Hi Kyle,<br>
>>>><br>
>>>> > But this doesn't:<br>
>>>> ><br>
>>>> > appsrc is-live=true do-timestamp=true name=src ! queue ! opusparse !<br>
>>>> > opusdec ! audioconvert ! audioresample! queue ! pulsesink<br>
>>>><br>
>>>><br>
>>>> Try adding appsrc min-latency=40000000 (=40ms in nanoseconds) or such.<br>
>>>><br>
>>>> You might have to experiment with the values.<br>
>>>><br>
>>>> Do you always push in buffers of the same size / number of samples?<br>
>>>> Where do you get the audio data from?<br>
>>>><br>
>>>> Cheers<br>
>>>> Tim<br>
</blockquote></div>