<html><head></head><body><div class="ydp745f70e7yahoo-style-wrap" style="font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:13px;"><div></div> <div dir="ltr" data-setdir="false">Thank you so much for your detailed mail. Very nice!</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false"> Joe<br></div><div><br></div> </div><div id="yahoo_quoted_0587936476" class="yahoo_quoted"> <div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;"> <div> On Thursday, November 16, 2023, 01:00:44 p.m. EST, cfd new via gstreamer-devel <gstreamer-devel@lists.freedesktop.org> wrote: </div> <div><br></div> <div><br></div> <div><div id="yiv6872810570"><div><div style="font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:13px;" class="yiv6872810570ydp9879d28cyahoo-style-wrap"><div></div> <div dir="ltr">Hi, Philipp, <br clear="none"></div><div dir="ltr"><br clear="none"></div><div dir="ltr"> very interested in your code. I got the following message when I stream a live video to youtube. Maybe it is related to your case.</div><div dir="ltr"><br clear="none"></div><div dir="ltr"> Joe<br clear="none"></div><div dir="ltr"><div><pre class="yiv6872810570ydp7b9d05d4codeblock-buttons"><code class="yiv6872810570ydp7b9d05d4hljs yiv6872810570ydp7b9d05d4language-ruby"><span class="yiv6872810570ydp7b9d05d4hljs-number">0</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">03</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">35.140304346</span> <span class="yiv6872810570ydp7b9d05d4hljs-number">21323</span> <span class="yiv6872810570ydp7b9d05d4hljs-number">0x7f1fa004c240</span> <span class="yiv6872810570ydp7b9d05d4hljs-variable yiv6872810570ydp7b9d05d4constant_">WARN</span> audioresample gstaudioresample.<span class="yiv6872810570ydp7b9d05d4hljs-symbol">c:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">732</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:gst_audio_resample_check_discont</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:<audioresample1></span> encountered timestamp discontinuity of <span class="yiv6872810570ydp7b9d05d4hljs-number">639</span> samples = <span class="yiv6872810570ydp7b9d05d4hljs-number">0</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">00</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">00</span>.039937500 <span class="yiv6872810570ydp7b9d05d4hljs-number">0</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">03</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">35.141001930</span> <span class="yiv6872810570ydp7b9d05d4hljs-number">21323</span> <span class="yiv6872810570ydp7b9d05d4hljs-number">0x7f1fa004c240</span> <span class="yiv6872810570ydp7b9d05d4hljs-variable yiv6872810570ydp7b9d05d4constant_">WARN</span> audioresample gstaudioresample.<span class="yiv6872810570ydp7b9d05d4hljs-symbol">c:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">732</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:gst_audio_resample_check_discont</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:<audioresample1></span> encountered timestamp discontinuity of <span class="yiv6872810570ydp7b9d05d4hljs-number">639</span> samples = <span class="yiv6872810570ydp7b9d05d4hljs-number">0</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">00</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">00</span>.039937500 <span class="yiv6872810570ydp7b9d05d4hljs-number">0</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">03</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">35.141314636</span> <span class="yiv6872810570ydp7b9d05d4hljs-number">21323</span> <span class="yiv6872810570ydp7b9d05d4hljs-number">0x7f1fa004c240</span> <span class="yiv6872810570ydp7b9d05d4hljs-variable yiv6872810570ydp7b9d05d4constant_">WARN</span> audioresample gstaudioresample.<span class="yiv6872810570ydp7b9d05d4hljs-symbol">c:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">732</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:gst_audio_resample_check_discont</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:<audioresample1></span> encountered timestamp discontinuity of <span class="yiv6872810570ydp7b9d05d4hljs-number">639</span> samples = <span class="yiv6872810570ydp7b9d05d4hljs-number">0</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">00</span><span class="yiv6872810570ydp7b9d05d4hljs-symbol">:</span><span class="yiv6872810570ydp7b9d05d4hljs-number">00</span>.039937500 </code></pre></div><div><br clear="none"></div></div> </div><div id="yiv6872810570yqt75092" class="yiv6872810570yqt3524390563"><div id="yiv6872810570yahoo_quoted_1151053006" class="yiv6872810570yahoo_quoted"> <div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;"> <div> On Thursday, November 16, 2023, 06:35:55 a.m. EST, Philipp B via gstreamer-devel <gstreamer-devel@lists.freedesktop.org> wrote: </div> <div><br clear="none"></div> <div><br clear="none"></div> <div><div dir="ltr">Hi,<br clear="none"><br clear="none">[dislaimer: I didnt work with gstreamer for some time, my terminology<br clear="none">might be a bit rusty]<br clear="none"><br clear="none">I also had audio issues when streaming video files over webrtc, and<br clear="none">judging by your video, it could be the same issue you face.<br clear="none"><br clear="none">For me it turned out to be small rounding errors in the audio frame<br clear="none">timestamps. Due to the realtime nature, browsers playing webrtc audio<br clear="none">are not aiming for a perfect smooth playback, but rather for perfect<br clear="none">realtime. Small deviations in timestamps instantly lead to the browser<br clear="none">inserting small pieces of silence, or dropping a few ms of audio.<br clear="none"><br clear="none">The most obvious effect was the volume being way lower than it should<br clear="none">be. This was caused by the player/browser constantly fading segments<br clear="none">in and out, to overlap them with the next one, or with silence. Also,<br clear="none">I had similar distortion artifacts than you have.<br clear="none"><br clear="none">To be clear, my issue seemed to be related to transmission of<br clear="none">non-live, pre-encoded video only. WebRTC is made for live<br clear="none">transmissions, where audio frame timestamps are directly related to<br clear="none">the wall clock time at which a segment has been recorded. In contrast,<br clear="none">a non-live video file typically has a stream of audio segments, which<br clear="none">are expected to be played back without gaps, even when there are small<br clear="none">gaps according to the time stamps.<br clear="none"><br clear="none">What happens if you increase the audio frame size? In my case, this<br clear="none">was improving the situation dramatically, but not fully fixing it.<br clear="none"><br clear="none">In the end, I got this solved by "smoothing" audio time stamps. I<br clear="none">wrote my own gstreamer element, that kept an estimation of the next<br clear="none">frames timestamp, based on the last frame. In case, there is a small<br clear="none">deviation (e.g. below 5 ms), the frames timestamp will be changed to<br clear="none">that of the estimation. I can provide the code of that element in case<br clear="none">your interested.<br clear="none"><br clear="none">Philipp<br clear="none"><br clear="none">Am Mi., 15. Nov. 2023 um 23:39 Uhr schrieb Vincent Deconinck via<br clear="none">gstreamer-devel <<a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:gstreamer-devel@lists.freedesktop.org" target="_blank" href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a>>:<div id="yiv6872810570yqtfd11681" class="yiv6872810570yqt1701785150"><br clear="none">><br clear="none">> Hi,<br clear="none">><br clear="none">> I made some progress in my quest to stream video files to a browser using webrtcssink, but audio is not OK. I built the following pipeline:<br clear="none">> +-----------------------------------------------------------------------------+<br clear="none">> | uridecodebin.audio ->- audioconvert ->- audioresample ->- webrtcssink.audio |<br clear="none">> | uridecodebin.video ----->----- videoconvert ------>------ webrtcssink.video |<br clear="none">> +-----------------------------------------------------------------------------+<br clear="none">><br clear="none">> The webrtcsink pads are statically linked, and the uridecodebin pads are dynamically linked upon "pad_added" signal (as in <a rel="nofollow noopener noreferrer" shape="rect" target="_blank" href="https://gstreamer.freedesktop.org/documentation/tutorials/basic/dynamic-pipelines.html">https://gstreamer.freedesktop.org/documentation/tutorials/basic/dynamic-pipelines.html</a> ).<br clear="none">><br clear="none">> Problem: The video looks OK in the browser, but the audio is choppy. A quick and dirty (sorry) video tells a thousand words: <a rel="nofollow noopener noreferrer" shape="rect" target="_blank" href="https://youtu.be/1EbPHLZvVLc?t=27">https://youtu.be/1EbPHLZvVLc?t=27</a><br clear="none">><br clear="none">> (using signalling server, JS API and gstreamer built 2 days ago from git HEAD)<br clear="none">><br clear="none">> I was first using the http URI to the source video but I also tried using a local copy of the file, to no avail.<br clear="none">><br clear="none">> Any idea what could be wrong ? Or an idea how I could track down that issue ?<br clear="none">><br clear="none">> Kind regards,<br clear="none">><br clear="none">> Vincent<br clear="none"></div></div></div> </div> </div></div></div></div></div> </div> </div></body></html>