<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 2012-08-31 11:13, Elis POPESCU
wrote:<br>
</div>
<blockquote
cite="mid:1346404436.80947.YahooMailNeo@web112005.mail.gq1.yahoo.com"
type="cite">
<div style="color:#000; background-color:#fff; font-family:tahoma,
new york, times, serif;font-size:10pt">
<div>
<div><font face="tahoma, new york, times, serif" size="2">Hi,</font></div>
<div><font face="tahoma, new york, times, serif" size="2"><br>
</font></div>
<div><font face="tahoma, new york, times, serif" size="2">I'm
trying to get multiple audio clients synchronized and I'm
using GstNetClientClock (on the audio clients as the pipe
clock) connected to a GstNetTimeProvider (on the server,
based on sys clock) using Gstreamer 0.10.36. I'm using
RTP/RTCP protocol over UDP multicast (from the server to
the clients) and I'm also setting "ntp-sync" and
"use-pipeline-clock" to true on gstrtpbin on server and
clients. However if the network is loaded then the clients
go out of sync. </font></div>
<div><font face="tahoma, new york, times, serif" size="2"><br>
</font></div>
<div><font face="tahoma, new york, times, serif" size="2">Do I
miss something? Any ideas? Can it be a bug?</font></div>
<div><font face="tahoma, new york, times, serif" size="2"><br>
</font></div>
<div><font face="tahoma, new york, times, serif" size="2">Are
there fixes or big changes in the way RTP/RTCP
synchronization works in the new version of Gstreamer
(1.0) that is coming soon?</font></div>
<div><font face="tahoma, new york, times, serif" size="2"><br>
</font></div>
<div><font face="tahoma, new york, times, serif" size="2">Thanks
!</font></div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
gstreamer-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a>
<a class="moz-txt-link-freetext" href="http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel">http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a>
</pre>
</blockquote>
What you describe is a use case that is unfortunately not covered by
the docs.<br>
The docs write about inter-stream synchronization, for example for
A/V sync, and lip sync.<br>
What is does NOT describe is what you are doing: one sender, N
receivers, the receivers shall play synchronously, that is, the
phase difference between the receivers shall be below a certain
amount of ms.<br>
<br>
I have been developing similar, and ran into this issue. Once you
keep in mind that "stream sync" means "inter-stream sync" in the
docs, things become much clearer.<br>
Here is what I did to get synchronization to work:<br>
<br>
<br>
- use the net client clock, just like you do.<br>
<br>
- buffer-mode set to none : this is perhaps the most important one.
The default setting performs clock slaving, which is actually not
what you want here. You already synchronize the pipeline clock, all
you want is<br>
for the audio buffer timestamps on the receiver side to be corrected
using the RTP timestamp and RTCP SR information. Setting 0 (= none)
gives you just that. Some additional notes on this follow later.<br>
<br>
- jitter buffer size must be at least as large as the maximum
expected network delay. If you expect your packets to be up to 500
ms late, and you use a 200ms jitter buffer, you will run into
trouble.<br>
<br>
- Do not forget about the audio sink. It is what actually schedules
audio buffers to be played based on the timestmap. This is one big
difference between sinks in GStreamer and sinks in many other
libraries and programs: most sinks are not just simple output
devices that emit data as soon as you push it into them, they pay
attention to buffer timestamps. With this in mind, I recommend to
use GST_BASE_AUDIO_SINK_SLAVE_SKEW as slave-method for the sink
(which is the default value anyway), and give the drift-tolerance
property a close look. Its default value is 40 ms. Once the rtpbin
corrects buffer timestamps based on the RTP timestamps and the RTCP
SR packets, and the sink then detects a resulting drift between the
internal audio clock and these timestamps that is greater than the
drift-tolerance value, it will skew the playout pointer,
compensating for the drift. The smaller the drift-tolerance value,
the more likely the skewing. If for example you try to use 0.5 ms
for the drift-tolerance, skewing will happen often, and playback
will sound very bad. Pick a sensible middle ground between
synchronized playback and stable playback.<br>
<br>
<br>
Some more notes about all of this:<br>
<br>
- buffer-mode: initially, I used mode 1 (clock slaving). And in the
beginning, synchronization was good. But then I noticed that after
network disturbances (sudden delays, packet losses etc.), the rtpbin
did not compensate (even though the phase difference between the
receivers was over 100 ms). Once the synchronization got lost, it
never recovered. I now believe that the clock-slaving mode is not
supposed to do that in the first place. It concerns itself primarily
with replicating the sender clock to allow for smooth playback.
Transmission delays are only a concern when it affects the
smoothness of the playback. In other words, as long as playback is
smooth, everything else is unimportant in this mode.<br>
<br>
- The audio sink slave method "skew" is the recommended one. Others
are to ignore drifts and to use a resampler. The resample method
produces REALLY bad results and is not recommended, for good
reasons. However, skewing itself is not optimal, because it is
equivalent to throwing away samples or filling in null samples. I
have plotted the output of the receivers on an oscilloscope, and
this sudden skewing is clearly visible. Such a behavior is however
very bad, because hard cuts like that introduce very high frequency
artifacts. What other systems do to solve this is to either mute
audio for a few ms, then unmute, masking the cut, or more advanced
(and complicated) techniques with pattern matching, trying to
recognize common areas and periods etc. I think the mute idea could
be added to the sink easily, and would help greatly.<br>
<br>
- A signal in the rtpbin that informs me that the rtpbin is now in
sync, since it received the RTCP SR information, would be a good
addition. It would allow me to mute the sound in the beginning,
until it gets synchronized (since until the rtpbin receives the SR
packet, playback happens on a best-effort basis, without any
synchronization).<br>
<br>
<br>
Now, I could be wrong about many of these things, but unfortunately,
docs are very sparse there, and I obtained this information by
digging through the rtp bin/jitterbuffer/session code and the base
sink code.<br>
What would be great is for one of the core developers to comment on
my points and refer to whats right and what is inaccurate or wrong.
Please? Pretty please? :)<br>
</body>
</html>