RTP/RTCP Support and Extension Capabilities of GStreamer
Mario Montagud Climent
mamontor at upvnet.upv.es
Mon Jan 9 03:24:50 PST 2012
Dear Gstreamer developers,
My name is Mario Montagud. I am a PhD student at Politechnic
University of Valencia (UPV) in Spain.
My PhD work is about Multimedia Communications and Synchronization
Techniques. Concretely, our research group is designing an
Inter-Destination Multimedia Synchronization (IDMS) proposal that aims
to synchronize the playout of media streams across separated clients
(which can be placed in different networks, with different network
latency). For that purpose, we take advantage of the feedback and
extension capabilities of RTP/RTCP protocols.
In the initial stages of my PhD, I have implemented this IDMS approach
in a network simulator (NS-2), and I?ve added some enhancements to
that solution (reference [Bor11]).
Our goal is to synchronize a RTP network stream across multiple
geographically separated clients. On the one hand, a new RTCP report
block (RTCP XR for IDMS block) has been specified to inform the source
(or a sync manager) about receipt and playout times for specific RTP
media packets (for that purpose all the participants must share a
common available clock source, e.g. NTP, GPS, or other solutions ?) at
the receiver side (in a compound RTCP packet). On the other hand a new
RTCP packet type (RTCP IDMS packet) has been defined to send playout
setting instructions to the sync clients (if an out of sync situation
is detected by the sync manager). See [Bra11].
My next objective is to implement and test that IDMS proposal in a
real media framework/player. So, I?m searching for the most
appropriated solution for it.
I am a newbie in GStreamer. For that reason, I send this mail asking
for your recommendation. During last days, I?ve reading the Gstreamer
documentation (GStreamer Plugin Writer?s Guide, and GStreamer
Application Development Manual) to get a better understanding of this
I?ve seen Gstreamer supports RTP/RTCP functionalities
(/gst-plugins-good-plugins-gstrtpbin.html) and also lip-sync mechanisms.
I?ve also found a post from Thomas Roos to the Gstreamer Developers
list about the synchronisation of audio playback (multi room feature)
on multiple receivers. He did not succeed on the configuration of such
multi room sync scenario. However, he told me that Gstreamer is the
right way to do it! ;)
Thus, I would like to ask four your recommendation/suggestions before
I begin with the implementation efforts.
First, In order to implement our IDMS solution, I need to implement
the new RTCP packet types and block reports defined in [Bra11].
In our IDMS solution, we tackle the sync problem by dividing it in two
1) To ensure that all the receivers initiate the playout of the media
stream at the same time.
For that purpose, I need to implement some mechanism to ensure that
all the clients will buffer the incoming media units (encapsulated in
RTP packets) until a global initial playout instant (this info will be
received in a new RTCP packet type for IDMS from the server) is
reached. Then, the playout process must start.
2) To maintain the playout processes between distributed clients in a
synchronized way during the streaming session lifetime.
During the session, receivers will send a new RTCP XR blocks including
arrival/presentation times for a specific RTP packet (e.g. the last
timestamp received from one RTP packet containing a media unit ? video
frame-). A sync manager (e.g. the server) will gather this reports
from all the clients and, if an out of sync is detected (exceeding an
asynchrony threshold), it will send a new RTCP packet type for IDMS
including playout setting instructions to the clients.
Once the clients receive this packet, they will have to trigger
playout adjustment to get in sync according to the playout info
included in that control message.
I have seen the ?ts-offset? calculation for lip-sync purposes in
?gstrtpbin? module. By the way, how are the required playout
adjustments managed in the lip sync mechanism? (smooth) playback speed
I think the most difficult task will be the management of the playout
processes in each media client to get in sync (e.g. to report on
presentation time for a specific RTP packet ?e.g. the last one
encapsulating a video frame-, and to manage the necessary playback
adjustments. Can I access to the playback buffer to know about
presentation times of a specific video frame and the current playback
I apologize for the extension of the mail, but I will appreciate very
much to get some initial guidelines from Gstreamer experts before
starting with the design and implementation efforts.
Thank you very much in advance.
Regards from Spain,
REFERENCES: (If you are interested, and don?t have access to them,
please, tell me)
[Bor08] F. Boronat, J.C. Guerri, and J. Lloret, ?An RTP/RTCP based
approach for multimedia group and inter-stream synchronization?,
Multimedia Tools and Applications Journal, Vol. 40 (2), 285-319, June
[Bor11] F. Boronat, M. Montagud, and V. Vidal, ?Smooth Control of
Adaptive Media Playout to Acquire IDMS in Cluster-based Applications?,
IEEE LCN 2011, Bonn, October 2011.
[Bra11] R. V. Brandenburg, H. Stokking, M.O. V. Deventer, Boronat, F.
M. Montagud, K. Gross, ?RTCP for Inter-Destination Media
Synchronization?, IETF Internet Draft, October 2011.
More information about the gstreamer-devel