Synchronised RTP Client

Sun May 29 13:07:17 UTC 2016

Hi All,

Sebastian - thank you so much for pointing me in the right direction with
this earlier. It was very helpful in allowing me to progress with this and
I have finally had a chance to put some code together.

I have written an RTSP server in Python trying to re-implement the code
that was in test-netclock.c / test-netclock-client.c, however I have a few
problems with synchronisation. I must be close.

My server is configured with a pipeline as follows:

My launch string for the factory on the server is as follows:

udpsrc uri=udp://127.0.0.1:6001 caps="application/x-rtp,
media=(string)audio, clock-rate=(int)44100, channels=(int)2,
format=(string)S16LE" ! rtpL16depay ! audioconvert ! audioresample !
opusenc bitrate=96000 inband-fec=1 ! rtpopuspay name=pay0

It captures raw audio through RTP on port 6001 and converts it to OPUS.
which will be sent out to the clients (which should all play out in sync).

For this factory, I'm also setting:

factory.set_shared(True)
factory.set_retransmission_time(5000)
factory.set_clock(clock)
factory.set_profiles(GstRtsp.RTSPProfile.AVPF)
Where clock is a time provider that I want to run from the server and use
that to sync the clients:

clock = Gst.SystemClock.obtain()
clock_provider = GstNet.NetTimeProvider.new(clock, None, 8555)

On my clients, where I have sync=true on my alsaaudiosink, I get about 2
seconds of audio followed by silence and a lot of this:

audiobasesink
gstaudiobasesink.c:1807:gst_audio_base_sink_get_alignment:<autoaudiosink0-actual-sink-alsa>
Unexpected discontinuity in audio timestamps of +1012391:15:11.969041666,
resyncing

audiobasesink
gstaudiobasesink.c:1512:gst_audio_base_sink_skew_slaving:<autoaudiosink0-actual-sink-alsa>
correct clock skew -0:00:00.020016938 < -+0:00:00.020000000

The client comprises the following pipeline (location is the server and
buffer is usually configured as 1000 or 5000)

My client contains the following:

self.pipeline = Gst.parse_launch('rtspsrc location=%s latency=%s
buffer-mode=synced ntp-time-source=clock-time ntp-sync=1 do-rtcp=true !
rtpopusdepay name=pay0 ! opusdec ! audioconvert ! autoaudiosink sync=true
async=false' % (src, latency))

# make a clock slaving to the network
self.clock = GstNet.NetClientClock.new('clock0', clock_ip, clock_port, 0)

# Wait for initial clock sync
self.clock.wait_for_sync(Gst.CLOCK_TIME_NONE)

self.pipeline.use_clock(self.clock)
self.pipeline.set_start_time(Gst.CLOCK_TIME_NONE)
self.pipeline.set_latency(latency * Gst.MSECOND)

self.pipeline.set_state(Gst.State.PLAYING)

# Wait until error or EOS.
self.bus = self.pipeline.get_bus()
self.bus.add_signal_watch()
self.bus.connect('message', self.bus_handler)

The intention for this set up is to:
1. Buffer 1-5 seconds of audio.
2. Within this buffer period, allow for retransmits.
3. Otherwise, keep the clients in sync with the server's clock using the
GST NetClientClock.
This configuration plays audio fine if I change "sync=false" on the audio
sink, but this also breaks synchronisation.

Any pointers as to what I've done wrong? I'm currently running GStreamer
out of git.

Kind Regards,
Jonathan

> Message: 5
> Date: Mon, 25 Apr 2016 09:24:53 +0300
> From: Sebastian Dröge <sebastian at centricular.com>
> To: Discussion of the development of and with GStreamer
>         <gstreamer-devel at lists.freedesktop.org>
> Subject: Re: Synchronised RTP Clients
> Message-ID: <1461565493.2297.107.camel at centricular.com>
> Content-Type: text/plain; charset="utf-8"
>
> On Mo, 2016-04-25 at 13:19 +1000, Jonathan Thorpe wrote:
> >
> > I have clearly chosen a very complex project to get started with
> > GStreamer on, but would be very grateful if someone could assist with
> > helping establish pipelines (Client and Server) that will achieve
> > this.
>
> Take a look at my presentation from last year's GStreamer conference:
>
> https://gstconf.ubicast.tv/videos/synchronised-multi-room-media-playback-and-distributed-live-media-processing-and-mixing-with-gstreamer/
>
> Also these two example applications are basically doing exactly what
> you want, just with the GStreamer netclock instead of PTP. But
> exchanging that is a matter of changing a couple of lines.
>
>
> https://cgit.freedesktop.org/gstreamer/gst-rtsp-server/tree/examples/test-netclock.c
>
> https://cgit.freedesktop.org/gstreamer/gst-rtsp-server/tree/examples/test-netclock-client.c
>
> The examples are using RTSP and gst-rtsp-server, but there's nothing
> RTSP specific in how all this works. It's just for simplicity to make
> it easier to exchange the SDP between sender and clients.
>
> --
> Sebastian Dröge, Centricular Ltd · http://www.centricular.com
>
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: signature.asc
> Type: application/pgp-signature
> Size: 949 bytes
> Desc: This is a digitally signed message part
> URL: <
> https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20160425/31cca1a2/attachment-0001.sig
> >
>
> ------------------------------
>
> Message: 6
> Date: Mon, 25 Apr 2016 09:20:48 +0100
> From: Andy Robinson <andy at seventhstring.com>
> To: gstreamer-devel at lists.freedesktop.org
> Subject: Re: How to query the GstSegment "rate"?
> Message-ID: <571DD360.10109 at seventhstring.com>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> On 25/04/16 07:21, Sebastian Dröge wrote:
> > On Sa, 2016-04-23 at 14:55 +0100, Andy Robinson wrote:
> >> I imagine there must be some way of querying the
> >> pipeline for the rate once it is in paused state, but how?
> >
> > You could try the SEGMENT query, if something in your pipeline is
> > answering it then it will contain the rate.
> >
> > But the bigger question is why you need to query the rate and don't
> > know it already. In the end it was your code that was setting that
> > exact rate via a seek :)
> >
> >
> > Also what's the bigger picture here? Why do you want to convert from
> > stream time to the scaled stream time by rate (which is not exactly the
> > running time in general, but maybe you actually want the running
> > time?)?
> >
>
> Even if I don't do any seek, a Segment event goes down the pipeline and
> it has a rate of 0.5 - I know this from putting diagnostics in the
> pipeline.
>
> The file is here if you are interested:
> http://www.seventhstring.com/other2/JAttendraiShort50.mov
> It plays ok in Parole Media Player on Linux, or QuickTime on Mac. It's
> 60 secs long and was produced, using QuickTime, by slowing down a 30 sec
> clip to half speed.
>
> When I play it in my app, I find that when I want to seek using
> gst_element_seek_simple within this video I must use the pre-slowdown
> times, e.g. if I want to seek to the 40th second of the 60 second video,
> I must actually seek to a time of 20 secs, and also must make a similar
> adjustment to the values returned by gst_element_query_position.
>
> It seems to me that I need to get that 0.5 rate number and use it as a
> multiplier when I call gst_element_seek_simple but if there is a better
> way, please advise me.
>
> Of course, maybe the file is simply erroneous, illegal, though it does
> play ok in some (not all) players.
>
> Regards,
> Andy Robinson, Seventh String Software, www.seventhstring.com
>
>
> ------------------------------
>
> Message: 7
> Date: Mon, 25 Apr 2016 08:28:23 +0200
> From: marc dingemans <marcdingemans62 at gmail.com>
> To: gstreamer-devel at lists.freedesktop.org
> Subject: muliple gst_element_seek for creating new file
> Message-ID:
>         <CAN2V=
> fCEOhsBCx8YPfWdiL5OXcdYHQEcqP1cED0vO_V49SdC9A at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi
>
> I want to extract some video parts from a mkv file and create based on
> these parts one new mkv file.
>
> This is done with muliple gst_element_seek calls with a time frame.
> When a eos is received, then a the next gst_element_seek is started until
> done.
> The destination file is generated, but can not be played with VLC.
> The seek is done executed on element "matroskademux" (also used pipeline)
> with GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_SNAP_BEFORE properties
>
> Used pipeline :
> filesrc| matroskademux| h264parse | matroskamux | filesink
>
>
> When I use next pipeline, then everything works fine.
> filesrc| matroskademux| h264parse | avdec_h264| autovideosink
>
>
> Why does the seek seems work to gui and not to file.
> Must I use other approaches to put parts from a file in a new one
>
> Thanks for your help
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20160425/407be008/attachment.html
> >
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
> ------------------------------
>
> End of gstreamer-devel Digest, Vol 63, Issue 76
> ***********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20160529/16986a00/attachment.html>