synchronize two rtsp streams from two sources

[Pietro Bonfa']

Hi everybody,

I'm trying to synchronize two rtsp streams from two sources (2 RPis)
connected with a 100Mbps Ethernet link.

I'm using test-netclock.c with a command line like

./test-netclock "( rpicamsrc name=src preview=false fullscreen=0
inline-headers=true bitrate=3000000 drc=off shutter-speed=12000
contrast=10 brightness=50 saturation=10 ! video/x-h264,width=320,
height=240, framerate=25/1 ! h264parse ! rtph264pay name=pay0 pt=96 )"

on the two RPis and this code for the client:

https://gist.github.com/anonymous/3237e4e78ae4e3f923e9

(it is a mashup of test-netclock-client.c and the opencv gstreamer input
class)

If the client is faster than the streaming frame rate (CASE 1),
everything works fine for the first ~10/20 minutes. Then the video has
some hiccups and the presentation time of the various buffers is always
the same.
With respect to the source code already mentioned, at the beginning logs
report:

Frames 13380 sync: -0.904128 ms start_time -0.212636 s FPS 1: ~40.053404
FPS 2: ~40.035587 FPS prog: ~39.930779
Frames 13381 sync: -0.904128 ms start_time -0.212636 s FPS 1: ~inf FPS
2: ~inf FPS prog: ~39.933287
Frames 13382 sync: -0.926350 ms start_time -0.212636 s FPS 1: ~20.013342
FPS 2: ~20.022247 FPS prog: ~39.933751

and after some time:

Frames 168970 sync: 2.245400 ms start_time -0.212636 s FPS 1: ~inf FPS
2: ~inf FPS prog: ~39.996505
Frames 168971 sync: 2.245400 ms start_time -0.212636 s FPS 1: ~inf FPS
2: ~inf FPS prog: ~39.996590
Frames 168972 sync: 2.245400 ms start_time -0.212636 s FPS 1: ~inf FPS
2: ~inf FPS prog: ~39.996663
Frames 168973 sync: 2.245400 ms start_time -0.212636 s FPS 1: ~inf FPS
2: ~inf FPS prog: ~39.996736

where FPS 1 and FPS 2 are 1/(buffer->pts - previous buffer->pts) and FPS
prog is the one over the time it takes to complete the while loop of the
acquisition. When all the "inf" appear, synchronization is usually lost
(worse than 50 ms) or it tends to worsen.

If the client is slower than the sender (it cannot handle the framerate,
CASE 2) synchronization is fine for the first ~5000 frames and then it
goes bananas.

(Complete log files are here:
http://www.fis.unipr.it/~pietro.bonfa/logfilessync.tar.bz2)

So my questions are:

1) can someone please help understand the source of the problem of CASE
1 where  presentation timestamps remain fixed and video has hiccups? Is
there something terribly wrong with my pipelines?
2) why is synchronization fine at the beginning and then it is suddenly
lost in CASE 2?
3) Is it there a better way to achieve syncronization between the two
streams? I read about gst_rtp_bin_associate on the internet but I could
not find any documentation.

Thanks,
Pietro