<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Tue, May 14, 2013 at 3:37 AM, Edward Hervey <span dir="ltr"><<a href="mailto:bilboed@gmail.com" target="_blank">bilboed@gmail.com</a>></span> wrote:<br>
<div><br></div><div>Thanks hugely for your response. I very much appreciate the time you took to explain the inner model of the sync process.<br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im"><br>
On Mon, 2013-05-13 at 15:13 -0400, Stirling Westrup wrote:<br>
> This may, or may not, be a GStreamer issue. I've written a video-wall<br>
> application that splits a playing video into pieces and shows the<br>
> pieces on zero-clients connected to the host computer via USB or<br>
> Ethernet cable.<br>
><br>
><br>
> This basic functionality is working, but when playing large (ie 4K<br>
> resolution) videos, we sometimes see momentary differences between<br>
> when different monitors get updated with the next frame of video.<br>
><br>
><br>
> Now, I can easily imagine a number of places that a video frame may be<br>
> delayed so as to show after its neighbor:<br>
><br>
><br>
> 1) Inside the GStreamer pipeline.<br>
><br>
> 2) Inside the network stack<br>
><br>
> 3) due to USB bandwidth issues.<br>
><br>
><br>
> So far the only code in GStreamer that tries to maintain sync is that<br>
> in the queue that handles each output monitor. The assumption made was<br>
> that that would be sufficient. I could (with a lot of rewriting)<br>
> replace all N output queues with a single N-way multiqueue, but I'm<br>
> unsure that such a move is actually necessary.<br>
<br>
</div> That's not quite the reality.<br>
1) The queue's job is to dedicate a thread to the rendering process<br>
(i.e. the thread;s sole taks is to pick the next buffer available from<br>
the queue (if any) and push it to the sink). Switching to multiqueue<br>
won't change anything.<br>
2) The synchronization (i.e. waiting for the render time) is done in<br>
the sink.<br></blockquote><div><br></div><div>Ah. Thanks. I had not really understood that, despite reading both the main manual and the element writing manual.<br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Based on what you mentioned, you have N different sinks for the<br>
various outputs. I'll go under that assumption.<br></blockquote><div><br></div><div>Correct.<br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
The issue is that the delay between the moment GStreamer (and the<br>
GstBaseSink base class) says "yes, this buffer time corresponds to this<br>
clock time, render it at this clock time" and the moment it actually<br>
gets displayed to the user is:<br>
1) not immediate<br>
2) not the same for every sink<br>
<br>
In order to get perfect sync between your displays (and also with the<br>
audio, which you don't mention), the sinks need to know the latency<br>
introduced in the render process.<br>
<br>
The current synchronization model in video sinks (and all other sink<br>
element that use the standard GstBaseSink synchronization model) is<br>
"wait-then-render".<br>
When a buffer comes in, GstBaseSink will figure out a target clock<br>
running time, wait for that moment and then call your sink's "render"<br>
method (which does the actual display/output/...).<br>
<br>
And this is where it gets tricky...<br>
<br>
In order to compensate for that render delay in the wait-then-render<br>
model, your sink needs to report that to the base class (render_delay<br>
property), so that it will subtract that value from the target clock<br>
running_time and call your render method that much earlier...<br>
... which will only work if:<br>
1) you can efficiently calculate that delay<br>
2) it is constant (some jitter might be acceptable)<br>
<br>
If you can't calculate that delay and if it's not roughly constant...<br>
you're out of luck with the current model.<br>
You could try to do some empirical testing and come up with some<br>
"better" values ("in average it's 100ms over usb and 50ms over<br>
ethernet") but it will only mitigate the problem, it won't solve it.<br>
This might be acceptable though for your use-case, you might want to try<br>
that.<br></blockquote><div><br></div><div>This sounds like a likely approach. I've been looking around at our competition and the current 'default' synchronization that GStreamer achieves is already better than many commercial products, so any improvement we can achieve would be good. We don't need 'perfect'. (And you're right, it probably can't be achieved with this hardware, but we are going for 'good, cheap, video-wall' so ...)<br>
<br>We haven't much worried about sound synchronization because many clients won't have sound turned on, and those that do will have systems that introduce delays (like surround sound systems) and so we'll probably need a hand-tuning parameter for sound sync in any particular space.<br>
<br></div><div>I will try adding some extra time to the 'render-delay' parameter in xvimagesink and see what I can accomplish, but that does make me curious. What is the downside to being over-aggressive in setting that parameter. ie, if I set it to 3 seconds, what is the downside, other than delaying start of video playing by 3 seconds?<br>
<br></div><div>I ask because we can't distinguish if the client has connected their zero-clients via USB or via Ethernet, and so can't give different values for delay that automatically compensate. Thus we're likely to set a default worst-case delay and let the client tune it with some sort of config parameter.<br>
<br></div><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">_______________________________________________<br><div class="HOEnZb"><div class="h5">
gstreamer-devel mailing list<br>
<a href="mailto:gstreamer-devel@lists.freedesktop.org">gstreamer-devel@lists.freedesktop.org</a><br>
<a href="http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel" target="_blank">http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>Stirling Westrup<br>Programmer, Entrepreneur.<br><a href="https://www.linkedin.com/e/fpf/77228" target="_blank">https://www.linkedin.com/e/fpf/77228</a><br><a href="http://www.linkedin.com/in/swestrup" target="_blank">http://www.linkedin.com/in/swestrup</a><br>
<a href="http://technaut.livejournal.com" target="_blank">http://technaut.livejournal.com</a><br><a href="http://sourceforge.net/users/stirlingwestrup" target="_blank">http://sourceforge.net/users/stirlingwestrup</a>
</div></div>