[Spice-devel] 回复： Sync of audio in android

[Marc-André Lureau]

----- Original Message -----
> Hi Jojo and SPICE developers,
> 
> First of all, Jojo, thank you for your contribution to aSPICE. I am
> currently working on integrating your audio code into the main trunk.
> 
> I have also noticed that audio and video get out of sync. The issue
> seems to be more severe when the bandwidth of the connection is lower.
> At home, my phone achieves about ~21Mbit/s with my current router, and
> at work it achieves ~38Mbit/s. At 21Mbits/s, the issue is quite
> severe, but at 38Mbit/s it is a lot less noticeable. My hypothesis is
> that the sync issue occurs in part due to bandwidth limitations.
> Perhaps with a 802.11N device that is connected at 150Mbit/s or more,
> the issue will not be nearly as noticeable. A typical test of mine
> consists of playing a video through youtube, keeping the video size
> default. Can somebody comment on the bandwidth requirements of such a
> scenario and whether my observations are in line with expectations?
> 
> Regardless of what is exacerbating the out of sync issue, in the
> future we should be making use of timestamps in order to ensure that
> audio and video are synced even in scenarios with limited bandwidth.
> This would require dropping either audio packets or video frames, and
> we can have an option that controls which one is prioritized. Can
> somebody comment on that?
> 
> I am thinking that performance will be impacted by audio for the
> majority of users at the moment, and there should be an option to turn
> it on or off. I have not made up my mind what the default should be,
> though, what do you guys think?


Afaik, in general, video synchronization is done based on audio clock. Spice tries to implement a simple audio clock synchronization. Yonit has made changes to adapt the audio latency dynamically to compensate video jitter, but afaik, this is only working with the pulse audio backend. The video stream bandwidth should also be adjusted dynamically (this requires recent spice server, and the best is to ask Yonit for more details)

Audio bandwidth is small compared to video, and it's more annoying to drop audio than dropping a few video frames.

The gstreamer backend should be improved, with the end goal to drop the pulse backend. But changing latency dynamically with GStreamer seems to be tricky. Help welcome!