[gst-devel] Streaming Lip Sync

[Dave Baker]

Hi again,

An update on this situation - the cause of the timestamps going out of sync in 
MPlayer was the streaming server hitting the limits of its processing power. 
I've now reduced the audio sampling rate and replaced deinterlace with 
ffdeinterlace and the problem has gone away, so that solves that one (which 
also explains why it happened when the video started moving - moving video 
being harder to encode!) Although it does highlight that if GStreamer hits 
the limit of the processor then it will break the stream until it's next 
restarted.

However, that hasn't got me any further with lip sync, since I'm now having 
roughly the same problem as I was when using V4l2. The video leads the audio 
by just under a second. I tried leaving the stream running for a couple of 
hours to see if it drifted out more, but when I came back and checked it, it 
was *in* sync, which just confused me even more.

If anyone has any flashes of inspriation as to why this might be, as ever I'd 
love to hear them, but I'm swaying more and more towards the whole DV idea.


Dave


On Wednesday 10 August 2005 09:47, Thomas Vander Stichele wrote:
> Hi,
>
> > The latest results are that I've noticed the timestamps of the audio and
> > video that MPlayer displays on the console while it's playing drift out
> > whilst MPlayer is running. This suggests to me that GStreamer thinks the
> > streams are in sync, wheras MPayer is doing the counting and finding that
> > it's getting way more audio than video, and trying to sort out the mess.
>
> Actually, that could indicate a different bug.  The ogg muxer had some
> issues in older gst-plugins prereleases and was doing the muxing in the
> wrong order.  MPlayer gets confused by this really easily, especially on
> live streams.  What version of gst-plugins are you running ? You
> definately should be running the latest - 0.8.10, and if you're not,
> it's going to mux wrong.
>
> Some other things you should try:
> - wget the file over http, and play that file locally.  That will give
> mplayer a lot more stuff to base its calculations on, and allows you to
> rule out any network issues
> - run oggz-validate on this file; this tool is part of liboggz.  It will
> allow you to verify that the files are muxed properly.
>
> > ~15:00	Mplayer gives up and starts ignoring the audio! Message: "Too many
> > video packets in the buffer: (4096 in 3939504 bytes)."
>
> This is definately a good indication that the muxing is going wrong.
> Let me know first what version of gst-plugins you were running.
>
> > The other interesting thing to note is that the timestamps stay in sync
> > for quite a while after flumotion is started, but then move apart much
> > faster when MPlayer is restarted. I can't think why this might be -
> > perhaps somenody else can.
>
> I'm not entirely sure, but my guess is that that's because timing info
> for ogg is only available at the end of decoding a full ogg page.  So at
> the start MPlayer is making a guess, and then later corrects.  I've seen
> that happen quite a few times.
>
> > It's a real shame I haven't managed to get this working as we'd be, as
> > far as I know, the first TV station broadcasting worldwide in Theora,
> > which would be great for YSTV, GStreamer and Xiph. However this is
> > driving me spare, and we need to have this running soon, so this is
> > really my last appeal for help.
>
> Well, there definately are some other options still that we could help
> you with.  By far the easiest solution, if you're worried about keeping
> sync, is to just not use v4l.  You can buy a simple Pinnacle DV
> converter box which will take in your analog feed, and output DV.  Since
> the DV format already has muxed audio and video, you will stay in sync.
> We should probably take the rest of this discussion either privately or
> on flumotion-devel though.
>
> Thomas
>
> Dave/Dina : future TV today ! - http://www.davedina.org/
> <-*- thomas (dot) apestaart (dot) org -*->
> I took your advice
> Bought the microscope
> I can't find anything
> I wanna see up close
> <-*- thomas (at) apestaart (dot) org -*->
> URGent, best radio on the net - 24/7 ! - http://urgent.fm/