[gst-devel] Streaming Lip Sync

[Dave Baker]

Hi,

Thanks very much for the reply. There were some very useful suggestions there, 
which I've tested:

I tried wget-ing the stream, and MPLayer does handle it better - it plays 
seemingly random chunks of audio for a short time, and then gives up and just 
plays the video. It looks like the same as what it's doing whilst streaming, 
expect that the audio it's looking for is either present or not, and it 
doesn't wait for it as it does when streaming. oggz-validate can't find any 
problems with the wget-ed files (returns nothing).

Oh, and yes - I'm running gst-plugins 0.8.10.

I did discover something quite interesting though. As it happens, our 
continuity service machine must have crashed last night, so it's showing a 
still frame. Oddly, when I started flumotion and MPlayer in the same way, the 
timestamps stayed exactly in sync. I left it doing this for quite some time 
with no problems (I don't know if they really were in sync, since it was just 
a still frame and silence). Having done this, I played some video into it, 
and almost immediately the same problem occurred - the timestamps started to 
ebb away from each other and MPayer started to pause the playback.

So it seems that the problem only occurs with moving video! I've just played 
the audio and video from the same file separately down the stream, and it 
does only happen with the video, and not the audio. It seems to be able to 
cope fine with small changes - cutting to our logo didn't affect it, and 
neither did the logo animation which lasts about a second. The still slide 
isn't exactly still either since it goes through a nasty multiplexer which 
adds moving analogue artifact. However the revealing/hiding text on our 
ticker bar was enough to cause problems, and the video I've been testing with 
has a huge amount of motion.

I'll leave you to draw conclusions from this, since I don't know the internals 
of GStreamer, but theroa's VBR must be confusing something.

That all said, I'm very tempted by the DV converter box option, as it might 
make things so much simpler and more reliable. I'll probably ask on 
flumotion-devel about that.

Thanks for your help!


Dave


On Wednesday 10 August 2005 09:47, Thomas Vander Stichele wrote:
> Hi,
>
> > The latest results are that I've noticed the timestamps of the audio and
> > video that MPlayer displays on the console while it's playing drift out
> > whilst MPlayer is running. This suggests to me that GStreamer thinks the
> > streams are in sync, wheras MPayer is doing the counting and finding that
> > it's getting way more audio than video, and trying to sort out the mess.
>
> Actually, that could indicate a different bug.  The ogg muxer had some
> issues in older gst-plugins prereleases and was doing the muxing in the
> wrong order.  MPlayer gets confused by this really easily, especially on
> live streams.  What version of gst-plugins are you running ? You
> definately should be running the latest - 0.8.10, and if you're not,
> it's going to mux wrong.
>
> Some other things you should try:
> - wget the file over http, and play that file locally.  That will give
> mplayer a lot more stuff to base its calculations on, and allows you to
> rule out any network issues
> - run oggz-validate on this file; this tool is part of liboggz.  It will
> allow you to verify that the files are muxed properly.
>
> > ~15:00	Mplayer gives up and starts ignoring the audio! Message: "Too many
> > video packets in the buffer: (4096 in 3939504 bytes)."
>
> This is definately a good indication that the muxing is going wrong.
> Let me know first what version of gst-plugins you were running.
>
> > The other interesting thing to note is that the timestamps stay in sync
> > for quite a while after flumotion is started, but then move apart much
> > faster when MPlayer is restarted. I can't think why this might be -
> > perhaps somenody else can.
>
> I'm not entirely sure, but my guess is that that's because timing info
> for ogg is only available at the end of decoding a full ogg page.  So at
> the start MPlayer is making a guess, and then later corrects.  I've seen
> that happen quite a few times.
>
> > It's a real shame I haven't managed to get this working as we'd be, as
> > far as I know, the first TV station broadcasting worldwide in Theora,
> > which would be great for YSTV, GStreamer and Xiph. However this is
> > driving me spare, and we need to have this running soon, so this is
> > really my last appeal for help.
>
> Well, there definately are some other options still that we could help
> you with.  By far the easiest solution, if you're worried about keeping
> sync, is to just not use v4l.  You can buy a simple Pinnacle DV
> converter box which will take in your analog feed, and output DV.  Since
> the DV format already has muxed audio and video, you will stay in sync.
> We should probably take the rest of this discussion either privately or
> on flumotion-devel though.
>
> Thomas
>
> Dave/Dina : future TV today ! - http://www.davedina.org/
> <-*- thomas (dot) apestaart (dot) org -*->
> I took your advice
> Bought the microscope
> I can't find anything
> I wanna see up close
> <-*- thomas (at) apestaart (dot) org -*->
> URGent, best radio on the net - 24/7 ! - http://urgent.fm/