Playback and audio/video synchronization issues

[karimchtx]

Hi all,

I do experience stream synchronization issues between video/audio.
The audio is 0.5 sec in advance over the video :(

I wrote a custom pipeline with gstreamer api for a player application.
The pipeline is built according to the media type etc.
The audio and video streams caps are collected during the
/demuxer_pad_added_cb/.
In /demuxer_no_more_pad_cb / The video source pad gets connected to a video
sink bin.

The first audio source pad /audio_0/  gets connected to a custoom audio
sink.

That works fine.

But, in order to manage the switch between audio tracks (eng or fra or etc)
I've slightly changed the mechanism for the audio.
I'm using probes callback as described in gstreamer tutorial:
https://gstreamer.freedesktop.org/documentation/application-development/advanced/pipeline-manipulation.html?gi-language=c

As an audio track is selected, i call:

/gst_pad_add_probe(p->audio_blocking_pad,
GST_PAD_PROBE_TYPE_BLOCK_DOWNSTREAM, MediaPlayerPrivate::audio_pad_probe_cb,
p.get(), nullptr);
/
Wherep->audio_bliocking_pad is the current audio_X of the demuxer.

I then wrote, as suggested in the tutorial:
/
/////////////////////////////////////////////////////////////////////////////////////////////////////
GstPadProbeReturn
MediaPlayerPrivate::audio_pad_probe_cb(GstPad* pad, GstPadProbeInfo* info,
gpointer user_data)
{
    GstPad* srcpad, * sinkpad;

    MediaPlayerPrivate * p = static_cast<MediaPlayerPrivate *>(user_data);

    GST_DEBUG_OBJECT(p->pipeline, "pad is blocked now");

    // Remove the probe first
    gst_pad_remove_probe(pad, GST_PAD_PROBE_INFO_ID(info));

	GstElement *queue0 = gst_bin_get_by_name(GST_BIN(p->audio_sink),
"audio_queue0");

    // Install new probe for EOS
    srcpad = gst_element_get_static_pad(queue0, "src");
    gst_pad_add_probe(srcpad,
                      static_cast<GstPadProbeType>(GST_PAD_PROBE_TYPE_BLOCK
| GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM),
                      MediaPlayerPrivate::audio_event_probe_cb,
                      user_data,
                      nullptr);
    gst_object_unref(srcpad);

	gst_object_unref(queue0);

    // Push EOS into the element, the probe will be fired when the
    // EOS leaves the effect and it has thus drained all of its data

    sinkpad = gst_element_get_static_pad(p->audio_sink, "sink");
    gst_pad_send_event(sinkpad, gst_event_new_eos());
    gst_object_unref(sinkpad);

    return GST_PAD_PROBE_OK;
}/

And finally :

///////////////////////////////////////////////////////////////////////////////////////////////////////
GstPadProbeReturn
MediaPlayerPrivate::audio_event_probe_cb(GstPad* pad, GstPadProbeInfo* info,
gpointer user_data)
{
    MediaPlayerPrivate * p = static_cast<MediaPlayerPrivate *>(user_data);

    if (GST_EVENT_TYPE(GST_PAD_PROBE_INFO_DATA(info)) != GST_EVENT_EOS)
        return GST_PAD_PROBE_PASS;

    gst_pad_remove_probe(pad, GST_PAD_PROBE_INFO_ID(info));

   gst_element_set_state(p->audio_sink, GST_STATE_PAUSED);
    
    ...
    // Check if pad is already connected
    // Disconnect it, xcreates a new audio sink bin based on the audio caps
    
    gst_element_set_state(p->audio_sink, GST_STATE_PLAYING);
}
/

But that way, the audio is played 1/2 sec in advance over the video.
I'm not an expert with probes so I wonder where I should take dive into to
figure out what's happening exactly.

Any suggestion is welcome.

Cheers.

K.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/