query about audio track switch with decodebin2 (single audio decode, not multiple decode as standard gstreamer)

[Nitin Mahajan]

Dear Edward, Jan and other experts,

I have a specific query. My demux plugin outputs video and audio packets
(elementary streams) whereas hlsdemux outputs video and audio but in
container. The gstreamer pipeline is running fine.

Now I want to switch audio track but number of pads would remain the same
i.e. one video pad and one audio pad. And there is a change in audio codec.
So that essentially means to have new audio chain.

As suggested by Edward Hervy on following links:
https://lwn.net/Articles/661403/
https://gstreamer.freedesktop.org/data/events/gstreamer-conference/2015/Edward%20Hervey%20-%20decodebin3.pdf

There are some known issues (explained by Edward in above links) that we
can not just remove audio pad and add new audio pad. We would have to
create new video and audio pad, push eos on old video pad and old audio
pad, remove old video pad and remove old audio pad.And then start pushing
packets on the new video pad and new audio pad. Currently my gstreamer
version is 1.4.5 which is based on decodebin2.

In such scenario just before track change:
- significant video of future timestamps is already buffered in the video
chain (old chain)
- significant audio of future timestamps is already buffered in the audio
chain  (old chain)

Now when switching to push buffers on new pads, for audio case, we have
capability to "rewind" the PTS as per the current presentation time. But
for video case, the video packets would have timestamps which is much ahead
as compared to audio. I'm not sure if hlsdemux also rewinds the video when
it performs the track switch ?

I think it will create problem at synchronisation stage when video is much
ahead.
How such issues are handled at hlsdemux or inside any other demux plugin
which perform decode group switch ? Do they also "rewind" video before
pushing on video chain ?

Do we need to also FLUSH *old *video and audio chains also before start
pushing buffers on *new *video and audio pad ?

Now we also have a situation when there is no codec change we are trying to
flush audio without removing audio pad and observing synchronization issues:
Please have a look at,
http://gstreamer-devel.966125.n4.nabble.com/a-v-freeze-while-flushing-audio-chain-of-pipeline-in-running-state-clock-and-renderer-sync-related-td4683168.html
and suggest how we can address this behaviour.

In both cases we would not intend to flush video. Is there any specific
requirement to flush video also to handle a/v freeze..

Kindly share your expert insights to help with decodebin2 based audio track
switch with single audio pad that outputs elementary stream.

Thanks
--Nitin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20170609/346a4197/attachment.html>