decodebin decodegroup/decodechain management ==> was Re: observing problems with dynamic pipeline

Wed May 3 07:09:20 UTC 2017

Hello,

Gentle reminder. Your suggestions/opinion would certainly be of great help.

Best Regards
Nitin

On Mon, May 1, 2017 at 8:36 PM, Nitin Mahajan <rise.era at gmail.com> wrote:

> Updating the subject after further analysis.
> Refining problem statement:
> decodebin2 is nased on DecodeGroup which contains DecodeChain per stream
> (video/audio)
> as I explained earlier, my requirement is to remove one stream from a
> Decodegroup (remove audio DecodeChain) by pushing EOS on demux audio src
> pad and remove the pad itself.
> Then create Decodechain with new audio stream by creating new demuxer
> audio src pad.
> In this process, the problem by going through decodebin design seems that
> the new decodegroup will be a separate decodegroup which essentially means
> that an audio stream (audio chain) specific DecodeGroup. That would result
> into 2 separate DecodeGroups which means that A/V playback can not work as
> both streams are in separate DecodeGroup. Is it correct understanding ?
> In order to solve this problem, probably it would mean to delete all pads
> of original DecoderGroup and recreate new DecodeGroup with all streams
> again Older Video Stream and new audio stream. But this would involve a
> video switch and with an objective to just change audio, the overall exp
> will be probably a black screen or video freeze when switching across
> decodegroups. Is it correct understanding ?
>
> Also do we have strict sequence to follow to add new pad, send eos on old
> pads and removal of old pads in order that original DecodeGroup is drained
> successfully and switch to new DecodeGroup is successful ?
>
> Also noted some documentation on DecodeBin3
> https://gstreamer.freedesktop.org/data/events/gstreamer-
> conference/2015/Edward%20Hervey%20-%20decodebin3.pdf
>
> Looking forward to understand from the decodebin design perspective as
> currently struggling to even remove older DecoderGroup and create new
> DecodeGroup, in my example it seems that there are 2 Decodegroups with
> video stream in original and audio stream in latest one.
>
> Thanks in advance for your help to chelp understand decodebin design and
> usage.
>
> BR
> Nitin
>
>
> On Sat, Apr 29, 2017 at 2:14 PM, Nitin Mahajan <rise.era at gmail.com> wrote:
>
>> Hello folks,
>>
>> I'm facing some problem while playing around with with dynamic pipelines.
>> Need your help / suggestions based on below scenario in case observer
>> something wrong in approach.
>>
>> I have a gstreamer demuxer plugin (not the standard plugin) with 1 video
>> and 1 audio pad.
>> This plugin has 2 source pads: video pad, audio pad. (always 2
>> pads, doesn't have multiple audio pads per stream as per regular demuxers)
>>
>> For each pad there is dedicated gst_buffer_pool allocated which contains
>> pool of buffers to host Video and Audio ES post demuxing. In
>> this configuration A/V playback is working OK (the pipeline is constructed
>> with playbin)
>>
>> Now on top of this I have to support switching to another audio stream
>> present in the container (MP4) at run time when gstreamer pipeline is in
>> playing state (with same reason as explained above that number of src pads
>> is fixed)
>>
>> demux has to switch to another audio stream as mentioned above which has *different
>> *codec type. So following the dynamic pipeline principles, following
>> steps are done in sequence to attain this objective:
>> While injection to demux video pad is ongoing (I guess it should be ok to
>> let continue video inject because we are considering deletion and creation
>> of new src pad for audio only chain) ,older audio pad is removed by
>> following below steps:
>> flush the pipeline, send eos event and remove pad followed by no more pad
>> notification:
>> 1. gst_pad_push_event (audio_stream->pad, gst_event_new_flush_start ());
>> 2. gst_pad_push_event (audio_stream->pad, gst_event_new_flush_stop
>> (TRUE));
>> 3. gst_pad_push_event (audio_stream->pad, gst_event_new_eos ());
>> 4. Flush the buffer pool associated associated with audio stream
>> 5. gst_element_remove_pad (xx, audio_stream->pad);
>> 6. gst_element_no_more_pads (xx);
>>
>> And then a new audio pad is created with following operations
>> 1. gst_pad_set_active pad
>> 2. gst_pad_use_fixed_caps
>> 3. gst_pad_push_event gst_event_new_stream_start
>> 4. gst_pad_set_caps pad
>> 5. gst_element_add_pad
>> 6. new segment event
>> 7. caps event
>> 8. push buffers
>>
>> gstreamer version: 1.4.5
>>
>> Note that all above steps are done in sequence and in same thread context
>> which we can call as streaming thread as per gstreamer design. Being
>> sequential operations there is no possibility to push buffers or events on
>> audio stream src pad (old and then new) from any other thread.
>> But during this period I have another thread which keeps on pushing the
>> buffers on video src pad (which although little different from standard
>> gstreamer streaming thread concept but since the ongoing operations are on
>> audio stream pad so it should not create problem .. problem with buffer
>> timestamps in case ?
>>
>> The expectation was, after above steps, we can inject audio buffers on
>> new audio pad and overall there should be just change in audio stream with
>> no problem on video.
>> But the observations are not as per expectation !!!!
>>
>> Video is stalled for some time when older audio pad is deleted and new
>> audio pad is created.
>> Thereafter video playback is fine but there is no audio and from demux
>> perspective audio packets are pushed without any error on new audio pad.
>> And the pipeline dump gives an idea that older chain is still not deleted
>> and new chain is not built yet.
>>
>> Taking dump of GST pipeline it seems that older pipeline is not deleted
>> and new pipeline from new audio pad is partially created upto proxypad16
>> inside decodebin,
>> This pad is connected with Gstaacparse src pad but the chain meets dead
>> end at this stage !!!!
>>
>> gst_pad_push() is always successful on new audio pad. The buffers which
>> are pushed on audio pad are allocated from buffer pool.
>>
>> Any suggestions for debugging will be helpful especially with dynamic
>> pipeline management (gstreamer 1.4.5)
>>
>> Thanks
>> Nitin
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20170503/7ba4d775/attachment-0001.html>