Pipeline not going into Playing state when uridecodebin with custom handler is used

Mon Apr 8 09:32:25 PDT 2013

Thanks Wind for sharing your experience.

I think that the problem is well understood now and GStreamer maintainers
confirmed that some scenarios are not handled properly.  My problem is
twofold:
* There's a *live* element being added *dynamically*, so there's *no*
pre-roll anywhere downstream of that element.  Currently there's no way in
GStreamer to communicate this fact in certain contexts (like from within a
decodebin)
*  There's some state change inconsistency in non-top level bins when
there's asynchronous state change and NO_PREROLL state change return
involved [1]

I worked-around both problems by:
* Setting both "async" and "sync" properties of my sink element to FALSE.
 These properties do not matter in my situation anyway as the sink is not a
genuine media stream consumer--it passes it to some other process which
handles synchronisation on its own.
* Forcing a new live element to go to state PLAYING within my bin after
synchronising its state with the parent

[1] https://bugzilla.gnome.org/show_bug.cgi?id=632782

Cheers,
Kris

2013/4/8 Yuan, Feng <feng.yuan at intel.com>

>  Hi,****
>
>      I didn’t read every thread carefully, but just want to share some
> experience, and hope may help you.****
>
>      If some elements ever returned < GST_STATE_CHANGE_ASYNC>, e.g. from
> READY to PAUSE, this element need handle preroll to post a
> gst_message_new_state_changed when first buffer coming, in order to make
> states changing going on. In this situation, some case like single thread
> may get dead-lock if multiple elements need to handle the same issue. Maybe
> you can try to add a <queue> between these elements or make the elment(or
> bin) post the message in another thread/async.****
>
>      Just hope this can help you.****
>
> Thanks,****
>
> Wind****
>
> ** **
>
> *From:* gstreamer-devel-bounces+feng.yuan=intel.com at lists.freedesktop.org[mailto:
> gstreamer-devel-bounces+feng.yuan=intel.com at lists.freedesktop.org] *On
> Behalf Of *Krzysztof Konopko
> *Sent:* Monday, April 08, 2013 5:59 PM
> *To:* Discussion of the development of and with GStreamer
> *Subject:* Re: Pipeline not going into Playing state when uridecodebin
> with custom handler is used****
>
> ** **
>
> According to the conversation on IRC in #gstreamer on 04/04/2013 my
> problem seems to be specifically an issue when a *live* element is added to
> the pipeline through a decodebin.****
>
> Wim reproduced it with sdpdemux and concluded that adding a live element
> to the pipeline should be signalled with some sort of new message but
> there's no room for new messages in GstMessageType.****
>
> ** **
>
> Questions:****
>
> 1) Is adding a live element dynamically another problem and is loosely
> coupled with [1] (which BTW has been addressed so far with work-arounds
> only)?****
>
> ** **
>
> 2) Does extending GstMessageType really require changing its type?****
>
> Wouldn't it be possible to ignore the fact that it's been designed to be a
> mask and start adding new values (existing ones that happen to be bit masks
> would be just another values)?****
>
> ** **
>
> 3) If there's a message emitted to inform that a new live element has been
> added, who should react upon it?****
>
> ** **
>
> 4) Any suggested work-arounds?****
>
> Unfortunately setting "async" and "sync" on the sink to FALSE doesn't help
> reliably without explicitly prodding the new live element to go to state
> PLAYING. ****
>
> ** **
>
> [1] https://bugzilla.gnome.org/show_bug.cgi?id=632782****
>
> ** **
>
> Cheers,****
>
> Kris****
>
> ** **
>
> 2013/4/3 Krzysztof Konopko <krzysztof.konopko at gmail.com>****
>
> Do you think that gst_message_new_request_state() [1] could be of any help
> here?****
>
> ** **
>
> Is it worthwhile to send a request on behalf of the new live element to
> set it to playing and handle it in a more controlled manner in the main
> application rather than from within a custom bin in the context of the
> typefinder?  This would be a work-around I guess as the core issue still
> exists [2].****
>
> ** **
>
> [1]
> http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstMessage.html#gst-message-new-request-state
> ****
>
> [2] https://bugzilla.gnome.org/show_bug.cgi?id=632782****
>
> ** **
>
> Cheers,****
>
> Kris****
>
> ** **
>
> 2013/4/2 Krzysztof Konopko <krzysztof.konopko at gmail.com>****
>
> OK, after some debugging I've come across a comment in gsturidecodebin.c
> that led me to this bug:****
>
> https://bugzilla.gnome.org/show_bug.cgi?id=632782****
>
> ** **
>
> Looks like instantiating live elements through type-finding either does
> not work or at least it's racy.****
>
> If this is all true and if there are any ideas of fixing it, I'd be
> tempted to have a look and tinker a little bit with it.****
>
> ** **
>
> Cheers,****
>
> Kris****
>
> ** **
>
> 2013/3/28 Krzysztof Konopko <krzysztof.konopko at youview.com>****
>
> Interestingly /decodebin2/ does not propagate instantiated element's
> GstStateChangeReturn return value as it's a type-found signal callback.
> gst-plugin-base/gst/playback/gstdecodebin2.c:connect_pad():
>
> /* Bring the element to the state of the parent */
>     if ((gst_element_set_state (element,
>                 GST_STATE_PAUSED)) == GST_STATE_CHANGE_FAILURE) {
>
> so no way to say there will be no pre-roll.
>
> Does it make any sense to try to instantiate a live element through
> typfinding?
>
> Cheers,
> Kris****
>
>
> On 11/01/13 17:05, Krzysztof Konopko wrote:
> > Hi,
> >
> > I had a closer look into it. The problem can be reduced to something
> > like this (hope you don't mind if I rehash some bits of previous
> > description):
> >
> > gst-launch-1.0 \
> >   uridecodebin uri=file:///tmp/test.sdp caps=video/mpegts \
> >   ! fakesink
> >
> > On my laptop it doesn't matter whether it's a genuine HTTP or filesystem
> > request. It deadlocks.
> >
> >
> > There's a custom vqesdpdemux element which has application/sdp
> > capability on its sink. Here's how the gist of the problem goes:
> >
> > 1. uridecodebin instantiates filesrc, queue2 and vqesdpdemux (typefinder
> > magic)
> >
> > 2. vqesdpdemux receives data (buffers) from filesrc until it gets EOS
> > event on its sink pad (receives SDP content, simple text file)
> >
> > 3. In the same thread were the EOS event was received vqesdpdemux
> > (GstBin) creates a genuine custom vqesrc element which is supposed to
> > receive the RTP traffic
> >
> > 4. The new vqesrc element's state is synchronised with its parent
> > (vqesdpdemux) by calling gst_element_sync_state_with_parent() but this
> > can "trap" it in PAUSED or READY state (depends how far in the
> > transition the pipeline got after sending EOS).
> >
> > Now because vqesrc is a *live* element, it's activated pad thread waits
> > in gst_base_src_wait_playing() for vqesrc to get into PLAYING. But it
> > won't transition to that state if it's "trapped" in pipeline's
> > transition towards READY (depending on timing conditions).
> >
> >
> >
> > I'll be digging into it (*) but if it bells with anyone, please shout.
> >
> > Basically we're looking for (or trying to find/understand) a good
> > pattern to create a new source element somewhere in the *middle* of the
> > pipeline based on some information (buffers) already received and
> > potentially ditch/replace (lop off) the old bit of upstream (if
> > possible) after it's done its job.
> >
> > It works pretty well if the source is explicit (e. g. filesrc). Things
> > get complicated if one wants to use typefind magic and
> > auto-instantiating/connecting (some decodebin element).
> >
> > Cheers,
> > Kris
> >
> > (*) It's a quite involving piece of work to looking into. I'm not doing
> > it because I'm magnanimous but because Mariusz is sitting almost next to
> > me :)
> >
> > On 07/01/13 17:57, Mariusz Buras wrote:
> >> Hi all,
> >>
> >> I have a quite complex problem to describe so first I'll show you my
> >> pipeline setup:
> >>
> >>    uridecodebin uri=http://some_address_somewhere.co.uk/test.sdp
> >> caps=video/mpegts
> >>        ! queue ! tsdemux ! video/x-h264 ! decodebin ! queue !
> xvimagesink
> >>
> >> Where "test.sdp" is an SDP file which refers to a multicast RTP stream.
> >>   I have created an element "vqesrc"[1] which is a live source for this
> >> multicast RTP data.  It takes the contents of an SDP file as a property.
> >>   To get it to be autoplugged by decodebin I have also created
> >> vqesdpdemux which handles "application/sdp" caps much like hlsdemux or
> >> sdpdemux works.
> >>
> >> vqesdpdemux is a bin.  It creates a vqesrc, adds it to itself, sets the
> SDP
> >> file data as a one of the properties of vqesrc and adds a ghost pad on
> >> vqesdpdemux exposing the source pad of the gstvqesrc to outside of the
> >> bin. So when everything works I should be getting something like that on
> >> the diagram[2].  As you can see everything gets autoplugged as expected
> >> and I indeed can see live video on my screen. I have also attached a log
> >> file that shows what's going on when it works [3].
> >>
> >> In vqesdpdemux when I create the vqesrc child element (which is a live
> >> source) I am careful to call gst_element_sync_state_with_parent for the
> >> child.  Also I am careful to return GST_STATE_CHANGE_NO_PREROLL from
> >> vqesdpdemux whenever a request to enter state PAUSED is received as I
> >> know the child I will be adding will be live.
> >>
> >>
> >> The problem starts when uridecodebin is pointed to a locally stored SDP
> >> file, something like this:
> >>
> >>    uridecodebin uri=file:///home/mariusz.buras/tmp/test.sdp
> >> caps=video/mpegts
> >>      ! queue ! tsdemux ! video/x-h264 ! decodebin  ! queue ! xvimagesink
> >>
> >> Basically, what seems to happen is that my pipeline never fully enters
> >> PAUSED state (only vqesdpdemux does) and gets stuck in READY forever.
> >> As shown in [4] pipeline start to preroll but never finishes.  My
> >> experiments showed that in "http case" SDP file is delivered while the
> >> pipline is changing it's state to PLAYING. While in the "broken" case
> >> it's delivered when the pipeline is transitioning to the PAUSED state.
> >> This leads me to believe that something wrong happens (or something good
> >> doesn't happen) when vqesdpdemux receives a sink event callback with a
> >> buffer containing the SDP file.
> >>
> >> What I have tried to fix/mitigate the problem*:
> >>
> >> 1. Forcing vqesrc element state to PLAYING just after it's added to
> >> vqesdpdemux bin with gst_set_state rather than
> >> gst_element_sync_state_with_parent. This makes it work, but I don't
> >> think this is a right thing to do.  It makes me a bit uneasy using  a
> >> half baked "fix" to a problem I don't understand.
> >> 2. Sending a reconfigure event. Doesn't help.
> >> 3. Delaying the creation of vqesrc element - this doesn't work.
> >> 4. Plenty of weird/desperate hacks on vqesdpdemux - same result.
> >>
> >> * when I write that "it doesn't work" I mean that it behaves in the same
> >> or very similar fashion to when the "fix" wasn't applied.
> >>
> >> I've also attached [5] a diagram of the last state the pipeline gets
> >> into when it doesn't work. This is not very helpful though, because
> >> the last state my pipeline fully enters is READY when no auto-plugging
> >> happened yet.
> >>
> >>
> >> I think possibly what is happening is that the pipeline is prerolling in
> >> the PAUSED state so vqesrc doesn't produce any data so the pipeline
> >> never exits the prerolling state so makes it to state PLAYING but then
> >> I'm unsure why it would work with the http case but not with local file.
> >>
> >>
> >> Can someone help me with that?
> >>
> >> Regards,
> >> Mariusz.
> >>
> >>
> >> [1] The code for both of these can be found on github:
> >> https://github.com/sqward/gst-vqe
> >> [2] An example working pipeline attached as
> "example_working_pipeline.jpg"
> >> [3] An example working log attached as "example_working_log.txt"
> >> [4] Log of a broken case attached as "broken_case.txt"
> >> [5] A diagram of "broken" pipeline when it gets stuck in READY state
> >> attached as "broken_case.png"
> >>
> >>
> >>
> >> _______________________________________________
> >> gstreamer-devel mailing list
> >> gstreamer-devel at lists.freedesktop.org
> >> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >>
> > _______________________________________________
> > gstreamer-devel mailing list
> > gstreamer-devel at lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >****
>
> This transmission contains information that may be confidential and
> contain personal views which are not necessarily those of YouView TV Ltd.
> YouView TV Ltd (Co No:7308805) is a limited liability company registered in
> England and Wales with its registered address at YouView TV Ltd, 3rd Floor,
> 10 Lower Thames Street, London, EC3R 6YT. For details see our web site at
> http://www.youview.com
>
> ****
>
> ** **
>
> ** **
>
> ** **
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20130408/5c9584b7/attachment-0001.html>