Pipeline not going into Playing state when uridecodebin with custom handler is used

Mon Apr 8 02:59:19 PDT 2013

According to the conversation on IRC in #gstreamer on 04/04/2013 my problem
seems to be specifically an issue when a *live* element is added to the
pipeline through a decodebin.
Wim reproduced it with sdpdemux and concluded that adding a live element to
the pipeline should be signalled with some sort of new message but there's
no room for new messages in GstMessageType.

Questions:
1) Is adding a live element dynamically another problem and is loosely
coupled with [1] (which BTW has been addressed so far with work-arounds
only)?

2) Does extending GstMessageType really require changing its type?
Wouldn't it be possible to ignore the fact that it's been designed to be a
mask and start adding new values (existing ones that happen to be bit masks
would be just another values)?

3) If there's a message emitted to inform that a new live element has been
added, who should react upon it?

4) Any suggested work-arounds?
Unfortunately setting "async" and "sync" on the sink to FALSE doesn't help
reliably without explicitly prodding the new live element to go to state
PLAYING.

[1] https://bugzilla.gnome.org/show_bug.cgi?id=632782

Cheers,
Kris

2013/4/3 Krzysztof Konopko <krzysztof.konopko at gmail.com>

> Do you think that gst_message_new_request_state() [1] could be of any help
> here?
>
> Is it worthwhile to send a request on behalf of the new live element to
> set it to playing and handle it in a more controlled manner in the main
> application rather than from within a custom bin in the context of the
> typefinder?  This would be a work-around I guess as the core issue still
> exists [2].
>
> [1]
> http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstMessage.html#gst-message-new-request-state
> [2] https://bugzilla.gnome.org/show_bug.cgi?id=632782
>
> Cheers,
> Kris
>
>
> 2013/4/2 Krzysztof Konopko <krzysztof.konopko at gmail.com>
>
>> OK, after some debugging I've come across a comment in gsturidecodebin.c
>> that led me to this bug:
>> https://bugzilla.gnome.org/show_bug.cgi?id=632782
>>
>> Looks like instantiating live elements through type-finding either does
>> not work or at least it's racy.
>> If this is all true and if there are any ideas of fixing it, I'd be
>> tempted to have a look and tinker a little bit with it.
>>
>> Cheers,
>> Kris
>>
>>
>> 2013/3/28 Krzysztof Konopko <krzysztof.konopko at youview.com>
>>
>>> Interestingly /decodebin2/ does not propagate instantiated element's
>>> GstStateChangeReturn return value as it's a type-found signal callback.
>>> gst-plugin-base/gst/playback/gstdecodebin2.c:connect_pad():
>>>
>>> /* Bring the element to the state of the parent */
>>>     if ((gst_element_set_state (element,
>>>                 GST_STATE_PAUSED)) == GST_STATE_CHANGE_FAILURE) {
>>>
>>> so no way to say there will be no pre-roll.
>>>
>>> Does it make any sense to try to instantiate a live element through
>>> typfinding?
>>>
>>> Cheers,
>>> Kris
>>>
>>> On 11/01/13 17:05, Krzysztof Konopko wrote:
>>> > Hi,
>>> >
>>> > I had a closer look into it. The problem can be reduced to something
>>> > like this (hope you don't mind if I rehash some bits of previous
>>> > description):
>>> >
>>> > gst-launch-1.0 \
>>> >   uridecodebin uri=file:///tmp/test.sdp caps=video/mpegts \
>>> >   ! fakesink
>>> >
>>> > On my laptop it doesn't matter whether it's a genuine HTTP or
>>> filesystem
>>> > request. It deadlocks.
>>> >
>>> >
>>> > There's a custom vqesdpdemux element which has application/sdp
>>> > capability on its sink. Here's how the gist of the problem goes:
>>> >
>>> > 1. uridecodebin instantiates filesrc, queue2 and vqesdpdemux
>>> (typefinder
>>> > magic)
>>> >
>>> > 2. vqesdpdemux receives data (buffers) from filesrc until it gets EOS
>>> > event on its sink pad (receives SDP content, simple text file)
>>> >
>>> > 3. In the same thread were the EOS event was received vqesdpdemux
>>> > (GstBin) creates a genuine custom vqesrc element which is supposed to
>>> > receive the RTP traffic
>>> >
>>> > 4. The new vqesrc element's state is synchronised with its parent
>>> > (vqesdpdemux) by calling gst_element_sync_state_with_parent() but this
>>> > can "trap" it in PAUSED or READY state (depends how far in the
>>> > transition the pipeline got after sending EOS).
>>> >
>>> > Now because vqesrc is a *live* element, it's activated pad thread waits
>>> > in gst_base_src_wait_playing() for vqesrc to get into PLAYING. But it
>>> > won't transition to that state if it's "trapped" in pipeline's
>>> > transition towards READY (depending on timing conditions).
>>> >
>>> >
>>> >
>>> > I'll be digging into it (*) but if it bells with anyone, please shout.
>>> >
>>> > Basically we're looking for (or trying to find/understand) a good
>>> > pattern to create a new source element somewhere in the *middle* of the
>>> > pipeline based on some information (buffers) already received and
>>> > potentially ditch/replace (lop off) the old bit of upstream (if
>>> > possible) after it's done its job.
>>> >
>>> > It works pretty well if the source is explicit (e. g. filesrc). Things
>>> > get complicated if one wants to use typefind magic and
>>> > auto-instantiating/connecting (some decodebin element).
>>> >
>>> > Cheers,
>>> > Kris
>>> >
>>> > (*) It's a quite involving piece of work to looking into. I'm not doing
>>> > it because I'm magnanimous but because Mariusz is sitting almost next
>>> to
>>> > me :)
>>> >
>>> > On 07/01/13 17:57, Mariusz Buras wrote:
>>> >> Hi all,
>>> >>
>>> >> I have a quite complex problem to describe so first I'll show you my
>>> >> pipeline setup:
>>> >>
>>> >>    uridecodebin uri=http://some_address_somewhere.co.uk/test.sdp
>>> >> caps=video/mpegts
>>> >>        ! queue ! tsdemux ! video/x-h264 ! decodebin ! queue !
>>> xvimagesink
>>> >>
>>> >> Where "test.sdp" is an SDP file which refers to a multicast RTP
>>> stream.
>>> >>   I have created an element "vqesrc"[1] which is a live source for
>>> this
>>> >> multicast RTP data.  It takes the contents of an SDP file as a
>>> property.
>>> >>   To get it to be autoplugged by decodebin I have also created
>>> >> vqesdpdemux which handles "application/sdp" caps much like hlsdemux or
>>> >> sdpdemux works.
>>> >>
>>> >> vqesdpdemux is a bin.  It creates a vqesrc, adds it to itself, sets
>>> the SDP
>>> >> file data as a one of the properties of vqesrc and adds a ghost pad on
>>> >> vqesdpdemux exposing the source pad of the gstvqesrc to outside of the
>>> >> bin. So when everything works I should be getting something like that
>>> on
>>> >> the diagram[2].  As you can see everything gets autoplugged as
>>> expected
>>> >> and I indeed can see live video on my screen. I have also attached a
>>> log
>>> >> file that shows what's going on when it works [3].
>>> >>
>>> >> In vqesdpdemux when I create the vqesrc child element (which is a live
>>> >> source) I am careful to call gst_element_sync_state_with_parent for
>>> the
>>> >> child.  Also I am careful to return GST_STATE_CHANGE_NO_PREROLL from
>>> >> vqesdpdemux whenever a request to enter state PAUSED is received as I
>>> >> know the child I will be adding will be live.
>>> >>
>>> >>
>>> >> The problem starts when uridecodebin is pointed to a locally stored
>>> SDP
>>> >> file, something like this:
>>> >>
>>> >>    uridecodebin uri=file:///home/mariusz.buras/tmp/test.sdp
>>> >> caps=video/mpegts
>>> >>      ! queue ! tsdemux ! video/x-h264 ! decodebin  ! queue !
>>> xvimagesink
>>> >>
>>> >> Basically, what seems to happen is that my pipeline never fully enters
>>> >> PAUSED state (only vqesdpdemux does) and gets stuck in READY forever.
>>> >> As shown in [4] pipeline start to preroll but never finishes.  My
>>> >> experiments showed that in "http case" SDP file is delivered while the
>>> >> pipline is changing it's state to PLAYING. While in the "broken" case
>>> >> it's delivered when the pipeline is transitioning to the PAUSED state.
>>> >> This leads me to believe that something wrong happens (or something
>>> good
>>> >> doesn't happen) when vqesdpdemux receives a sink event callback with a
>>> >> buffer containing the SDP file.
>>> >>
>>> >> What I have tried to fix/mitigate the problem*:
>>> >>
>>> >> 1. Forcing vqesrc element state to PLAYING just after it's added to
>>> >> vqesdpdemux bin with gst_set_state rather than
>>> >> gst_element_sync_state_with_parent. This makes it work, but I don't
>>> >> think this is a right thing to do.  It makes me a bit uneasy using  a
>>> >> half baked "fix" to a problem I don't understand.
>>> >> 2. Sending a reconfigure event. Doesn't help.
>>> >> 3. Delaying the creation of vqesrc element - this doesn't work.
>>> >> 4. Plenty of weird/desperate hacks on vqesdpdemux - same result.
>>> >>
>>> >> * when I write that "it doesn't work" I mean that it behaves in the
>>> same
>>> >> or very similar fashion to when the "fix" wasn't applied.
>>> >>
>>> >> I've also attached [5] a diagram of the last state the pipeline gets
>>> >> into when it doesn't work. This is not very helpful though, because
>>> >> the last state my pipeline fully enters is READY when no auto-plugging
>>> >> happened yet.
>>> >>
>>> >>
>>> >> I think possibly what is happening is that the pipeline is prerolling
>>> in
>>> >> the PAUSED state so vqesrc doesn't produce any data so the pipeline
>>> >> never exits the prerolling state so makes it to state PLAYING but then
>>> >> I'm unsure why it would work with the http case but not with local
>>> file.
>>> >>
>>> >>
>>> >> Can someone help me with that?
>>> >>
>>> >> Regards,
>>> >> Mariusz.
>>> >>
>>> >>
>>> >> [1] The code for both of these can be found on github:
>>> >> https://github.com/sqward/gst-vqe
>>> >> [2] An example working pipeline attached as
>>> "example_working_pipeline.jpg"
>>> >> [3] An example working log attached as "example_working_log.txt"
>>> >> [4] Log of a broken case attached as "broken_case.txt"
>>> >> [5] A diagram of "broken" pipeline when it gets stuck in READY state
>>> >> attached as "broken_case.png"
>>> >>
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> gstreamer-devel mailing list
>>> >> gstreamer-devel at lists.freedesktop.org
>>> >> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>> >>
>>> > _______________________________________________
>>> > gstreamer-devel mailing list
>>> > gstreamer-devel at lists.freedesktop.org
>>> > http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>> >
>>>
>>> This transmission contains information that may be confidential and
>>> contain personal views which are not necessarily those of YouView TV Ltd.
>>> YouView TV Ltd (Co No:7308805) is a limited liability company registered in
>>> England and Wales with its registered address at YouView TV Ltd, 3rd Floor,
>>> 10 Lower Thames Street, London, EC3R 6YT. For details see our web site at
>>> http://www.youview.com
>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/gstreamer-devel/attachments/20130408/ac6c88cf/attachment-0001.html>