demuxing mp4 and queues

[Wudo Balmus]

Thanks for the answers :) , they're really helpful. I played a bit with the
code and here are some comments. If there is a mistake, I would be thankful
for correction.

For the pipeline like that (no queues):

gst-launch-1.0 -v filesrc location=working.mp4 ! qtdemux name=a a. !
avdec_h264 ! videoconvert ! autovideosink a. ! avdec_aac ! autoaudiosink

It looks that autoaudiosink (checked by aborting the process and inspecting
coredump) blocks the pipeline while calling the function
gst_base_sink_wait_preroll:

GstFlowReturn
gst_base_sink_wait_preroll (GstBaseSink * sink)
{
  sink->have_preroll = TRUE;
  GST_DEBUG_OBJECT (sink, "waiting in preroll for flush or PLAYING");
  /* block until the state changes, or we get a flush, or something */
  GST_BASE_SINK_PREROLL_WAIT (sink); <-- HERE IT'S BLOCKING
[...]

It happens when all elements in the pipeline are changing state from READY
to PAUSED. There is a buffer sent from qtdemux->avdec_aac->autoaudiosink
and in the end of the chained call, in the audio sink there is called the
function gst_base_sink_wait_preroll which blocks infinitely. Then perhaps
it blocks qtdemux from sending video frames to preroll video sink since it
all happens in the same thread. In the end, the state of all elements
doesn't go to PAUSED and then PLAYING.

However when there are queues between demuxer and decoders, then the
demuxer, video sink and audio sink all live in their own threads. Demuxer
is gently pushing buffers to the queues in one thread and decoders in their
own threads pick them up. In the end all sinks are prerolled and the state
of pipeline can change to PAUSED and then PLAYING.

Best regards,
Wudo

śr., 6 mar 2019 o 16:43 David Ing <ding at panopto.com> napisał(a):

> I have also learned that queues are needed on the branches of a tee (for
> probably the same reasons).
>
> For more information on what I learned about queues and tees:
> http://gstreamer-devel.966125.n4.nabble.com/Usage-of-tee-with-queue-td4689894.html
>
> There is also some great documentation about how queues function as thread
> boundaries:
> https://gstreamer.freedesktop.org/documentation/application-development/advanced/threads.html
>
> On Wed, Mar 6, 2019 at 3:45 AM Mathieu Duponchelle <
> mathieu at centricular.com> wrote:
>
>> This is a classical problem, and a good illustration why you usually need
>> queues after elements
>> with N src pads, and before elements with N sink pads:
>>
>> The demuxer is pushing to its source pads from the same thread, and if
>> you have connected
>> these source pads with sync=true sinks, their chain function blocks until
>> prerolling is done,
>> that is until each sink with sync=true has received a buffer. Adding
>> queues decouples branches of
>> the pipeline, as they will start their own streaming thread.
>>
>> On 3/5/19 11:11 PM, Wudo Balmus wrote:
>>
>> Hello,
>> I'm trying to demux and playback an mp4 file with h.264 video and aac
>> audio. This pipeline works fine:
>>
>> gst-launch-1.0 -v filesrc location=working.mp4 ! qtdemux name=a a. !
>> queue ! avdec_h264 ! videoconvert ! autovideosink a. ! queue ! avdec_aac !
>> autoaudiosink
>>
>> However when the queues are removed, it hangs, nothing is played back:
>>
>> gst-launch-1.0 -v filesrc location=working.mp4 ! qtdemux name=a a. !
>> avdec_h264 ! videoconvert ! autovideosink a. ! avdec_aac ! autoaudiosink
>>
>> It works fine without a queue when I just try to playback video without
>> audio:
>>
>> gst-launch-1.0 -v filesrc location=working.mp4 ! qtdemux name=a a. !
>> avdec_h264 ! videoconvert ! autovideosink
>>
>> I'm trying to understand why queues are necessary in this scenario.
>> Cannot qtdemux just write data to video/audio decoder without the queues?
>>
>> Best regards,
>> Wudo
>>
>> _______________________________________________
>> gstreamer-devel mailing listgstreamer-devel at lists.freedesktop.orghttps://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20190306/02e54f24/attachment.html>