GStreamer playback of MP4 files with and without audio

[Terry Barnaby]

Many thanks Gareth.

Looks like your code was about the same as mine.

I have managed to get my test code to work however I don't know the 
reason it didn't work before. Enclosed is my code that works for anyone 
else in the same boat.

In my attached code only the function gstAudioAddPadCallback3() works as 
the "pad-added" callback for adding an MP3 audio processing sub pipeline.

It seems:
1. If you use decodebin in the attached sub pipeline, the pipeline will 
not go to the PLAYING state for some reason (no errors/warnings).

2. If you use gst_parse_bin_from_description(), gst_bin_add() and 
gst_element_sync_state_with_parent() the pipeline will not go to the 
PLAYING state for some reason (no errors/warnings).

No idea on why these issue occur. Using decodebin would have been more 
useful to handle different audio formats automatically (I assume it 
would have done that).

Terry
On 03/10/2024 09:24, Gareth Alldread via gstreamer-devel wrote:
> I haven't got experience with filesrc - but I attached my example 
> doing something similar with uridecodebin (which will work with an 
> video file btw).  I have stripped this out of my app - so it is 
> encapsulated inside a class - but I'm sure you get the idea.
>
>
>
> On Wed, 2 Oct 2024 at 17:46, Terry Barnaby via gstreamer-devel 
> <gstreamer-devel at lists.freedesktop.org> wrote:
>
>     I have tried using g_signal_connect(m_sourceElement, "pad-added",
>     ...) and within that adding a sub pipeline to the main pipeline,
>     but I am getting something wrong when I add the sub pipeline.
>
>     My simple program attached, sees the signal callback, but when I
>     add the sub pipeline and link it to the new pad, the displayed
>     video just freezes.
>     I've looked at the "Basic tutorial 3: Dynamic pipelines" and it
>     looks like I am doing the right thing and no errors are seen.
>
>     Any ideas on what I am ding wrong ?
>
>
>     On 02/10/2024 08:26, Gareth Alldread via gstreamer-devel wrote:
>>     I would add a handler for the "pad-added" event on the filesrc
>>     element and only create/add the audio part of the pipeline if you
>>     get an audio pad created.  That is what I have done to handle
>>     rtspsrc streams that may or may not have audio.  Reply if you
>>     want more details.
>>
>>     g_signal_connect(m_sourceElement, "pad-added",
>>     G_CALLBACK(&SynxRTSPPipeline::onPadAdded), this);
>>
>>     On Wed, 2 Oct 2024 at 07:30, Terry Barnaby via gstreamer-devel
>>     <gstreamer-devel at lists.freedesktop.org> wrote:
>>
>>         I am developing a C++ video inspection program that creates
>>         MP4/H264/MP3 files and needs to play them back with the video
>>         stream processed by various gstreamer elements and audio
>>         separately.
>>
>>         In general this has been working fine when just video was
>>         being recorded/played back, but I am just adding audio to the
>>         mix. Some MP4 files will contain only a video stream and some
>>         will contain both video and audio streams.
>>
>>         I need some way to handle the playback of these MP4 files
>>         that may or may not have MP3 audio streams.
>>
>>         As a simple idea if I use something like (The real C++ code
>>         constructs the gstreamer piple line and tees the video stream
>>         to various gstreamer sub pipelines):
>>
>>         gst-launch-1.0 -v filesrc location=temp.mp4 ! qtdemux name=demux
>>
>>           demux.video_0 ! queue ! h264parse ! openh264dec ! glimagesink
>>
>>           demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink
>>
>>         This plays back the video and audio streams fine (not sure
>>         how well synchronised?) from an MP4 with video and audio
>>         streams, but hangs if the MP4 only has a video stream.
>>
>>         So I think I need to:
>>
>>         1. Maybe there is some gstreamer element or attribute that
>>         can ignore the audio stream if not present somehow ?
>>
>>         2. Check if the MP4 file has an audio stream before creating
>>         the gstreamer pipeline in C++.
>>
>>         3. Create the basic C++ pipleline in C++ and interrogate pads
>>         or something somehow and add the "demux.audio_0 ! queue !
>>         decodebin ! audioconvert ! pulsesink" sub pipeline if it is
>>         seen an audio stream is present.
>>
>>         Any ideas on the simplest/most CPU efficient way of doing
>>         this with gstreamer ?
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20241007/208eece4/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test1.cpp
Type: text/x-c++src
Size: 7467 bytes
Desc: not available
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20241007/208eece4/attachment.cpp>