GStreamer playback of MP4 files with and without audio

[Tim Müller]

Hi Terry,

> 1. Maybe there is some gstreamer element or attribute that can ignore
> the audio stream if not present somehow ?

Have you tried the playbin3 [1] element yet?

You just set the file location in form of a URI (gst_filename_to_uri()
[2]) on it via the "uri" property, and it will figure out the
demuxer/parsers/decoders itself.

If there's video it will plug video decoders and a video sink,
otherwise not.

If there's audio it will plug audio decoders and an audio sink,
otherwise not.

You can configure specific sinks via the audio-sink and video-sink
properties as well.

You can see how it behaves with the gst-play-1.0 command line utility
for what it's worth, which is a wrapper around playbin/playbin3.

Cheers
 Tim

PS: most discussion has shifted from the mailing list to our shiny
new Discourse forum [3] these days.

On Wed, 2024-10-02 at 07:21 +0100, Terry Barnaby via gstreamer-devel
wrote:
>  
> I am developing a C++ video inspection program that creates
> MP4/H264/MP3 files and needs to play them back with the video stream
> processed by various gstreamer elements and audio separately.
>  
> In general this has been working fine when just video was being
> recorded/played back, but I am just adding audio to the mix. Some MP4
> files will contain only a video stream and some will contain both
> video and audio streams.
>  
> I need some way to handle the playback of these MP4 files that may or
> may not have MP3 audio streams.
>  
> As a simple idea if I use something like (The real C++ code
> constructs the gstreamer piple line and tees the video stream to
> various gstreamer sub pipelines):
>  
> gst-launch-1.0 -v filesrc location=temp.mp4 ! qtdemux name=demux
>  
>  demux.video_0 ! queue ! h264parse ! openh264dec ! glimagesink
>  
>  demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink
>  
> This plays back the video and audio streams fine (not sure how well
> synchronised?) from an MP4 with video and audio streams, but hangs if
> the MP4 only has a video stream.
>  
> So I think I need to:
>  
> 1. Maybe there is some gstreamer element or attribute that can ignore
> the audio stream if not present somehow ?
>  
> 2. Check if the MP4 file has an audio stream before creating the
> gstreamer pipeline in C++.
>  
> 3. Create the basic C++ pipleline in C++ and interrogate pads or
> something somehow and add the "demux.audio_0 ! queue ! decodebin !
> audioconvert ! pulsesink" sub pipeline if it is seen an audio stream
> is present.
> Any ideas on the simplest/most CPU efficient way of doing this with
> gstreamer ?
> 
>  
>  


[1] playbin3
    https://gstreamer.freedesktop.org/documentation/playback/playbin3.html?gi-language=c#playbin3-page
[2] gst_filename_to_uri()
    https://gstreamer.freedesktop.org/documentation/gstreamer/gsturihandler.html?gi-language=c#gst_filename_to_uri
[3] Discourse forum https://discourse.gstreamer.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20241002/613f2569/attachment.htm>