GStreamer playback of MP4 files with and without audio

[Terry Barnaby]

Thanks, but in my real case I have to do a fair bit with the separate 
video and audio streams (multiple elements with tees, AppSinks etc).

On 02/10/2024 12:36, Tim Müller via gstreamer-devel wrote:
> Hi Terry,
>
>> 1. Maybe there is some gstreamer element or attribute that can ignore 
>> the audio stream if not present somehow ?
>
> Have you tried the playbin3 
> <https://gstreamer.freedesktop.org/documentation/playback/playbin3.html?gi-language=c#playbin3-page> element 
> yet?
>
> You just set the file location in form of a URI (gst_filename_to_uri() 
> <https://gstreamer.freedesktop.org/documentation/gstreamer/gsturihandler.html?gi-language=c#gst_filename_to_uri>) 
> on it via the "uri" property, and it will figure out the 
> demuxer/parsers/decoders itself.
>
> If there's video it will plug video decoders and a video sink, 
> otherwise not.
>
> If there's audio it will plug audio decoders and an audio sink, 
> otherwise not.
>
> You can configure specific sinks via the audio-sink and video-sink 
> properties as well.
>
> You can see how it behaves with the gst-play-1.0 command line utility 
> for what it's worth, which is a wrapper around playbin/playbin3.
>
> Cheers
>  Tim
>
> PS: most discussion has shifted from the mailing list to our shiny new 
> Discourse forum <https://discourse.gstreamer.org/> these days.
>
> On Wed, 2024-10-02 at 07:21 +0100, Terry Barnaby via gstreamer-devel 
> wrote:
>>
>> I am developing a C++ video inspection program that creates 
>> MP4/H264/MP3 files and needs to play them back with the video stream 
>> processed by various gstreamer elements and audio separately.
>>
>> In general this has been working fine when just video was being 
>> recorded/played back, but I am just adding audio to the mix. Some MP4 
>> files will contain only a video stream and some will contain both 
>> video and audio streams.
>>
>> I need some way to handle the playback of these MP4 files that may or 
>> may not have MP3 audio streams.
>>
>> As a simple idea if I use something like (The real C++ code 
>> constructs the gstreamer piple line and tees the video stream to 
>> various gstreamer sub pipelines):
>>
>> gst-launch-1.0 -v filesrc location=temp.mp4 ! qtdemux name=demux
>>   demux.video_0 ! queue ! h264parse ! openh264dec ! glimagesink
>>   demux.audio_0 ! queue ! decodebin ! audioconvert ! pulsesink
>>
>> This plays back the video and audio streams fine (not sure how well 
>> synchronised?) from an MP4 with video and audio streams, but hangs if 
>> the MP4 only has a video stream.
>>
>> So I think I need to:
>>
>> 1. Maybe there is some gstreamer element or attribute that can ignore 
>> the audio stream if not present somehow ?
>>
>> 2. Check if the MP4 file has an audio stream before creating the 
>> gstreamer pipeline in C++.
>>
>> 3. Create the basic C++ pipleline in C++ and interrogate pads or 
>> something somehow and add the "demux.audio_0 ! queue ! decodebin ! 
>> audioconvert ! pulsesink" sub pipeline if it is seen an audio stream 
>> is present.
>>
>> Any ideas on the simplest/most CPU efficient way of doing this with 
>> gstreamer ?
>>
>>
>

-- 
Dr Terry Barnaby            BEAM Ltd
Phone: +44 1454 324512      Northavon Business Center,
Email:terry at beam.ltd.uk     Dean Rd, Yate
Web:www.beam.ltd.uk         Bristol, BS37 5NH, UK
BEAM Engineering: Instrumentation, Electronics/Software/Systems
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20241002/4bc7351d/attachment-0001.htm>