[gst-devel] User question: remux MP4 file/stream (H.264/AAC) to MPEG2TS file/stream

Thu Jul 16 03:24:26 CEST 2009

Hi all,

Sorry to post this on the devel list, but I'm not sure where else to
go...

I'm just getting started with gstreamer and am trying to create a
gst-launch pipeline to read an .m4v (MP4 file, but eventually an rtp or
udp stream, with H.264 & AAC data inside) and remux it to an MPEG2 TS
file (this will be changed to a UDP stream when it's working). I'm
testing with VLC v1.0.0 on WinXP.

On CentOS 5.3:

# yum list installed '*gstreamer*'
Loaded plugins: fastestmirror, priorities
Installed Packages
gstreamer.i386                                        0.10.20-3.el5
installed
gstreamer-ffmpeg.i386                                 0.10.2-2.el5.rf
installed
gstreamer-plugins-bad.i386                            0.10.8-3.el5.rf
installed
gstreamer-plugins-base.i386
0.10.20-3.0.1.el5_3                            installed
gstreamer-plugins-good.i386                           0.10.9-1.el5_3.2
installed
gstreamer-plugins-ugly.i386                           0.10.9-1.el5.rf
installed
gstreamer-tools.i386                                  0.10.20-3.el5
installed

Input file is 95sec of: 400Kb/s 480x320, 10fps H.264 video, with 20Kb/s
22050Hz, stereo AAC-LC audio.

# ls -l infile.m4v
-rw-r--r-- 1 root root 5300576 Jul  3 15:23 infile.m4v

[1] At first I hoped that the plumbing would sort itself out and tried
the really naive:

#  gst-launch-0.10 -v filesrc location=infile.m4v ! qtdemux  !
ffmux_mpegts ! filesink location=outfile.ts
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/pipeline0/ffmux_mpegts0.video_0: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 132343612 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
/pipeline0/ffmux_mpegts0.video_0: caps = NULL
/pipeline0/qtdemux0.audio_00: caps = NULL
/pipeline0/qtdemux0.video_00: caps = NULL
Setting pipeline to NULL ...
FREEING pipeline ...

# ls -l outfile.ts
-rw-r--r-- 1 root root 8036060 Jul 16 10:20 outfile.ts

This pipeline runs, but VLC can't play the larger (8MB, from a 5MB
source) outfile.ts... It opens the file, but has "0:00/0:00" in the time
field and while it scrolls the playback position from left to right in
approx the right time, there is no audio or video output.

[2] Next, after some googling, I tried:

# gst-launch-0.10 -v filesrc location=infile.m4v ! qtdemux name=d \
     d. ! video/x-h264 ! m. \
     d. ! audio/aac ! m. \
     ffmux_mpegts name=m ! filesink location=outfile.ts
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/pipeline0/capsfilter2: caps = video/x-h264
/pipeline0/capsfilter3: caps = audio/aac
/pipeline0/capsfilter4: caps = audio/aac
/pipeline0/capsfilter2.src: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
/pipeline0/capsfilter2.sink: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
/pipeline0/m.video_0: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 132576270 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
/pipeline0/m.video_0: caps = NULL
/pipeline0/capsfilter2.src: caps = NULL
/pipeline0/capsfilter2.sink: caps = NULL
/pipeline0/d.audio_00: caps = NULL
/pipeline0/d.video_00: caps = NULL
Setting pipeline to NULL ...
FREEING pipeline ...

# ls -l outfile.ts
-rw-r--r-- 1 root root 8036060 Jul 16 10:26 outfile.ts

Same result as [1] before...

[3] After some more googling and experimentation:

# gst-launch-0.10 -v filesrc location=infile.m4v ! qtdemux name=d \
     d.video_00 ! queue ! m.video_00 \
     d.audio_00 ! queue ! m.audio_00 \
     ffmux_mpegts name=m ! filesink location=outfile.ts
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/pipeline0/queue0.sink: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
/pipeline0/queue1.sink: caps = audio/mpeg, mpegversion=(int)4,
framed=(boolean)true, codec_data=(buffer)1290, rate=(int)32000,
channels=(int)2
/pipeline0/queue1.sink: caps = audio/mpeg, mpegversion=(int)4,
framed=(boolean)true, codec_data=(buffer)1290, rate=(int)32000,
channels=(int)2
/pipeline0/m.video_0: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
/pipeline0/queue1.src: caps = audio/mpeg, mpegversion=(int)4,
framed=(boolean)true, codec_data=(buffer)1290, rate=(int)32000,
channels=(int)2
/pipeline0/m.audio_0: caps = audio/mpeg, mpegversion=(int)4,
framed=(boolean)true, codec_data=(buffer)1290, rate=(int)32000,
channels=(int)2
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 189110943 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
/pipeline0/m.audio_0: caps = NULL
/pipeline0/m.video_0: caps = NULL
/pipeline0/queue1.src: caps = NULL
/pipeline0/queue1.sink: caps = NULL
/pipeline0/queue0.src: caps = NULL
/pipeline0/queue0.sink: caps = NULL
/pipeline0/d.audio_00: caps = NULL
/pipeline0/d.video_00: caps = NULL
Setting pipeline to NULL ...
FREEING pipeline ...

# ls -l outfile.ts
-rw-r--r-- 1 root root 6259272 Jul 16 10:34 outfile.ts

File size is better, but same VLC result as in [1]

[4] Switch to ffmpeg demux...

# gst-launch-0.10 -v filesrc location=infile.m4v !
ffdemux_mov_mp4_m4a_3gp_3g2_mj2 name=d \
     d.video_00 ! queue ! m.video_00 \
     d.audio_00 ! queue ! m.audio_00 \
     ffmux_mpegts name=m ! filesink location=outfile.ts
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/pipeline0/queue0.sink: caps = video/x-h264, width=(int)480,
height=(int)320, framerate=(fraction)10/1,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8
/pipeline0/queue1.sink: caps = audio/mpeg, rate=(int)32000,
channels=(int)2, mpegversion=(int)4, codec_data=(buffer)1290
/pipeline0/queue1.sink: caps = audio/mpeg, rate=(int)32000,
channels=(int)2, mpegversion=(int)4, codec_data=(buffer)1290
/pipeline0/m.video_0: caps = video/x-h264, width=(int)480,
height=(int)320, framerate=(fraction)10/1,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8
/pipeline0/queue1.src: caps = audio/mpeg, rate=(int)32000,
channels=(int)2, mpegversion=(int)4, codec_data=(buffer)1290
/pipeline0/m.audio_0: caps = audio/mpeg, rate=(int)32000,
channels=(int)2, mpegversion=(int)4, codec_data=(buffer)1290
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 180962744 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
/pipeline0/m.audio_0: caps = NULL
/pipeline0/m.video_0: caps = NULL
/pipeline0/queue1.src: caps = NULL
/pipeline0/queue1.sink: caps = NULL
/pipeline0/queue0.src: caps = NULL
/pipeline0/queue0.sink: caps = NULL
/pipeline0/d.audio_00: caps = NULL
/pipeline0/d.video_00: caps = NULL
Setting pipeline to NULL ...
FREEING pipeline ...

# ls -l outfile.ts
-rw-r--r-- 1 root root 6259272 Jul 16 10:36 outfile.ts

No difference from [3] (i.e. same VLC result as [1])

[5] Try muxing to MP4 container

# gst-launch-0.10 -v filesrc location=infile.m4v ! qtdemux name=d \
     d.video_00 ! queue ! m.video_0 \
     d.audio_00 ! queue ! m.audio_0 \
     ffmux_mp4 name=m ! filesink location=outfile.mp4
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/pipeline0/queue0.sink: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
/pipeline0/queue1.sink: caps = audio/mpeg, mpegversion=(int)4,
framed=(boolean)true, codec_data=(buffer)1290, rate=(int)32000,
channels=(int)2
/pipeline0/queue1.sink: caps = audio/mpeg, mpegversion=(int)4,
framed=(boolean)true, codec_data=(buffer)1290, rate=(int)32000,
channels=(int)2
/pipeline0/m.video_0: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
/pipeline0/m.video_0: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 200009206 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
/pipeline0/m.audio_0: caps = NULL
/pipeline0/m.video_0: caps = NULL
/pipeline0/queue1.src: caps = NULL
/pipeline0/queue1.sink: caps = NULL
/pipeline0/queue0.src: caps = NULL
/pipeline0/queue0.sink: caps = NULL
/pipeline0/d.audio_00: caps = NULL
/pipeline0/d.video_00: caps = NULL
Setting pipeline to NULL ...
FREEING pipeline ...

# ls -l outfile.mp4
-rw-r--r-- 1 root root 5300514 Jul 16 10:39 outfile.mp4

outfile.mp4 is about the right size and plays back perfectly in VLC.
Crap! So the pipeline is basically ok, just the TS muxing is bad.

[6] Next, I moved to a FC10 machine and yum installed the latest
gstreamer RPMs. This let me use the mpegtsmux module...

# yum list installed '*gstreamer*'
Loaded plugins: refresh-packagekit
Installed Packages
PackageKit-gstreamer-plugin.i386                     0.3.9-4.fc10
installed
gstreamer.i386                                       0.10.21-2.fc10
installed
gstreamer-ffmpeg.i386                                0.10.5-1.fc10
@rpmfusion-free
gstreamer-plugins-bad.i386                           0.10.9-1.fc10
@rpmfusion-free
gstreamer-plugins-base.i386                          0.10.21-2.fc10
installed
gstreamer-plugins-flumpegdemux.i386                  0.10.15-4.fc10
installed
gstreamer-plugins-good.i386                          0.10.13-1.fc10
installed
gstreamer-plugins-ugly.i386                          0.10.10-2.fc10
@rpmfusion-free-updates
gstreamer-python.i386                                0.10.12-1.fc10
installed
gstreamer-tools.i386                                 0.10.21-2.fc10
installed
totem-gstreamer.i386                                 2.24.3-1.fc10
installed

Same infile.m4v:

# ls -l infile.m4v
-rw-r--r-- 1 root root 5300576 2009-07-03 15:23 infile.mv4

# gst-launch-0.10 filesrc location=infile.m4v ! qtdemux name=d \
    d.video_00 ! queue ! m.sink_0 \
    d.audio_00 ! queue ! m.sink_1 \
    mpegtsmux name=m ! filesink location=artbeats.ts
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
*** glibc detected *** gst-launch-0.10: free(): invalid next size
(fast): 0x09b6d400 ***
======= Backtrace: =========
/lib/libc.so.6[0x450394]
/lib/libc.so.6(cfree+0x96)[0x452346]
/lib/libglib-2.0.so.0(g_free+0x36)[0x3238e6]
/usr/lib/libgstreamer-0.10.so.0[0x491810b]
/usr/lib/libgstreamer-0.10.so.0(gst_mini_object_unref+0xb3)[0x49388a3]
/usr/lib/gstreamer-0.10/libgstmpegtsmux.so[0x600c0d]
/usr/lib/gstreamer-0.10/libgstmpegtsmux.so[0x6055e9]
/usr/lib/gstreamer-0.10/libgstmpegtsmux.so[0x605b28]
/usr/lib/gstreamer-0.10/libgstmpegtsmux.so[0x6047eb]
/usr/lib/gstreamer-0.10/libgstmpegtsmux.so[0x60254b]
/usr/lib/libgstbase-0.10.so.0[0x4a17377]
/usr/lib/libgstbase-0.10.so.0[0x4a17f50]
/usr/lib/libgstreamer-0.10.so.0[0x493e7d9]
/usr/lib/libgstreamer-0.10.so.0(gst_pad_push+0x33a)[0x493fa1a]
/usr/lib/gstreamer-0.10/libgstcoreelements.so[0x5b6202]
/usr/lib/libgstreamer-0.10.so.0[0x495ed33]
/lib/libglib-2.0.so.0[0x347346]
/lib/libglib-2.0.so.0[0x345caf]
/lib/libpthread.so.0[0x3cd51f]
/lib/libc.so.6(clone+0x5e)[0x4c501e]
======= Memory map: ========
00110000-00111000 r-xp 00110000 00:00 0          [vdso]
00111000-00115000 r-xp 00000000 09:00 212774
/lib/libgthread-2.0.so.0.1800.4
00115000-00116000 rw-p 00003000 09:00 212774
/lib/libgthread-2.0.so.0.1800.4
00116000-00119000 r-xp 00000000 09:00 212754
/lib/libgmodule-2.0.so.0.1800.4
00119000-0011a000 rw-p 00002000 09:00 212754
/lib/libgmodule-2.0.so.0.1800.4
0011a000-00122000 r-xp 00000000 09:00 214862     /lib/librt-2.9.so
00122000-00123000 r--p 00007000 09:00 214862     /lib/librt-2.9.so
00123000-00124000 rw-p 00008000 09:00 214862     /lib/librt-2.9.so
00124000-00127000 r-xp 00000000 09:00 214866     /lib/libdl-2.9.so
00127000-00128000 r--p 00002000 09:00 214866     /lib/libdl-2.9.so
00128000-00129000 rw-p 00003000 09:00 214866     /lib/libdl-2.9.so
0012f000-0014f000 r-xp 00000000 09:00 214859     /lib/ld-2.9.so
00150000-00151000 r--p 00020000 09:00 214859     /lib/ld-2.9.so
00151000-00152000 rw-p 00021000 09:00 214859     /lib/ld-2.9.so
00152000-0029d000 r-xp 00000000 09:00 409495
/usr/lib/libxml2.so.2.7.3
0029d000-002a2000 rw-p 0014b000 09:00 409495
/usr/lib/libxml2.so.2.7.3
002a2000-002a3000 rw-p 002a2000 00:00 0
002a3000-002e3000 r-xp 00000000 09:00 212770
/lib/libgobject-2.0.so.0.1800.4
002e3000-002e4000 rw-p 00040000 09:00 212770
/lib/libgobject-2.0.so.0.1800.4
002e4000-003c6000 r-xp 00000000 09:00 212750
/lib/libglib-2.0.so.0.1800.4
003c6000-003c7000 rw-p 000e2000 09:00 212750
/lib/libglib-2.0.so.0.1800.4
003c7000-003dd000 r-xp 00000000 09:00 214861     /lib/libpthread-2.9.so
003dd000-003de000 r--p 00015000 09:00 214861     /lib/libpthread-2.9.so
003de000-003df000 rw-p 00016000 09:00 214861     /lib/libpthread-2.9.so
003df000-003e1000 rw-p 003df000 00:00 0
003e1000-0054f000 r-xp 00000000 09:00 214860     /lib/libc-2.9.so
0054f000-00551000 r--p 0016e000 09:00 214860     /lib/libc-2.9.so
00551000-00552000 rw-p 00170000 09:00 214860     /lib/libc-2.9.so
00552000-00555000 rw-p 00552000 00:00 0
00555000-00568000 r-xp 00000000 09:00 214878     /lib/libz.so.1.2.3
00568000-00569000 rw-p 00012000 09:00 214878     /lib/libz.so.1.2.3
00569000-00590000 r-xp 00000000 09:00 214873     /lib/libm-2.9.so
00590000-00591000 r--p 00026000 09:00 214873     /lib/libm-2.9.so
00591000-00592000 rw-p 00027000 09:00 214873     /lib/libm-2.9.so
00592000-0059d000 r-xp 00000000 09:00 212762
/lib/libnss_files-2.9.so
0059d000-0059e000 r--p 0000a000 09:00 212762
/lib/libnss_files-2.9.so
0059e000-0059f000 rw-p 0000b000 09:00 212762
/lib/libnss_files-2.9.so
0059f000-005cb000 r-xp 00000000 09:00 500485
/usr/lib/gstreamer-0.10/libgstcoreelements.so
005cb000-005cc000 rw-p 0002c000 09:00 500485
/usr/lib/gstreamer-0.10/libgstcoreelements.so
005cc000-005ec000 r-xp 00000000 09:00 500757
/usr/lib/gstreamer-0.10/libgstqtdemux.so
005ec000-005ed000 rw-p 00020000 09:00 500757
/usr/lib/gstreamer-0.10/libgstqtdemux.so
005ed000-005fe000 r-xp 00000000 09:00 412179
/usr/lib/libgstrtp-0.10.sAborted

Whoops! So no luck there, either.

I downloaded the latest packages from
http://gstreamer.freedesktop.org/src/ but I couldn't find a new enough
flex package to build on CentOS 5.3 and could not figure out how to get
around the:

configure: No package 'gstreamer-0.10' found
configure: error: no gstreamer-0.10 >= 0.10.23 (GStreamer) found

when running gst-plugins-bad-0.10.13's ./configure on FC10.

Does my pipeline look ok towards the end? Is it looking like a muxing
problem?

Any ideas, suggestions, etc will be warmly received...

Thanks in advance,

ChrisB

--
Chris Burns
chris.burns at abdeus.com