Memory usage seems to be proportional to file size when playing m4a files.
Lei Miao
leimiao09 at gmail.com
Fri Jan 17 14:58:21 PST 2014
After reading Apple's QuickTime File Format Specification and digging
into gstreamer's code, I figured out what caused my problem.
I should first point it out that when playing m4a files, the memory
usage of gstreamer is proportional to the time duration of the file, not
the file size.
QuickTime stores media data in samples. The number of a samples in m4a
files are huge. For example, a 6hr m4a file has about 950,000 samples.
Because qtdemux.c creates a sample table whose size is proportional to
the number of samples, the buffer could be very big. For an m4a file
6:11:46 long, I got:
"qtdemux qtdemux.c:6306:qtdemux_stbl_init:<qtdemux0> allocating
n_samples 960669 * 32 (29.32 MB)"
Element qtdemux has a cap of 50MB for this table:
/* if the sample index is larger than this, something is likely wrong */
#define QTDEMUX_MAX_SAMPLE_INDEX_SIZE (50*1024*1024)
The current design has two problems for m4a files:
1. m4a files with duration greater than 12hrs won't play:
qtdemux qtdemux.c:6306:qtdemux_stbl_init:<qtdemux0> allocating
n_samples 1921335 * 32 (58.63 MB)
qtdemux qtdemux.c:6312:qtdemux_stbl_init:<qtdemux0> not allocating
index of 1921335 samples, would be larger than 50MB (broken file?)
2. It may impose a memory burden for embedded systems.
Could somebody take a look at this issue and improve qtdemux? Any
comments/suggestions are appreciated.
LM
On 1/16/2014 2:13 PM, Lei Miao wrote:
> Hi there,
>
> We are using gstreamer in embedded Linux environment, and its memory
> usage exceeds our limit when playing large m4a audio files. After
> investigation, it seems that the memory usage is somewhat proportional
> to the size of the m4a file. I also did some experiments in desktop
> Linux environment, and the results were similar. In particular, I
> first tried a 87MB file, and gstreamer used about 16MB memory:
>
> "lei at lei-OptiPlex-390:/media/LEI$ gst-launch-0.10 playbin
> uri=file:///media/LEI/White_Noise_320kbps_1Hour.m4a &
> [1] 23790
> lei at lei-OptiPlex-390:/media/LEI$ Setting pipeline to PAUSED ...
> Pipeline is PREROLLING ...
> Pipeline is PREROLLED ...
> Setting pipeline to PLAYING ...
> New clock: GstPulseSinkClock
> lei at lei-OptiPlex-390:/media/LEI$ cat /proc/23790/statm
> 32565 4003 1328 7 0 11705 0
> "
>
> I then tried a 609MB file, and gstreamer used over 50MB memory:
>
> "lei at lei-OptiPlex-390:/media/LEI$ killall gst-launch-0.10
> lei at lei-OptiPlex-390:/media/LEI$ gst-launch-0.10 playbin
> uri=file:///media/LEI/White_Noise_320kbps_7Hour.m4a &
> [2] 23826
> [1] Terminated gst-launch-0.10 playbin
> uri=file:///media/LEI/White_Noise_320kbps_1Hour.m4a
> lei at lei-OptiPlex-390:/media/LEI$ Setting pipeline to PAUSED ...
> Pipeline is PREROLLING ...
> Pipeline is PREROLLED ...
> Setting pipeline to PLAYING ...
> New clock: GstPulseSinkClock
> lei at lei-OptiPlex-390:/media/LEI$
> lei at lei-OptiPlex-390:/media/LEI$ cat /proc/23826/statm
> 41648 13086 1328 7 0 20788 0
> lei at lei-OptiPlex-390:/media/LEI$ cat /proc/23826/statm
> 41648 13086 1328 7 0 20788 0
> "
>
> I also tried pretty recent gstreamer 1.2 and various aac decoder
> elements. Results were the same. Below are the stats provided by
> tracelib. It looks like some of the pads had buffers whose size is
> proportional to the file size.
>
> "lei at lei-OptiPlex-390:~/Documents/gst-tracelib$
> LD_PRELOAD=src/.libs/libgsttracelib.so gst-launch-0.10 playbin
> uri=file:///media/LEI/White_Noise_320kbps_7Hour.m4a
> Setting pipeline to PAUSED ...
> Pipeline is PREROLLING ...
> Pipeline is PREROLLED ...
> Setting pipeline to PLAYING ...
> New clock: GstPulseSinkClock
> ^CCaught interrupt -- handling interrupt.
> Interrupt: Stopping pipeline ...
> Execution ended after 5256616552 ns.
> Setting pipeline to PAUSED ...
> Setting pipeline to READY ...
> Setting pipeline to NULL ...
> Freeing pipeline ...
>
> Overall Statistics:
> Number of Threads: 5
> Number of Elements: 17
> Number of Bins: 4
> Number of Pads: 31
> Number of GhostPads: 4
> Number of Buffers passed: 7068
> Number of Events sent: 82
> Number of Message sent: 16
> Number of Queries sent: 0
> Time: 0:00:05.371006122
> Avg/Max CPU load: 8 %, 58 %
> Max QOS ratio: 0 %
> Max Memory usage: 1562416 kb
>
> Thread 0xb2e03ac0 Statistics:
> Time: 0:00:00.102057797, 1 %
> Avg/Max CPU load: 0 %, 58 %
> Pad Statistics:
> Thread 0x84b8b50 Statistics:
> Time: 0:00:00.186786480, 3 %
> Avg/Max CPU load: 0 %, 0 %
> Pad Statistics:
> < faad0_sink : buffers 364 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) 353/ 559/ 598,
> time 0:00:05.198284149, bytes/sec 39142.916041
> > decodebin0_src0 : buffers 363 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) ......./ 4096/.......,
> time 0:00:05.194604287, bytes/sec 286229.309848
> > faad0_src : buffers 363 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) ......./ 4096/.......,
> time 0:00:05.194604287, bytes/sec 286229.309848
> > selector_audio_src0_src : buffers 363 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) ......./ 4096/.......,
> time 0:00:05.194606439, bytes/sec 286229.191270
> < preroll_audio_src0_sink : buffers 363 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) ......./ 4096/.......,
> time 0:00:05.194606439, bytes/sec 286229.191270
> > preroll_audio_src0_src : buffers 232 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) ......./ 4096/.......,
> time 0:00:05.193298109, bytes/sec 182980.445192
> < abin_sink : buffers 232 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) ......./ 4096/.......,
> time 0:00:05.193298109, bytes/sec 182980.445192
> > aconv_src : buffers 232 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) ......./ 4096/.......,
> time 0:00:05.192087403, bytes/sec 183023.113103
> < aresample_sink : buffers 232 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) ......./ 4096/.......,
> time 0:00:05.192087403, bytes/sec 183023.113103
> > aresample_src : buffers 232 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) ......./ 4096/.......,
> time 0:00:05.191095777, bytes/sec 183058.074985
> < volume_sink : buffers 232 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) ......./ 4096/.......,
> time 0:00:05.191095777, bytes/sec 183058.074985
> > volume_src : buffers 232 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) ......./ 4096/.......,
> time 0:00:05.189524125, bytes/sec 183113.514286
> < audiosink_sink : buffers 232 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) ......./ 4096/.......,
> time 0:00:05.189524125, bytes/sec 183113.514286
> Thread 0x85d7ea0 Statistics:
> Time: 0:00:00.037589903, 0 %
> Avg/Max CPU load: 0 %, 7 %
> Pad Statistics:
> < decodebin0_sink : buffers 386 (ro 0,pre 0,dis
> 0,gap 0,dlt 0), size (min/avg/max) 16/ 22973/8682999,
> time 0:00:05.258568138, bytes/sec 1686310.373335
> < typefind_sink : buffers 386 (ro 0,pre 0,dis
> 0,gap 0,dlt 0), size (min/avg/max) 16/ 22973/8682999,
> time 0:00:05.258568138, bytes/sec 1686310.373335
> > typefind_src : buffers 386 (ro 0,pre 0,dis
> 0,gap 0,dlt 0), size (min/avg/max) 16/ 22973/8682999,
> time 0:00:05.258568513, bytes/sec 1686310.253081
> < qtdemux0_sink : buffers 386 (ro 0,pre 0,dis
> 0,gap 0,dlt 0), size (min/avg/max) 16/ 22973/8682999,
> time 0:00:05.258568513, bytes/sec 1686310.253081
> Thread 0x84b88f0 Statistics:
> Time: 0:00:00.118386171, 2 %
> Avg/Max CPU load: 0 %, 0 %
> Pad Statistics:
> > qtdemux0_audio_00 : buffers 380 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) 353/ 559/ 598,
> time 0:00:05.230470559, bytes/sec 40612.024789
> < queue0_sink : buffers 380 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) 353/ 559/ 598,
> time 0:00:05.230470559, bytes/sec 40612.024789
> > queue0_src : buffers 364 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) 353/ 559/ 598,
> time 0:00:05.230114535, bytes/sec 38904.692935
> < aacparse0_sink : buffers 364 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) 353/ 559/ 598,
> time 0:00:05.230114535, bytes/sec 38904.692935
> > aacparse0_src : buffers 364 (ro 0,pre 0,dis
> 1,gap 0,dlt 0), size (min/avg/max) 353/ 559/ 598,
> time 0:00:05.198284149, bytes/sec 39142.916041
> Thread 0xad603e00 Statistics:
> Time: 0:00:00.020254538, 0 %
> Avg/Max CPU load: 0 %, 0 %
> Pad Statistics:
>
> Element Statistics:
> GstTypeFindElement:typefind : buffers in/out
> 386/ 386 bytes in/out 8903581/ 8903581 first activity
> 0:00:00.023557397, ev/msg/qry sent 1/ 0/ 0
> GstQTDemux:qtdemux0 : buffers in/out
> 386/ 380 bytes in/out 8903581/ 220494 first activity
> 0:00:00.023603848, ev/msg/qry sent 3/ 2/ 0
> GstQueue:queue0 : buffers in/out
> 380/ 364 bytes in/out 220494/ 211195 first activity
> 0:00:00.051741935, ev/msg/qry sent 4/ 0/ 0
> GstBaseParse:aacparse0 : buffers in/out
> 364/ 364 bytes in/out 211195/ 211195 first activity
> 0:00:00.051859689, ev/msg/qry sent 3/ 0/ 0
> GstAudioDecoder:faad0 : buffers in/out
> 364/ 363 bytes in/out 211195/ 1486848 first activity
> 0:00:00.083756439, ev/msg/qry sent 11/ 0/ 0
> GstStreamSelector:selector_audio_src0 : buffers in/out
> -/ 363 bytes in/out -/ 1486848 first activity
> 0:00:00.087758125, ev/msg/qry sent 11/ 0/ 0
> GstQueue:preroll_audio_src0 : buffers in/out
> 363/ 232 bytes in/out 1486848/ 950272 first activity
> 0:00:00.087758126, ev/msg/qry sent 12/ 0/ 0
> GstBaseTransform:aconv : buffers in/out
> -/ 232 bytes in/out -/ 950272 first activity
> 0:00:00.089136064, ev/msg/qry sent 12/ 0/ 0
> GstBaseTransform:aresample : buffers in/out
> 232/ 232 bytes in/out 950272/ 950272 first activity
> 0:00:00.089136065, ev/msg/qry sent 12/ 0/ 0
> GstBaseTransform:volume : buffers in/out
> 232/ 232 bytes in/out 950272/ 950272 first activity
> 0:00:00.090205748, ev/msg/qry sent 12/ 0/ 0
>
> Bin Statistics:
> GstDecodeBin:decodebin0 : buffers in/out
> 363/ 386 bytes in/out 1486848/ 8903581 first activity
> 0:00:00.023557396, ev/msg/qry sent 22/ 5/ 0
> GstPlayBin:playbin0 : buffers in/out
> -/ - bytes in/out -/ - first activity
> 0:00:00.023557396, ev/msg/qry sent 82/ 16/ 0
> GstBin:abin : buffers in/out
> 232/ - bytes in/out 950272/ - first activity
> 0:00:00.087849537, ev/msg/qry sent 37/ 7/ 0
> GstAutoAudioSink:audiosink : buffers in/out
> 232/ - bytes in/out 950272/ - first activity
> 0:00:00.091847714, ev/msg/qry sent 1/
> "
>
> Does anybody have any insights on what might be happening? The files I
> used are available at
> https://dl.dropboxusercontent.com/u/54923483/White_Noise_320kbps_1Hour.m4a
> and
> https://dl.dropboxusercontent.com/u/54923483/White_Noise_320kbps_7Hour.m4a
> .
>
> Thanks,
>
> LM
More information about the gstreamer-devel
mailing list