[gst-devel] reducing memory usage in avidemux 8and other demuxers)

Stefan Kost ensonic at hora-obscura.de
Fri Feb 9 12:39:03 CET 2007


avidemux currently sucks quite a lot. One example, I have an avi that  
is 300 mb. Unfortunately it has 446774 index entries. One index entry  
uses 52 bytes. Therefore the index needs 23232248 bytes, that is 22.15  
Mb. I repacked that structure a bit and saved 4 bytes. This reduces  
memory consumtion to 21445152 20.45 Mb. Now the bigger problem is that  
the index is first generated as a list and then copied into an array.  
Thus the peak memory consumtion even doubles. I currently check if we  
can avoid that at all.

Beside that I have some more ideas. This is the current structure:
typedef struct {
   guint   index_nr;      //  
   guchar  stream_nr;
   guchar  flags;
   guint64 ts;
   guint64 dur;           // =entry[1].ts-entry->ts
   guint64 offset;
   guint64 bytes_before;  // calculated
   guint32 frames_before; // calculated
   guint32 size;          // could be read from the chunk (if we don't split)
} gst_avi_index_entry;

as we always know the baseaddress of the indexentries, we could drop  
index_nr and calculate it. same for duration (would require to append  
a dummy entry). If we decide to add a configure switch to gstreamer  
core like e.g. --enable-memory-optimization, we could define  
conditional macros:
#define GST_AVI_INDEX_ENTRY_NR(entry) \
#define GST_AVI_INDEX_ENTRY_NR(entry) \
In the memory optimization case that would save 10 bytes per entry.

If the target platform does not support files>4GB (#ifndef  
__USE_LARGEFILE64) offset, bytes_before can be defined as guint32.  
That would save another 8 bytes.

In the avove example this would reduce the memory consumtion to 12.78 Mb.

Finally I notice that we use g_malloc and g_new. Imho it would be  
better, that whenever we read stuff from media and according to that  
do mallocs, to use g_try_new and g_try_mallo and handle oom situation.  
This does not guarantee that the app gets not terminated because of  
oom, but makes it a lot less like likely.

Any comments? Is there simillar stuff like the gst_avi_index_entry for  
other demuxers like ogg, mpeg?


More information about the gstreamer-devel mailing list