[gst-devel] reducing memory usage in avidemux 8and other demuxers)
Stefan Kost
ensonic at hora-obscura.de
Fri Feb 9 12:39:03 CET 2007
hi,
avidemux currently sucks quite a lot. One example, I have an avi that
is 300 mb. Unfortunately it has 446774 index entries. One index entry
uses 52 bytes. Therefore the index needs 23232248 bytes, that is 22.15
Mb. I repacked that structure a bit and saved 4 bytes. This reduces
memory consumtion to 21445152 20.45 Mb. Now the bigger problem is that
the index is first generated as a list and then copied into an array.
Thus the peak memory consumtion even doubles. I currently check if we
can avoid that at all.
Beside that I have some more ideas. This is the current structure:
typedef struct {
guint index_nr; //
=(entry-index_entries)/sizeof(gst_avi_index_entry);
guchar stream_nr;
guchar flags;
guint64 ts;
guint64 dur; // =entry[1].ts-entry->ts
guint64 offset;
guint64 bytes_before; // calculated
guint32 frames_before; // calculated
guint32 size; // could be read from the chunk (if we don't split)
} gst_avi_index_entry;
as we always know the baseaddress of the indexentries, we could drop
index_nr and calculate it. same for duration (would require to append
a dummy entry). If we decide to add a configure switch to gstreamer
core like e.g. --enable-memory-optimization, we could define
conditional macros:
#ifdef GST_ENABLE_MEMORY_OPTIMIZATION
#define GST_AVI_INDEX_ENTRY_NR(entry) \
(entry-index_entries)/sizeof(gst_avi_index_entry)
#else
#define GST_AVI_INDEX_ENTRY_NR(entry) \
entry->index_nr
#endif
In the memory optimization case that would save 10 bytes per entry.
If the target platform does not support files>4GB (#ifndef
__USE_LARGEFILE64) offset, bytes_before can be defined as guint32.
That would save another 8 bytes.
In the avove example this would reduce the memory consumtion to 12.78 Mb.
Finally I notice that we use g_malloc and g_new. Imho it would be
better, that whenever we read stuff from media and according to that
do mallocs, to use g_try_new and g_try_mallo and handle oom situation.
This does not guarantee that the app gets not terminated because of
oom, but makes it a lot less like likely.
Any comments? Is there simillar stuff like the gst_avi_index_entry for
other demuxers like ogg, mpeg?
Stefan
More information about the gstreamer-devel
mailing list