[gst-devel] Gst-python object cleanup

[Ron McOuat]

Thanks Edward, the links helped a lot. I do have a signal watch and was 
removing it in the message callback once the EOS arrived on the filesink 
element bus which was inserted in order to detect EOS. I was able to 
determine the Python object count is not increasing using some gc calls 
from the article referred to by the second link you gave me. Moving the 
application to Karmic from Jaunty and re-arranging some of the callback 
code has improved the memory use rate. The old record bin cleanup was in 
the EOS message callback and the new record bin was allocated right 
after injecting the EOS into the old record bin so there was a race 
between the two. The old and new bins had different class variable 
references bound to keep them apart so I am reasonably sure I wasn't 
stomping on the wrong things with a shared reference. I now create the 
new record bin in the EOS callback after the old one is cleaned up 
making the operations serial. There is a queue in front to accept frames 
while this is taking place so the front end doesn't lose data or block.

Watching the python process with gnome-system-monitor it seems to jump 
up about a MByte once in a while and not in a monotonic once per cycle 
way. It is like the Python arena gets fragmented somehow and so the 
virtual machine grabs a new chunk to work with - highly speculative but 
pymalloc front ends system malloc allocating large chucks. To put it in 
perspective, the stress testing cycles the application through about 2 
months of work every 24 hours of test run time, with the latest changes 
memory consumption will probably be acceptable.

BTW I would like to say thanks for gstreamer, it is a great multimedia 
processing library and is improving steadily. The Python bindings make 
it even easier to work with - I am relatively new to the language, it is 
so clean and concise - I come from Java since 2000, C, C++ since 1983 
and Fortran/Assembler since 1972.

Ron

Edward Hervey wrote:
>   The description of how you get rid of your pipelines seems correct.
>
>   There's only one other thing that *could* go wrong, is if you're using
> a signal watch on the bus, you should remove it (i.e.
> gst.Bus.remove_signal_watch()).
>
>   If you've made sure that's correct, I'd then suggest to start tracing
> the leaks at the python level. There's a couple of good articles/tools
> to do that:
>   http://www.lshift.net/blog/2008/11/14/tracing-python-memory-leaks
>   http://mg.pov.lt/blog/hunting-python-memleaks
>   And you can also have a look at how debugging memory leaks is done in
> the gst-python testsuite (gst-python/tests/common.py)
>
>     Edward
>
>   
>> Thanks for any suggestion.
>>
>> Ron McOuat
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
>> is the only developer event you need to attend this year. Jumpstart your
>> developing skills, take BlackBerry mobile applications to market and stay 
>> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
>> http://p.sf.net/sfu/devconference
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>>     
>
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay 
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> gstreamer-devel mailing list
> gstreamer-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>
>