python memory leak

[Bram Van Steenlandt]

Hi,

I have an application (rtsp camera viewing and recording) in python 
using gstreamer 1.0.

It all works but I think one (or more) of the pipelines are leaking 
memory, at first I thought the problem was related to the restarting of 
the stream but this doesn't seem to be the case.
It also doesn't happen (less camera and a more complicated pipeline with 
an extra tcpclient) at all on one machine and on another machine (where 
the load is higher) it happens "all the time" I think an external factor 
causes the leak (corrupt jpeg stream, packet loss etc.)
On the machine where it happens "all the time", I think the maximum time 
without leaking is 6 hours I think, it also not every 6 hours, sometimes 
it's less.

I now have a temporary fix where the application restarts when memory is 
running out, this works but it's not ideal.
I have 9 pipelines running, for another project I'm hoping to run 32 or 64.

Is there a way I can:
-Find out how much memory one pipeline is taking up from python ? sizeof 
didn't work
-Tell a pipeline to limit it's memory consumption to a maximum amount 
and "destroy" itself when it fails to do so.

What memory consumption is to be considered normal for streaming 9 
cameras (720p/15fps) ? I saw values of 4GB and up on the problem 
machine, the other machine seems stable at 200MB (only 2 streams).

The pipeline I'm using is along these lines:
rtspsrc name=src location=rtsp://x.x.x.x latency=100 ! rtpjpegdepay !  
tee name=a !  jpegdec ! queue  ! xvimagesink  a. ! queue ! videorate ! 
image/jpeg,framerate=5/1 !  matroskamux ! multifilesink 
aggregate-gops=true next-file=max-size max-file-size=500000000 
location=recordings/tmp/cam9.%02d.mkv

When looking at this, is there any element (matroskamux ?) that isn't 
suppose to be used in this way and may cause problems ?
I thought about replacing matroskamux with multipartmux (I can convert 
them afterwards) but as one installation is running fine with 
matroskamux maybe this isn't the issue
I also checked the trsp connections with netstat and there is only one 
connection to every camera.

rebooting the source (camera's) and thus launching a new stream does not 
free the memory.

Is it at all possible this is infact a hardware issue ? (bad memory?)

thx
Bram