Wednesday, June 16, 2010

memory profiling with python

tough to find good memory profiling for python. heapy-pe and the other (didn't bother to remember the name; pysizer?) turned out to be no good to me with numpy arrays which (surprise!) tend to be the biggest data structures i deal with. here a couple of others to try some time:
meliae is new and more cli-oriented, but looks easy enough to try (and script).
dowser spawned off of cherrypy, but i think it works for any python code with the web server as sort of a gui (i think unlike dozer, which targets wsgi apps. or maybe dozer is just a wsgi version of the 'gui'?).

here's an example of objgraph to analyse memory usage.
i think these are more garbage collector approaches, rather than hook-and-trace, so maybe more likely to work with libs like numpy.

memory_profiler also comes recommended and looks interesting. pure python, so portable and hackable.

No comments: