3

I have an embedded Linux system with 256MB of RAM. There is a large codebase running on it, most in C++ but some utilities are in Python. We have a Python "package manager" that handles updates to the system using the Python apt module (we distribute our updates as .deb files). When running, this application uses a large portion of the system RAM. I am monitoring RAM usage with top, looking at the RSS for the Python process (maybe this is not a valid way to analyze process memory usage? Open to suggestions).

The line

    cache = apt.Cache()

which is called periodically to check on the status of the system seems to consume about 20MB each time it is called, and it doesn't look like this memory is being returned. I tried deleting the cache object at the end of the function and running gc.collect(), that didn't seem to help much.

How can I reduce the memory usage of this program?

The platform is ARM Cortex A8, running Linux 3.2, Debian Wheezy and Python 2.7.3.

Charles
  • 50,943
  • 13
  • 104
  • 142
fred basset
  • 9,774
  • 28
  • 88
  • 138

2 Answers2

6

Garbage Collector interface module (gc) is exactly what you could use here to further debug this problem.

To debug a leaking program call:

gc.set_debug(gc.DEBUG_LEAK)

This will output debugging information about garbage collection into your console.

For further debugging, gc module provides more facilities, including ability to list out all objects tracked by the garbage collector, disable/enable collector or manually trigger garbage collection.

If you don't have a leak and want to just lower memory usage, try using gc.set_threshold to trigger garbage collection at a lower memory usage point.

jsalonen
  • 29,593
  • 15
  • 91
  • 109
  • Thank you, I was unaware of that function. Is analyzing the memory usage using "top" actually a valid way to measure memory usage? – fred basset Jul 12 '13 at 21:30
  • `top` provides you only with a simple metric on how much memory is currently allocated. It does not directly tell you whether there is a leak or not. – jsalonen Jul 12 '13 at 21:33
  • Btw. I'm suspecting that what you are experiencing here is just the normal way how garbage collection works. You can easily test if this is the case by explicitly calling `gc.collect()`. If that frees up the memory, there is no leak. – jsalonen Jul 12 '13 at 21:35
  • 1
    you'd need **atop**, that caches very quick processes, that **htop** (and top etc) would miss. – GitaarLAB Jul 12 '13 at 21:35
  • I'll check out atop. I actually did put a gc.collect() call in and the end of the function in question and it didn't seem to make a difference. – fred basset Jul 12 '13 at 21:46
  • I am seeing quite a lot of output on stdout since I added the gc.set_debug(gc.DEBUG_LEAK) call. What can I actually do about those leaky objects? – fred basset Jul 12 '13 at 21:54
  • 1
    See lot of great answers here: https://stackoverflow.com/questions/1435415/python-memory-leaks – Shital Shah Feb 26 '19 at 02:30
0

The clean command is used to free up the disk space by cleaning retrieved (downloaded) .deb files (packages) from the local repository.

$ sudo apt-get clean

Hope this helps!

GitaarLAB
  • 14,536
  • 11
  • 60
  • 80
  • alternatively (to keep signature related viruses alive) use `apt-get clean all` OR, to start new ones: `apt-get clean my junk` :-) [src](http://askubuntu.com/questions/144222/how-do-apt-get-clean-and-apt-get-clean-all-differ) – GitaarLAB Jul 12 '13 at 21:33