Determine free RAM in Python

Question

I would like my python script to use all the free RAM available but no more (for efficiency reasons). I can control this by reading in only a limited amount of data but I need to know how much RAM is free at run-time to get this right. It will be run on a variety of Linux systems. Is it possible to determine the free RAM at run-time?

Answer 1

On Linux systems I use this from time to time:

def memory():
    """
    Get node total memory and memory usage
    """
    with open('/proc/meminfo', 'r') as mem:
        ret = {}
        tmp = 0
        for i in mem:
            sline = i.split()
            if str(sline[0]) == 'MemTotal:':
                ret['total'] = int(sline[1])
            elif str(sline[0]) in ('MemFree:', 'Buffers:', 'Cached:'):
                tmp += int(sline[1])
        ret['free'] = tmp
        ret['used'] = int(ret['total']) - int(ret['free'])
    return ret

You can run this when your script starts up. RAM is usually used and freed pretty frequently on a busy system, so you should take that into account before deciding how much RAM to use. Also, most linux systems have a swappiness value of 60. When using up memory, pages that are least frequently used will be swapped out. You may find yourself using SWAP instead of RAM.

Hope this helps.

Answer 2

Another option is the Python package psutil:

stats = psutil.virtual_memory()  # returns a named tuple
available = getattr(stats, 'available')
print(available)

According to the documentation, the available field "is calculated by summing different memory values depending on the platform and it is supposed to be used to monitor actual memory usage in a cross platform fashion."

Note the return value will be in bytes

Answer 3

You could just read out /proc/meminfo. Be aware that the "free memory" is usually quite low, as the OS heavily uses free, unused memory for caching.

Also, it's best if you don't try to outsmart your OS's memory management. That usually just ends in tears (or slower programs). Better just take the RAM you need. If you want to use as much as you can on a machine with a previously unknown amount of memory, I'd probably check how much RAM is installed (MemTotal in /proc/meminfo), leave a certain amount for the OS and as safety margin (say 1 GB) and use the rest.

Answer 4

Since the original poster wrote the code should run on a variety of Linux system, I am posting here an Object oriented solution for Linux Memory query. psutil is a great library, but if you can't install it for some reason, you can simply use the following solution:

Example usage:

>>> f = FreeMemLinux()
>>> print f.total, f.used,  f.user_free
8029212 3765960 4464816
>>>
>>> f_mb = FreeMemLinux(unit='MB')
>>> print f_mb.total, f_mb.used,  f_mb.user_free
7841.02734375 3677.6953125 4360.171875
>>>
>>> f_percent = FreeMemLinux(unit='%')
>>> print f_percent.total, f_percent.used, f_percent.user_free
100.0 46.9032328453 55.60715049

Code:

class FreeMemLinux(object):
    """
    Non-cross platform way to get free memory on Linux. Note that this code
    uses the `with ... as`, which is conditionally Python 2.5 compatible!
    If for some reason you still have Python 2.5 on your system add in the
head of your code, before all imports:
    from __future__ import with_statement
    """

    def __init__(self, unit='kB'):

        with open('/proc/meminfo', 'r') as mem:
            lines = mem.readlines()

        self._tot = int(lines[0].split()[1])
        self._free = int(lines[1].split()[1])
        self._buff = int(lines[2].split()[1])
        self._cached = int(lines[3].split()[1])
        self._shared = int(lines[20].split()[1])
        self._swapt = int(lines[14].split()[1])
        self._swapf = int(lines[15].split()[1])
        self._swapu = self._swapt - self._swapf

        self.unit = unit
        self._convert = self._factor()

    def _factor(self):
        """determine the convertion factor"""
        if self.unit == 'kB':
            return 1
        if self.unit == 'k':
            return 1024.0
        if self.unit == 'MB':
            return 1/1024.0
        if self.unit == 'GB':
            return 1/1024.0/1024.0
        if self.unit == '%':
            return 1.0/self._tot
        else:
            raise Exception("Unit not understood")

    @property
    def total(self):
        return self._convert * self._tot

    @property
    def used(self):
        return self._convert * (self._tot - self._free)

    @property
    def used_real(self):
        """memory used which is not cache or buffers"""
        return self._convert * (self._tot - self._free -
                                self._buff - self._cached)

    @property
    def shared(self):
        return self._convert * (self._tot - self._free)

    @property
    def buffers(self):
        return self._convert * (self._buff)

    @property
    def cached(self):
        return self._convert * self._cached

    @property
    def user_free(self):
        """This is the free memory available for the user"""
        return self._convert *(self._free + self._buff + self._cached)

    @property
    def swap(self):
        return self._convert * self._swapt

    @property
    def swap_free(self):
        return self._convert * self._swapf

    @property
    def swap_used(self):
        return self._convert * self._swapu

Answer 5

I suppose you could use free (http://www.cyberciti.biz/faq/linux-check-memory-usage/), ps or maybe the MemoryMonitor class from the SO-thread posted at the very bottom of my answer to do this. Just cut some slack and leave some small amount of ram unused for other processes if they should need it urgently yo avoid disk writes on their behalf.

You will need to parse the output from free or ps if you use that, but that shouldn't be hard. Remember that you need to analyze available ram real time, so you can adjust if another process gets memory hungry for some reason.

also see this thread: How to get current CPU and RAM usage in Python?