How to get available memory C++/g++?

Question

I want to allocate my buffers according to memory available. Such that, when I do processing and memory usage goes up, but still remains in available memory limits. Is there a way to get available memory (I don't know will virtual or physical memory status will make any difference ?). Method has to be platform Independent as its going to be used on Windows, OS X, Linux and AIX. (And if possible then I would also like to allocate some of available memory for my application, someone it doesn't change during the execution).

Edit: I did it with configurable memory allocation. I understand it is not good idea, as most OS manage memory for us, but my application was an ETL framework (intended to be used on server, but was also being used on desktop as a plugin for Adobe indesign). So, I was running in to issue of because instead of using swap, windows would return bad alloc and other applications start to fail. And as I was taught to avoid crashes and so, was just trying to degrade gracefully.

Answer 1

On UNIX-like operating systems, there is sysconf.

#include <unistd.h>

unsigned long long getTotalSystemMemory()
{
    long pages = sysconf(_SC_PHYS_PAGES);
    long page_size = sysconf(_SC_PAGE_SIZE);
    return pages * page_size;
}

On Windows, there is GlobalMemoryStatusEx:

#include <windows.h>

unsigned long long getTotalSystemMemory()
{
    MEMORYSTATUSEX status;
    status.dwLength = sizeof(status);
    GlobalMemoryStatusEx(&status);
    return status.ullTotalPhys;
}

So just do some fancy #ifdefs and you'll be good to go.

Answer 2

There are reasons to do want to do this in HPC for scientific software. (Not game, web, business or embedded software). Scientific software routinely go through terabytes of data to get through one computation (or run) (and run for hours or weeks) -- all of which cannot be stored in memory (and if one day you tell me a terabyte is standard for any PC or tablet or phone it will be the case that the scientific software will be expected to handle petabytes or more). The amount of memory can also dictate the kind of method/algorithm that makes sense. The user does not always want to decide the memory and method - he/she has other things to worry about. So the programmer should have a good idea of what is available (4Gb or 8Gb or 64Gb or thereabouts these days) to decide whether a method will automatically work or a more laborious method is to be chosen. Disk is used but memory is preferable. And users of such software are not encouraged to be doing too many things on their computer when running such software -- in fact, they often use dedicated machines/servers.

Answer 3

There is no platform independent way to do this, different operating systems use different memory management strategies.

These other stack overflow questions will help:

• How to get memory usage at run time in c++?
• C/C++ memory usage API in Linux/Windows

You should watch out though: It is notoriously difficult to get a "real" value for available memory in linux. What the operating system displays as used by a process is no guarantee of what is actually allocated for the process.

This is a common issue when developing embedded linux systems such as routers, where you want to buffer as much as the hardware allows. Here is a link to an example showing how to get this information in a linux (in C):

• http://www.unix.com/programming/25035-determining-free-available-memory-mv-linux.html

Answer 4

Having read through these answers I'm astonished that so many take the stance that OP's computer memory belongs to others. It's his computer and his memory to do with as he sees fit, even if it breaks other systems taking a claim it. It's an interesting question. On a more primitive system I had memavail() which would tell me this. Why shouldn't the OP take as much memory as he wants without upsetting other systems?

Here's a solution that allocates less than half the memory available, just to be kind. Output was:

Required FFFFFFFF

Required 7FFFFFFF

Required 3FFFFFFF

Memory size allocated = 1FFFFFFF

#include <stdio.h>
#include <stdlib.h>

#define MINREQ      0xFFF   // arbitrary minimum

int main(void)
{
    unsigned int required = (unsigned int)-1; // adapt to native uint
    char *mem = NULL; 
    while (mem == NULL) {
        printf ("Required %X\n", required);
        mem = malloc (required);
        if ((required >>= 1) < MINREQ) {
            if (mem) free (mem);
            printf ("Cannot allocate enough memory\n");
            return (1);
        }
    }

    free (mem);
    mem = malloc (required);
    if (mem == NULL) {
        printf ("Cannot enough allocate memory\n");
        return (1);
    }
    printf ("Memory size allocated = %X\n", required);
    free (mem);
    return 0;
}

Answer 5

Mac OS X example using sysctl (man 3 sysctl):

#include <stdio.h>
#include <stdint.h>
#include <sys/types.h>
#include <sys/sysctl.h>

int main(void)
{
    int mib[2] = { CTL_HW, HW_MEMSIZE };
    u_int namelen = sizeof(mib) / sizeof(mib[0]);
    uint64_t size;
    size_t len = sizeof(size);

    if (sysctl(mib, namelen, &size, &len, NULL, 0) < 0)
    {
        perror("sysctl");
    }
    else
    {
        printf("HW.HW_MEMSIZE = %llu bytes\n", size);
    }
    return 0;
}

(may also work on other BSD-like operating systems ?)

Answer 6

The code below gives the total and free memory in Megabytes. Works for FreeBSD, but you should be able to use same/similar sysctl tunables on your platform and do to the same thing (Linux & OS X have sysctl at least)

#include <stdio.h>
#include <errno.h>

#include <sys/types.h>
#include <sys/sysctl.h>
#include <sys/vmmeter.h>

int main(){
    int rc;
    u_int page_size;
    struct vmtotal vmt;
    size_t vmt_size, uint_size; 

    vmt_size = sizeof(vmt);
    uint_size = sizeof(page_size);

    rc = sysctlbyname("vm.vmtotal", &vmt, &vmt_size, NULL, 0);
    if (rc < 0){
       perror("sysctlbyname");
       return 1;
    }

    rc = sysctlbyname("vm.stats.vm.v_page_size", &page_size, &uint_size, NULL, 0);
    if (rc < 0){
       perror("sysctlbyname");
       return 1;
    }

    printf("Free memory       : %ld\n", vmt.t_free * (u_int64_t)page_size);
    printf("Available memory  : %ld\n", vmt.t_avm * (u_int64_t)page_size);

    return 0;
}

Below is the output of the program, compared with the vmstat(8) output on my system.

~/code/memstats % cc memstats.c 
~/code/memstats % ./a.out 
Free memory       : 5481914368
Available memory  : 8473378816
~/code/memstats % vmstat 
 procs      memory      page                    disks     faults         cpu
 r b w     avm    fre   flt  re  pi  po    fr  sr ad0 ad1   in   sy   cs us sy id
 0 0 0   8093M  5228M   287   0   1   0   304 133   0   0  112 9597 1652  2  1 97

Answer 7

Linux currently free memory: sysconf(_SC_AVPHYS_PAGES) and get_avphys_pages()

The total RAM was covered at https://stackoverflow.com/a/2513561/895245 with sysconf(_SC_PHYS_PAGES);.

Both sysconf(_SC_AVPHYS_PAGES) and get_avphys_pages() are glibc extensions to POSIX that give instead the total currently available RAM pages.

You then just have to multiply them by sysconf(_SC_PAGE_SIZE) to obtain the current free RAM.

Minimal runnable example at: C - Check available free RAM?

Answer 8

The "official" function for this is was std::get_temporary_buffer(). However, you might want to test whether your platform has a decent implemenation. I understand that not all platforms behave as desired.

Answer 9

Instead of trying to guess, have you considered letting the user configure how much memory to use for buffers, as well as assuming somewhat conservative defaults? This way you can still run (possibly slightly slower) with no override, but if the user know there is X memory available for the app they can improve performance by configuring that amount.

Answer 10

Here is a proposal to get available memory on Linux platform:

/// Provides the available RAM memory in kibibytes (1 KiB = 1024 B) on Linux platform (Available memory in /proc/meminfo)
/// For more info about /proc/meminfo : https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/deployment_guide/s2-proc-meminfo
long long getAvailableMemory()
{
  long long memAvailable = -1;
  std::ifstream meminfo("/proc/meminfo");
  std::string line;
  while (std::getline(meminfo, line))
  {
    if (line.find("MemAvailable:") != std::string::npos)
    {
      const std::size_t firstWhiteSpacePos = line.find_first_of(' ');
      const std::size_t firstNonWhiteSpaceChar = line.find_first_not_of(' ', firstWhiteSpacePos);
      const std::size_t nextWhiteSpace = line.find_first_of(' ', firstNonWhiteSpaceChar);
      const std::size_t numChars = nextWhiteSpace - firstNonWhiteSpaceChar;
      const std::string memAvailableStr = line.substr(firstNonWhiteSpaceChar, numChars);
      memAvailable = std::stoll(memAvailableStr);
      break;
    }
  }

  return memAvailable;
}