2

under no-exception context, I have seen several posts saying 

Thing* t = new(std::nothrow) Thing; // returns NULL on failure
if (!t) {
  // allocation failure
}

e.g.

How to check memory allocation failures with new operator?

How to find out the return value if my C++ “new” memory allocation failure?

I believe this is the statement they got from C++ standards, as cited:

If the allocation function is declared with a non-throwing exception-specification , it returns null to indicate failure to allocate storage and a non-null pointer otherwise.

However, by Herb Sutter http://www.gotw.ca/publications/mill16.htm

The linux will overcommit memory, and this is not conformant with c++ standards. i.e., checking null is not relevant for linux system. "new" either succeed or fail with process killed by linux.

So what can we do? It seems no way to check the failure.

[UPDATE] about linux overcommit: overcommit_memory

0 — The default setting. The kernel performs heuristic memory overcommit handling by estimating the amount of memory available and failing requests that are blatantly invalid. Unfortunately, since memory is allocated using a heuristic rather than a precise algorithm, this setting can sometimes allow available memory on the system to be overloaded.

Community
  • 1
  • 1
pepero
  • 7,095
  • 7
  • 41
  • 72
  • The linked page from Herb Sutter is from 2001. Are you sure there is no newer documentation? Also I'm a person who says, that if new/malloc fails, you have bigger problems than thinking about how to handle that. Just let the application be killed and that's it. – hellow Jul 21 '15 at 12:53
  • I am not sure, that's why I ask, :-) – pepero Jul 21 '15 at 12:57
  • The Linux OOM killer kills _other_ processes to use their memory for the process allocating it. As the pages note, `malloc` and other allocation functions will never fail with memory overcommitment (unless they do, but then it's handled in the standard way), so there is no 'failure' to check for. – Colonel Thirty Two Jul 21 '15 at 14:11

1 Answers1

1

Linux does not overcommit by default. The real solution is not to configure it so it does.

You can't have your cake and eat it too. If you use linux in its default state (not overcommitting) there is no problem with detecting a memory allocation error. If you configure it to overcommit, then you need to either design your program so it cannot run out of memory (i.e. ensure there is more memory available than your program will ever attempt to allocate) or rearchitect your program so it terminates before a failure will have unwanted consequence (e.g. not destroying objects and saving state to a file).

However ..... if you insist on configuring your system to overcommit, then Herb sort of hints at the solution in that link.

Essentially, it is necessary to reorganise your program so all dynamic memory allocation is done up front AND touched. 

  p = new (nothrow) Thing;
  if (p != NULL)
      touch(p);
  else
      terminate_with_prejudice()

where touch() ensures the memory allocation is committed under linux (Herbs' page describes how to do that).

Alternatively, for throwing new;

  try
  {
        p = new Thing;
        touch(p);
  }
  catch (...)
  {
      terminate_with_prejudice()
  }

If a failure occurs, then the program will terminate abnormally on any system. If this is done before startup, there is no damage done (other than implied by the program not starting).

The difference is how the failure occurs on different systems. With both forms, a linux system will always call touch() and the program will execute abnormally on a failure. On a non-linux system, terminate_with_prejudice() will be called on a failure. Either way, the program will stop executing.

The catching of the exception is optional, but I've done that to make the two code samples equivalent (at least, on any chosen system).

The catch (no pun intended) with this is that it is non-trivial to rearchitect your whole program to avoid asking the host system for memory after startup. Not doing that will mean termination in the same way, but not a consistent guarantee that the program can do any necessary cleanup.

Peter
  • 35,646
  • 4
  • 32
  • 74
  • Peter, thanks for your comments. but as I cited from linux manual, even in the default setting memory available can also be overloaded. also, the "touch" is not the way to detect, it is only to grantee the memory is really allocated, and we will not have surprise when use it. – pepero Jul 21 '15 at 13:33
  • I realise the `touch()` is not a way to detect. My point is that there is no way to detect, and the solution is to design/configure and avoid the concern. That "linux manual" you cited is for a specific distribution - presumably one configured to target performance, even if that means an occasional failure. – Peter Jul 22 '15 at 09:49