4

I have a large application, that allocates large amounts of memory using malloc and mmap I want to trap all methods of failure and try and recover. To cover the case of running out of swap space, I check returns of malloc and realloc and if they're null, the application can prompt the user to clear some disk space before continuing.

The application also allocates many large data arrays using mmap, using sparse files. I wish to be able to recover from situations where a write to mapped memory fails due to disk space limitations. Is there any way of doing this, or am I better off incorporating disk space monitoring and halting the relevant threads in this scenario.

Im working in c, and the application needs to run on linux and solaris

camelccc
  • 2,847
  • 8
  • 26
  • 52
  • Freeing disk space is not likely to affect `malloc` or `realloc` failures, as they return space from your heap. Also, if you want to avoid mapped file write failures on disk full conditions, then don't use sparse files... – twalberg Dec 30 '14 at 18:06

2 Answers2

3

There is no way for a process to anticipate the fact a memory access will fail due to virtual memory shortage, especially with the mmaped sparse file technique you are using.

You might trap the sigsegv but there are issues with this approach (see Segmentation fault handling ). Otherwise, the simpler solution would be to monitor the file system available space and freeze/stop the most consuming process preventively.

Community
  • 1
  • 1
jlliagre
  • 29,783
  • 6
  • 61
  • 72
0

why dont you save the data you are getting as binary data to file and then read the data back when processing is required. Your RAM can get filled if you keep accumulating too much data for processing make use of the disk's enormous storage.

Hope this helps.

Ganesh Kamath - 'Code Frenzy'
  • 5,094
  • 3
  • 43
  • 60
  • the whole point of mmap is that data can be automaticially swapped to a disk of choice. This memory access will only fail if the disk runs out of space. – camelccc Dec 30 '14 at 15:13
  • @camelccc What are you working on that you are expecting to run out of disk space? – Degustaf Dec 30 '14 at 15:17