1

First, a little motivating background info: I've got a C++-based server process that runs on an embedded ARM/Linux-based computer. It works pretty well, but as part of its operation it creates a fairly large fixed-size array (e.g. dozens to hundreds of megabytes) of temporary/non-persistent state information, which it currently keeps on the heap, and it accesses and/or updates that data from time to time.

I'm investigating how far I can scale things up, and one problem I'm running into is that eventually (as I stress-test the server by making its configuration larger and larger), this data structure gets big enough to cause out-of-memory problems, and then the OOM killer shows up, and general unhappiness ensues. Note that this embedded configuration of Linux doesn't have swap enabled, and I can't (easily) enable a swap partition.

One idea I have on how to ameliorate the issue is to allocate this large array on the computer's local flash partition, instead of directly in RAM, and then use mmap() to make it appear to the server process like it's still in RAM. That would reduce RAM usage considerably, and my hope is that Linux's filesystem-cache would mask most of the resulting performance cost.

My only real concern is file management -- in particular, I'd like to avoid any chance of filling up the flash drive with "orphan" backing-store files (i.e. old files whose processes don't exist any longer, but the file is still present because its creating process crashed or by some other mistake forgot to delete it on exit). I'd also like to be able to run multiple instances of the server simultaneously on the same computer, without the instances interfering with each other.

My question is, does Linux have any built-it facility for handling this sort of use-case? I'm particularly imagining some way to flag a file (or an mmap() handle or similar) so that when the file that created the process exits-or-crashes, the OS automagically deletes the file (similar to the way Linux already automagically recovers all of the RAM that was allocated by a process, when the process exits-or-crashes).

Or, if Linux doesn't have any built-in auto-temp-file-cleanup feature, is there a "best practice" that people use to ensure that large temporary files don't end up filling up a drive due to unintentionally becoming persistent?

Note that AFAICT simply placing the file in /tmp won't help me, since /tmp is using a RAM-disk and therefore doesn't give me any RAM-usage advantage over simply allocating in-process heap storage.

Jeremy Friesner
  • 70,199
  • 15
  • 131
  • 234

3 Answers3

4

Yes, and I do this all the time...

open the file, unlink it, use ftruncate or (better) posix_fallocate to make it the right size, then use mmap with MAP_SHARED to map it into your address space. You can then close the descriptor immediately if you want; the memory mapping itself will keep the file around.

For speed, you might find you want to help Linux manage its page cache. You can use posix_madvise with POSIX_MADV_WILLNEED to advise the kernel to page data in and POSIX_MADV_DONTNEED to advise the kernel to release the pages.

You might find that last does not work the way you want, especially for dirty pages. You can use sync_file_range to explicitly control flushing to disk. (Although in that case you will want to keep the file descriptor open.)

All of this is perfectly standard POSIX except for the Linux-specific sync_file_range.

Nemo
  • 70,042
  • 10
  • 116
  • 153
  • Great answer, thanks! Given that I'm going to be treating this data as if it was in RAM (i.e. with no need for persistence), is there any reason why I might care when/how/if the data is ever flushed to disk? – Jeremy Friesner Sep 05 '18 at 20:46
  • 2
    @JeremyFriesner: Possibly, because by assumption you do not have enough physical RAM... The kernel will flush and load pages based on its caching logic, but you might be able to do (much) better since you know your own access patterns. In particular, if you know that some part of the array is "cold", you can probably improve performance by explicitly syncing it to disk and advising the kernel to reclaim those pages. If your access patterns are truly random, then sure, just let the kernel handle it. – Nemo Sep 05 '18 at 21:38
2

Yes, You create/open the file. Then you remove() the file by its filename.

The file will still be open by your process and you can read/write it just like any opened file, and it will disappear when the process having the file opened exits.

I believe this behavior is mandated by posix, so it will work on any unix like system. Even at a hard reboot, the space will be reclaimed.

nos
  • 223,662
  • 58
  • 417
  • 506
1

I believe this is filesystem-specific, but most Linux filesystems allow deletion of open files. The file will still exist until the last handle to it is closed. I would recommend that you open the file then delete it immediately and it will be automatically cleaned up when your process exits for any reason.

For further details, see this post: What happens to an open file handle on Linux if the pointed file gets moved, delete

Dark Falcon
  • 43,592
  • 5
  • 83
  • 98
  • 1
    This is [POSIX defined behavior](http://pubs.opengroup.org/onlinepubs/9699919799/functions/unlink.html), it works on all normal fs and Unix os – that other guy Sep 05 '18 at 19:44
  • @thatotherguy: true, but I would bet that not all filesystem drivers implement it in a way which ensures these files are properly cleaned up without additional work. For example, how would the FAT32 filesystem do this? [This question](https://superuser.com/a/486104/14348) indicates a hard reboot might leave orphan files on this filesystem. Since he said flash, who knows which filesystem is in use? – Dark Falcon Sep 05 '18 at 19:48
  • 1
    It works just fine on fat32. Orphan files are an issue on all FS, and they get cleaned up by fsck or journal recovery – that other guy Sep 05 '18 at 19:56