3

I'm trying to implement a file backed memory allocator for a swapless system.

For each new allocation, I use mkstemp to create a temporary file as backstore, mmap it as MAP_SHARED to allow pages to be swapped to the backstore when system's memory pressure is hight. I think I've got this part working.

However I'm having difficulty implementing the deallocation case.

Since at the moment of deallocation, neither the content of the back store, nor the content the resident pages or dirty pages matters any more, the quickest way to do this is to drop and free all resident pages and leave the backstore unchanged. However I didn't find a madvice flags that can do this.

MADV_DONTNEED seems excessive because it will commit the dirty pages to back store. (Not true, see answer below)

MADV_DONTNEED

After a successful MADV_DONTNEED operation, the semantics of memory access in the specified region are changed: subsequent accesses of pages in the range will succeed, but will result in either repopulating the memory contents from the up-to-date contents of the underlying mapped file (for shared file mappings, shared anonymous mappings, and shmem-based techniques such as System V shared memory segments) or zero-fill-on-demand pages for anonymous private mappings.

MADV_REMOVE seems excessive as well because not only does it drops resident pages, it also drops the backstore itself.

MADV_REMOVE

Free up a given range of pages and its associated backing store. This is equivalent to punching a hole in the corresponding byte range of the backing store (see fallocate(2)). Subsequent accesses in the specified address range will see bytes containing zero.

So what steps are the quickest path of unmap/close/delete a mmaped file?

Maybe mmap the same region again as MAP_PRIVATE (like this) and then munmap it?

user3528438
  • 2,737
  • 2
  • 23
  • 42

2 Answers2

1

According to this question, MADV_DONTNEED does exactly this: drop the pages without writing back to the back store.

The clause repopulating the memory contents from the up-to-date contents of the underlying mapped file means loads after MADV_DONTNEED will reload from the back store.

All the dirty pages before MADV_DONTNEED weren't committed to the back store and so will be lost.

In summary: MADV_DONTNEED drops all mapped pages (including dirty pages are not committed to the back store) and leave the back store as is.

Fun and informational video here

user3528438
  • 2,737
  • 2
  • 23
  • 42
  • I think you may be misreading the `msync` docs. It says "Without use of this call, there is no guarantee that changes are written back *before* munmap(2) is called.", which implies that *when* `munmap` is called, the changes will be written back (which makes sense; `msync` is roughly analogous to `fsync`, while `munmap` is analogous to `close`, and `close` doesn't require you to `fsync` first). – ShadowRanger Mar 06 '19 at 02:15
0

madvise MADV_DONTNEED on dirty pages don't discard the changes.
I just wrote a 5GB file using mmap/madvise MADV_DONTNEED/munmap with the last 2 steps taking place in 2-4MB chunks, and the file was created without a single error.
However MADV_DONTNEED has no effect on dirty pages. munmap reduces the RSS of the process, but writing still done at Linux's discretion.

Marcelo Pacheco
  • 152
  • 1
  • 5