10

I've been considering using mmap for file reading, and was wondering how portable that is. I'm developing on a Linux platform, but would like my program to work on Mac OS X and Windows.

Can I assume mmap is working on these platforms?

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
monkeyking
  • 6,670
  • 24
  • 61
  • 81
  • 1
    While my answer does not address the portability aspect of mmap, Boost offers alternatives in Boost.Interprocess and Boost.Iostreams which seems to work on Linux as Windows. – gast128 Jan 03 '19 at 11:27

4 Answers4

14

The mmap() function is a POSIX call. It works fine on MacOS X (and Linux, and HP-UX, and AIX, and Solaris).

The problem area will be Windows. I'm not sure whether there is an _mmap() call in the POSIX 'compatibility' sub-system. It is likely to be there — but will have the name with the leading underscore because Microsoft has an alternative view on namespaces and considers mmap() to intrude on the user name space, even if you ask for POSIX functionality. You can find a definition of an alternative Windows interface MapViewOfFile() and discussion about performance in another SO question (mmap() vs reading blocks).

If you try to map large files on a 32-bit system, you may find there isn't enough contiguous space to allocate the whole file in memory, so the memory mapping will fail. Do not assume it will work; decide what your fallback strategy is if it fails.

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
3

The principle of a memory mapped file is fairly portable, but you don't have mmap() on Windows (but things like MapViewOfFile() exist). You could take a peek at the python mmap modules c code to see how they do it for various platforms.

schlenk
  • 7,002
  • 1
  • 25
  • 29
3

Using mmap for reading files isn't portable if you rely on mapping large bits of large files into your address space - 32-bit systems can easily not have a single large usable space - say 1G - of address space available so mmap would fail quite often for a 1G mapping.

MarkR
  • 62,604
  • 14
  • 116
  • 151
1

I consider memory mapped io on UNIXs as not useable for interactive applications, as it may result in a SIGSEGV/SIGBUS (in case of the file has been truncated meanwhile by some other process). Ignoring such sick "solutions" as setjmp/longjmp there is nothing one can do other than to terminate the process after getting SIGSEGV/SIGBUS. The new G++ feature to convert such signals into exceptions seems to be intended mainly for apples OS, since the description states, that one needs runtime support for this G++ feature and there is no information to be found about this G++ feature anywhere. We probably have to wait a couple of years, until structured exception handling like it can be found on windows since more than 20 years makes its way into UNIXs.

  • Although you didn't directly answer the question I found this really helpful, thank you. – Gareth Davidson Feb 20 '14 at 18:05
  • memory mapped io not being useable for non trivial applications on UNIXs affects portability of memory mapped io –  Feb 24 '14 at 20:51
  • Why would you make the other process truncate your file if you know the app would crash? (As a protection against user incompetence there's the fairly standard fcntl()-based locking mechanism.) – exa Nov 28 '19 at 15:17