0

What I know is that mmap can map process' virtual memory pages to the ones of a file on a disk. We can write and read to and from the memory in a program and it gets reflected in a file's content.

How can this machinery make sequential read (and perhaps processing) of a file faster than, for instance, regular read sys-call? How can it make search (binary search if file is sorted) faster?

I've got it from several sources that mmap does accomplish what I said, but I couldn't find any elaboration on that.

too honest for this site
  • 12,050
  • 4
  • 30
  • 52
Sergey
  • 1,168
  • 2
  • 13
  • 28
  • System calls are very slow as compared to memory access. In case of the `mmap` certain number of pages will be picked up from the file and loaded in the physical pages assigned to the process. Only in case you read/write more that what has been loaded there will be a page fault and the processor will enter the kernel. But if you use repeated `read`s there will be a system most of the time (the OS does buffer a little) and it will be slower. – Ajay Brahmakshatriya Jul 19 '17 at 13:51
  • 2
    Possible duplicate of [Why mmap() is faster than sequential IO?](https://stackoverflow.com/questions/9817233/why-mmap-is-faster-than-sequential-io) – Tony Tannous Jul 19 '17 at 13:51
  • @AjayBrahmakshatriya The system call overhead is neglegible if reading big chunks of data. You shouldn't try to read the file bytewise, though... – Ctx Jul 19 '17 at 13:56
  • @Ctx the user mentions binary search. So I think they would do sequences of fseek and small freads into the file. – Ajay Brahmakshatriya Jul 19 '17 at 14:01
  • @AjayBrahmakshatriya fread is no problem, since it's buffered. – Ctx Jul 19 '17 at 14:03

1 Answers1

0

Since the limiting factor is the reading from disk, it probably isn't faster... With both methods you can configure a read-ahead to speed up sequential reading, which probably is the best you can do.

mmap()-ing a file however has other advantages compared to read()ing it: You do not have to care about the memory management. If the file is very large (exceeding the memory you wish to use in your process), you would have to manage yourself which parts of the file you keep and which you discard. In the case of mmap, the usual memory management routines from the OS decide, which parts of your file remain in memory and which are to discard in the case of memory contention, keeping an eye on the memory usage of the whole system, and not only your process. If you decide, that some parts have to remain always in memory, you can mlock() those.

But I do not see a big performance gain in the general case.

Ctx
  • 18,090
  • 24
  • 36
  • 51
  • One copy step is avoided. But indeed, this is very little compared to the overhead of the *actual* I/O, if you assume the contents are only needed once. –  Jul 19 '17 at 13:56
  • @FelixPalmen are you sure an extra copy is done in case of read? The kernel can swap the physical pages of the user with the pages in which the file has been read. – Ajay Brahmakshatriya Jul 19 '17 at 13:59
  • @AjayBrahmakshatriya the POSIX interface of [`read()`](http://man7.org/linux/man-pages/man2/read.2.html) lets the caller provide a buffer, this buffer could share its page with whatever other data of the program. So yes, the data must be copied there. Maybe a raw system call could work differently. –  Jul 19 '17 at 14:03
  • @FelixPalmen Yes, I agree. The physical pages could be shared with other virtual address ranges too (or even other processes) but the kernel knows where all the page is mapped, it can just replace the new page in all the places. This will be significantly faster than copying all the data. – Ajay Brahmakshatriya Jul 19 '17 at 14:05
  • @AjayBrahmakshatriya it would have to copy all **other** data from this already used page. Assuming a 4k page size and a 1k read buffer somewhere in the middle of it, this wouldn't make much sense. Of course, for buffers much larger than a page size, such ideas could be feasible, but the alignment has to match with what the kernel uses. –  Jul 19 '17 at 14:07
  • 1
    @AjayBrahmakshatriya `mmap()` has the interface that allows exactly this mapping approach without problems/workarounds. –  Jul 19 '17 at 14:08
  • @FelixPalmen Yes, I meant for the chunks that go across pages. For buffer smaller than a page anyway the copy wont be that expensive. – Ajay Brahmakshatriya Jul 19 '17 at 14:09
  • @AjayBrahmakshatriya Yes, but Felix is right here, why reinvent mmap() with read()? – Ctx Jul 19 '17 at 14:10
  • @FelixPalmen, Ctx Agreed! Just seemed like a simple optimization the kernel might be doing under the hood. Anyway, the discussion was unrelated to the question. – Ajay Brahmakshatriya Jul 19 '17 at 14:12