Boost mapped_file or file_mapping

Question

I'm creating a file-based Octree of a Point Cloud in order to manage filters on really big file (that can't be stored in ram). The bottleneck of this method, so far, is the I/O of the leaves (because I often have to write, rewrite files and read files ...)
So I would like to use memory mapped file technique to make it faster, and I heard about Boost.
But when I search for tutorials, I have seen 2 techniques :
One use #include <boost/iostreams/device/mapped_file.hpp>
and the other #include <boost/interprocess/file_mapping.hpp>.

In my casae, I will have to write really often entires vectors of 3D points in file, then get all informations in these files and recreate vectors from them. A lot of I/O operation with probably a lot of files.
I was wondering which one I have to use? In which case I have to use one instead of the other?

Thanks!

P.S : Is there a difference between boost iostream mapped file and boost interprocess mapped file? I have seen this post, but it doesn't help me for my particular problem.

Did you do any benchmarks? thanks – Oblivion May 02 '21 at 16:55 — Oblivion, May 02 '21 at 16:55

Maxim Egorushkin · Accepted Answer · 2018-11-20T10:51:50.593

3

You can use any of these, however:

Using only boost::iostreams::mapped_file object you get a file mapped into memory with array interface. It doesn't get any simpler than that. You can also use boost::iostreams::stream decorator to attach std::iostream interface to the mapped file.
Whereas boost::interprocess::file_mapping requires using boost::interprocess::mapped_region to map the file_mapping into memory. This method is more flexible as it allows to map parts of the file (rather than the entire file) with different access permissions but it is also more complex.

edited Nov 20 '18 at 10:51

answered Nov 20 '18 at 10:40

Maxim Egorushkin

131,725
17
180
271

I heard with my eyes that memory mapping would not be really efficient if the goal was to read entire file... Is that so? – Raph Schim Nov 20 '18 at 10:55
1

@RaphSchim File mappings primarily avoid copying memory between the kernel and the user space. Don't take their word for their dubious claims, demand benchmarks. – Maxim Egorushkin Nov 20 '18 at 10:57
I will!! :) And for my request here, since I only have to do basic operation with my file, the `boost::iostreams::mapped_file` will be sufficient ! – Raph Schim Nov 20 '18 at 11:00
Do you know where I can find a good tutorial? All the tutos I read about didn't contain enough information on what's happening really ! – Raph Schim Nov 20 '18 at 11:02
@RaphSchim From SO rules: _Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam._ – Maxim Egorushkin Nov 20 '18 at 11:08
Even in comment? But ok ^^ I will find it myself then! :) Thanks a lot! :) – Raph Schim Nov 20 '18 at 12:29
The link labelled `boost::interprocess::file_mapping` actually points to a page about sharing memory generally rather than the detailed documentation for the class, which is fine except it points to the wrong location within the page: it points to the heading about `boost::interprocess::shared_memory_object`. Here's a fixed link: [`boost::interprocess::file_mapping`](https://www.boost.org/doc/libs/1_42_0/doc/html/interprocess/sharedmemorybetweenprocesses.html#interprocess.sharedmemorybetweenprocesses.mapped_file) – Arthur Tacca Oct 28 '20 at 10:01

Boost mapped_file or file_mapping

1 Answers1