1

I'm planning to write a programm which has to access to a certain file many times in r/w. So I decided to use fstream, since I can use this class for both reading and writing purpose.

My idea is to open the file at the startup of the application and then close it as the application is closed too.

Since the file can be arbitrarily big, I was planning to use a "paging" structure, in which:

1) preallocate a fixed amount of memory for each page and a fixed number of page

2) load part of the file in to the first free page

3) if there is no free page, I select one non empty with a certain criterion, I commit all edit in it (if there are any) and then load the part of file in the page.

That's not so hard to code. But I was wondering If I'm going to reinvent the wheel... maybe the fstream itself is written in a smart way so that it also implements a similar paging mechanism. In that case, I would not take care about, just write and read at any time.

Some suggestion?

Rodrigo Guedes
  • 1,169
  • 2
  • 13
  • 27
the_candyman
  • 1,563
  • 4
  • 22
  • 36
  • 2
    I may be wrong but isn't memory mapping a best answer for what you want to do? – Uflex Dec 11 '12 at 11:55
  • @Uflex. What do you mean with memory mapping? – the_candyman Dec 11 '12 at 11:57
  • 2
    fstream has indeed write and read buffers. So it already does what you want to do. – log0 Dec 11 '12 at 12:04
  • @log0. Thanks! That's what I was wondering! :D So for example, when I call a "write", it will not be written to disk until the behind algo decide to commit according to a certain politic aimed to improve performance, right? – the_candyman Dec 11 '12 at 12:07
  • 1
    @the_candyman Exactly. Usually the policy is to flush after a certain size is reached or before each read. In fact you can force the output buffer to be written on the disk by calling `flush` http://www.cplusplus.com/reference/ostream/ostream/flush/ – log0 Dec 11 '12 at 13:28
  • @log0. The same reasoning holds even for reading, right? I mean, he will not read again from file a part that it has already into a buffer, doesn't it? – the_candyman Dec 11 '12 at 14:48
  • @the_candyman yes, same thing for reading, there is an input buffer. checkout http://www.cplusplus.com/doc/tutorial/files/ "Buffers and Synchronization" – log0 Dec 11 '12 at 14:58

2 Answers2

1

Internal working of the standard C++ library vary by implementation. Hence a test would be needed to get some real data on your preferred platform. Generally memory mapped files are considered to be the fastest way to access data stored in a file (as Uflex has mentioned in his comment, but it has some drawbacks as well (see the linked wiki page). You can either use the standard (POSIX) C functions mmap() and munmap(), or the Boost C++ libraries which also have a portable C++ interface for memory mapped files.

peterph
  • 980
  • 6
  • 11
1

Don't do this by yourself. Unless you are using very exotic implementation, the fstream class already implement such a mechanism efficiently.

Checkout http://www.cplusplus.com/doc/tutorial/files/ "Buffers and Synchronization"

There are possible issues if you are seek-ing into file larger than 2GB with a old kernel or implementation of the standard library. Check this Large file support in C++ or use Boost.Filesystem

Community
  • 1
  • 1
log0
  • 10,489
  • 4
  • 28
  • 62