2

My task is to write small parts of data to file frequently and guaranteed. It's a some type of logging system, that must provide a guaranteed safety and speed of saving data to disk. I found that brakes appear in the Flush() function that is called too often. So, I decided to get rid of it.

I used flags, that prohibit buffering and caching data (osNoBuffer and writeThrough). After this, as I understand, I could perform file I/O operations (write data to file) without use of flush()? Another one question. How can I understand that data is definitely written to disk without caching and buffering? Is it The wright way to do what I want or may be the other way to speed up frequent file write operations??

yurart
  • 593
  • 1
  • 6
  • 23
  • 2
    If want the bits to be written to the disk before the function returns, that will cause some performance hit. What level of confidence are you trying to achieve? –  Oct 12 '10 at 06:14
  • I wrote the test program, that write text data to file. If I use flush it's perfomance is about 50 flushes in second (every flush flushes 512 bytes)/ Without flush, with writing directly to disk - 5000 writes in second. it is 100 times faster. If I use theese flasg, can I count, that all my data is written to disk immediately without need to call flush()? – yurart Oct 12 '10 at 07:24

1 Answers1

3

flush() takes so long, precisely because it establishes the guarantee you're looking for. Files are usually written to disk, which (probably) are spinning rusty disks. Files are physically scattered over the surface of these disks, and it takes milliseconds to move the read/write head to the right place. 50 flushes per second means an average of 20 milliseconds per flush, which is close to what your disk mechanically can do.

Without flush, your writes are combined so multiple writes will require only one movement of the read/write head. Without even examining the specific flags, that must necessarily mean that data is temporarily at risk.

In software alone, you can never know whether data is truly written to disk. We know this because of research performed by Sun, when developing the ZFS file system. It turns out that cheaper, slower disks lie about flushing, and return before data is actually flushed. You will need server-grade disks to guarantee this.

SSDs perform quite well in this area. They have no need to physically move write heads. Therefore, there's less incentive for them to lie about flushes.

MSalters
  • 173,980
  • 10
  • 155
  • 350