3

I'd like to know if there any guarantees on the order of the operations on file/file system.

Consider I have a file foo.dat and I update it as following:

 lseek(fd,pos_a,SEEK_SET);
 write(fd,data_a,size_a);  //< - Operation A
 lseek(fd,pos_b,SEEK_SET);
 write(fd,data_b,size_b);  //< - Operation B
 lseek(fd,pos_c,SEEK_SET);
 write(fd,data_c,size_c);  //< - Operation C

Such that I do updates in file A, B, C. And some fault may occur - software crash or for example power failure.

Is there any guarantees that the operations if they are executed are done in same order.

i.e. That there would be either nothing or "A" or "A and B" or "A and B and C"

But not situations like "A and C" or "B" only.

I know that if I call between A and B fsync(fd) and same between A and C but it also guarantees that is actually on file system.

I less care about loose of data but rather about its consistency.

Does POSIX standard guarantees that there will be no out-of-order execution?

So:

  • As there any such guarantees?
  • On POSIX platform?
  • On Windows platform?
  • If not what guarantees (besides fsync) I can have?
Artyom
  • 31,019
  • 21
  • 127
  • 215

1 Answers1

3

This is what POSIX mandates for write:

After a write() to a regular file has successfully returned:

  • Any successful read() from each byte position in the file that was modified by that write shall return the data specified by the write() for that position until such byte positions are again modified.

  • Any subsequent successful write() to the same byte position in the file shall overwrite that file data.

This does not provide you with a guarantee that your data will hit the disk in that order at all. The implementation can re-order physical writes all it wants as long is what the applications "see" is consistent with the above two statements.

In practice, the kernel, and even the disk subsystem (think SANs for instance) can re-order writes (for performance reasons usually).

So you can't rely on the order of your write calls for consistency. You'll need f[data]syncs.

Interesting email thread on the PostgreSQL mailing list: POSIX file updates. Reading on how databases handle I/O is a great way to learn about this type of issue.

(Sorry, don't know about Windows in this respect.)

Community
  • 1
  • 1
Mat
  • 202,337
  • 40
  • 393
  • 406
  • I'd like to add that your answer made me read this document: http://developer.postgresql.org/pgdocs/postgres/wal.html and understand that there is no "magic" solutions. All I just need to use fsync in smart way. – Artyom Sep 19 '11 at 07:45