1

I want to know the difference between

    char *text = (char *)malloc(4096);
    memset(text, 'a', 4096);

    int fd = open(filepath, O_RDWR | O_CREAT | O_DIRECT);

    for(int i=0; i<1000; i++) {
        write(fd, (void *)text, 4096);
        fsync(fd);
    }

    close (fd);

and

    char *text = (char *)malloc(4096);
    memset(text, 'a', 4096);

    int fd = open(filepath, O_RDWR | O_CREAT | O_DIRECT | O_SYNC); //<----Difference

    for(int i=0; i<1000; i++) {
        write(fd, (void *)text, 4096);
        // fsync(fd);  <--------------------------------------------------Difference
    }

    close (fd);

The performance of the code above is way slower than that below.

codewario
  • 19,553
  • 20
  • 90
  • 159
Tim He
  • 350
  • 2
  • 13
  • 1
    Potentially no difference -- both failed undetected since you don't check the return values ;-). – Peter - Reinstate Monica Jan 21 '21 at 13:51
  • On a more serious side, according to the open(2) man page, on a file opened with the O_SYNC flag writing should work "as though each write(2) was followed by a call to fsync(2)", which is what you do, so the two should be equivalent. No idea why there are differences. Could a smart driver still combine successive write()s (but not if there is an intervening fsync())? That would in my book violate the O_SYNC semantic guarantee. – Peter - Reinstate Monica Jan 21 '21 at 13:52
  • 1
    Another thing I find remarkable is that the file systems/drivers appear to indeed flash the disk cache as well. The sync(2) man page warns that due to write delays and disk caches the data may not actually be on the disk yet when the call returns. That is, apparently, different with fsync(). – Peter - Reinstate Monica Jan 21 '21 at 13:58
  • fsync after the loop may increase the throughput considerably, however, as already commented, fsync after write is basically what O_SYNC does (without considering kernel's wisdom) – Jose Jan 21 '21 at 14:16
  • Thanks for the answer! The big performance difference appears in both of my SSDs (Samsung 860 evo & 980 pro). I am considering to disable the disk cache by `hdparm -W0 /dev/860evo` which is SATA SSD, but here comes another question: how to disable disk cache of NVMe SSD? – Tim He Jan 21 '21 at 16:53

1 Answers1

2

On the one hand there shouldn't be any difference because a similar amount of work has to happen in both cases (write then disk flush assuming no chicanery). On the other hand, the first case has to do twice as many syscalls so (in theory) has more overhead (especially if time it takes to make the syscall is a significant part of total time it takes to do the operation). In all probability it likely depends on the disk/kernel/CPU/size of I/O etc. as to whether there is a difference between the two and which is faster. Maybe in the second case the kernel can send the write down with the FUA bit set which would mean the difference could depend on just what file/device you were opening (because that may control whether such an optimisation can be done)...

Using O_SYNC also makes errors appear on the return of the write() call but as noted in other comments you're not checking the return codes...

Anon
  • 6,306
  • 2
  • 38
  • 56