How to measure wallclock time of file I/O

Question

I'm writing several benchmark programs in C to achieve the following tasks:

1. The speed with which one can read from a network disk. Print the seconds needed to read 8192 bytes.
2. The speed with which one can read from the local directory /tmp on your local machine. Print the seconds needed to read 8192 bytes.
3. The speed with which one can read from the disk page cache. Print the seconds needed to read 8192 bytes.
4. The speed with which one can write to a network disk. Print the seconds needed to write 8192 bytes.
5. The speed with which one can write to the local directory /tmp on your local machine. Print the seconds needed to write 8192 bytes.

The goal here is to measure just the time that doing the file read or write takes (using read and write to avoid any buffering time from fread)

My general approach for 1 and 2 is to create a file of 8192 bytes and write that to disk (whether it be the local directory or the network disk) and then call sleep(10) to wait for the page cache to flush so that I'm measuring the time of the actual I/O, not the cache I/O. Then I measure the time it takes to do an empty for loop several thousand times, then the time it takes to read 8192 bytes in then subtract the two, divided over the average of all iterations. My code for that looks like:

struct timespec emptyLoop1, emptyLoop2;
clock_gettime(CLOCK_REALTIME, &emptyLoop1);
for(i = 0, j = 0; i < ITERATIONS; i++) {
    j+=i*i;
}
clock_gettime(CLOCK_REALTIME, &emptyLoop2);
char readbuf[NUM_BYTES];

struct timespec beforeRead, afterRead;
clock_gettime(CLOCK_REALTIME, &beforeRead);
for(i = 0, j = 0; i < ITERATIONS; i++){
    j+=i*i;
    read(fd, readbuf, NUM_BYTES);
}

Is that sufficient for accurately measuring the times of reading from those locations?

Next, I'm confused as to how to read from the page cache. Where does that exist on disk and how do I access it? Finally, there's some tricks for 4 and 5 which are apparently much harder than they seem but I'm not sure what I'm missing.

Answer 1

Following is my file reading function, enabling choice of using or not using the memory based cache. If writing files first, similar open statements are needed. Note that direct I/O cannot be used over a LAN and caching can be unpredictable. More details and access to source codes and execution files can be found in http://www.roylongbottom.org.uk/linux_disk_usb_lan_benchmarks.htm.

int readFile(int use, int dsize)
{
    int p;

    if (useCache)
    {
          handle = open(testFile, O_RDONLY);
    }
    else
    {
          handle = open(testFile, O_RDONLY | O_DIRECT);
    }

    if (handle == -1)
    {
        printf (" Cannot open data file for reading\n\n");
        fprintf (outfile, " Cannot open data file for reading\n\n");
        fclose(outfile);
        printf(" Press Enter\n");
        g  = getchar();
        return 0;
    }

    for (p=0; p<use; p++)
    {
        if (read(handle, dataIn, dsize) == -1)
        {
            printf (" Error reading file\n\n");
            fprintf (outfile, " Error reading file\n\n");
            fclose(outfile);
            close(handle);
            printf(" Press Enter\n");
            g  = getchar();
            return 0;
        }           

    }
    close(handle);
    return 1;
}