How to do multiplexing I/O for read multiple file when the file size increase?

Question

There are many files generated on network shared file system (NFS). There is a similar question without proper solution: inotify with NFS.

I use select() to test if the file have new data could be read. (In fact, some are come from socket descriptor, just simplified here).

But, I found even the file till end of file, it still return ready to read state.

Could you suggest better method to write this code?

fd_set rfds;
struct timeval tv;
int retval;
int i,n,f1,f2,maxfd;
char buf[512];

f1 = fileno(fopen("f1", "rb"));
f2 = fileno(fopen("f2", "rb"));
maxfd = (f1 > f2) ? f1 : f2;

for (i=0; i<3; i++) {
    FD_ZERO(&rfds);
    FD_SET(f1, &rfds);
    FD_SET(f2, &rfds);

    tv.tv_sec = 5;
    tv.tv_usec = 0;

    retval = select(maxfd+1, &rfds, NULL, NULL, &tv);

    if (retval == -1)
        perror("select()");
    else if (retval) {
        printf("Data is available now.\n");
        if (FD_ISSET(f1, &rfds)) {
            n = read(f1, buf, sizeof(buf));
            printf("f1 is ready:%d read %d bytes\n", i, n);
        }
        if (FD_ISSET(f2, &rfds)) {
            n = read(f2, buf, sizeof(buf));
            printf("f2 is ready:%d read %d bytes\n", i, n);
        }
    } else
    printf("No data within five seconds.\n");
}

The output will like following if my f1 and f2 contains 3 bytes.

Data is available now.
f1 is ready:0 read 3 bytes
f2 is ready:0 read 3 bytes
Data is available now.
f1 is ready:1 read 0 bytes   <- I wish won't enter here
f2 is ready:1 read 0 bytes   <- I wish won't enter here
Data is available now.
f1 is ready:2 read 0 bytes   <- I wish won't enter here
f2 is ready:2 read 0 bytes   <- I wish won't enter here

score 0 · Answer 1 · answered Apr 22 '13 at 07:16

0

NFS doesn't have a way to notify clients when files change, so you're unfortunately out-of-luck. You'll need to poll.

answered Apr 22 '13 at 07:16

tmyklebu

13,915
3
28
57

do you mean if I use epoll/poll, it could solve this problem? (The shared file is on NFS, and I can not touch the NFS server box). – Daniel YC Lin Apr 22 '13 at 07:19

score 0 · Answer 2 · answered Apr 22 '13 at 10:00

In Unix, regular files are always considered "fast devices", so they cannot be polled. That is, as you have found out, they always return "ready" if you try to select() or poll() on them. IIRC the Linux-specific epoll returns an error outright if you try to poll on a regular fd.

If you want to integrate something like this into your event loop you'll have to apply some duct tape. E.g. have a separate thread which at suitable intervals tries to read()/fstat()/stat() the file/fd, then if it detects that new data is available, send a message to a pipe. In the main event loop you can then poll the pipe.

Thanks your comment,that's what I've done. But I'm wonder is there other method without any delay in my code and free the CPU load also. — Daniel YC Lin, Apr 23 '13 at 03:35

How to do multiplexing I/O for read multiple file when the file size increase?

2 Answers2