Reading from file which is being modified with C

Question

I have a programme which is writing results to a file and I would like to read in real-time from that file. It is a normal text file and external programme always write a whole line. I need to run it just on a Linux system.

int last_read = 0;
int res;
FILE *file;
char *buf;
char *end = buf + MAXLINE - 1;
int c;
int fileSize;
char *dst;
while (external_programme_is_running()) {
   file = fopen(logfile, "r"); //without opening and closing it's not working
   fseek(file, 0, SEEK_END);
   fileSize = ftell(file);
   if (fileSize > last_read) {
      fseek(file, last_read, SEEK_SET);
      while (!feof(file)) {
        dst = buf;
        while ((c = fgetc(file)) != EOF && c != '\n' && dst < end)
            *dst++ = c;
        *dst = '\0';
        res = ((c == EOF && dst == buf) ? EOF : dst - buf);

         if (res != -1) {
            last_read = ftell(file);
            parse_result(buf)
         }
      }
  }
  fclose(file);
}

Is this a correct approach? Or would it be better to check the modification time and then open the file? Is is possible that reading would crash in case that the file would be modified at the very same time?

why not using pipes? then you don't have to go through the hassle of file sharing, opening and seeking. see this: http://stackoverflow.com/questions/1072125/how-does-piping-work-in-linux — Kamyar Souri, Aug 31 '12 at 01:08
external programme is scheduled by a Portable Batch System (PBS queue). When a wall time is reached (timeout), would PBS kill also the piped process? — Tombart, Aug 31 '12 at 09:15

score 4 · Accepted Answer · answered Aug 30 '12 at 23:54

4

To avoid the need to close, re-open, and re-seek for every loop iteration, call clearerr on the stream after reading EOF.

answered Aug 30 '12 at 23:54

Alan Curry

14,255
3
32
33

score 1 · Answer 2 · answered Aug 30 '12 at 23:09

You shouldn't have any problems if you read at the same time the other program writes. The worst that would happen is that you wouldn't get to see what was written until the next time you open the file.

Also, your approach of comparing the last seek position to the end of the file is a fine way to look for additions to the file, since the external program is simply writing additional lines. I would recommend adding a sleep(1) at the end of your loop, though, so you don't use a ton of CPU.

thanks. yep, I have there a sleep(1) in the function `external_programme_is_running()` which also takes some time — Tombart, Aug 31 '12 at 07:09

score 0 · Answer 3 · answered Aug 30 '12 at 23:39

0

There's no problem in reading a file while another process is writing to it. The standard tail -f utility is used often for that very purpose.

The only caveat is that the writing process must not exclusively lock the file.

Regarding your read code (ie. using fgetc()), since you said that the writing process will be writing a line at a time, you might want to look at fgets() instead.

answered Aug 30 '12 at 23:39

Andrew Edgecombe

39,594
3
35
61

I guess `tail -f` wouldn't change anything, the file would be just modified by other process. yes, `fgets()` would make the code a bit more readable, but I would have to check line endings anyway – Tombart Aug 31 '12 at 09:24

Reading from file which is being modified with C

3 Answers3