How to make fprintf() writes immediately

Question

One way to write into a file is by using fprintf(). However, this function does not write the results into a file immediately. It rather seems to write everything at once when the program is terminated or finished.

My question is the following: I have a program that takes very long time to run (4-5 hours for big dataset). During this time, I want to see the intermediate results so that I don't have to wait for 5 hours. My university uses Sun Grid Engine for job submission. As most of you know, you have to wait until your job finishes to see your final results. Thus, I want to be able to write the intermediate results into a text file and see the updated results as the program is processing (Similarly if I am using printf).

How can I modify fprintf() to write anything I want immediately to the target file?

See also [What are the rules of automatic flushing stdout buffer in C?](https://stackoverflow.com/q/39536212/2410359). — chux - Reinstate Monica, Jul 27 '17 at 02:27

dbush · Accepted Answer · 2017-07-27T02:03:28.247

9

You can use the fflush function after each write to flush the output buffer to disk.

fprintf(fileptr, "writing to file\n");
fflush(fileptr);

If you're on a POSIX system (i.e. Linux, BSD, etc), and you really want to be sure the file is written to disk, i.e. you want to flush the kernel buffers as well as the userspace buffers, also use fsync:

fsync(fileno(fileptr));

But fflush should be sufficient. Don't bother with fsync unless you find that you need to.

edited Jul 27 '17 at 02:03

answered Jul 27 '17 at 01:32

dbush

205,898
23
218
273

Really! that simple!! – Medo Jul 27 '17 at 01:33
Thank you very much. Well explained. – Medo Jul 27 '17 at 01:40
Isn't `fsync` only for POSIX? – lost_in_the_source Jul 27 '17 at 01:59

score 2 · Answer 2 · answered Jul 27 '17 at 04:07

2

Maybe you can set FILE pointer _IONBF mode. Then you cloud not use fflush or fsync. FILE *pFilePointor = fopen(...); setvbuf(pFilePointor, NULL, _IONBF, 0);

fprintf(...) fprintf(...)

answered Jul 27 '17 at 04:07

anka

31
1

Austin · Answer 3 · 2017-07-27T03:23:30.123

fflush
This works on FILE *. For your case it looks more appropriate. Please note fflush(NULL) will update all opened files / streams and my be CPU intensive. You may like to use/avoid fflush(NULL) for performance reason.

fsync
This works on int descriptor. It not only updates file/stream, also metadata. It can work even in system crash / reboot cases as well. You can check man page for more details.

Personally I use fflush, and it works fine for me (in Ubuntu / Linux).

How to make fprintf() writes immediately

3 Answers3

Linked