0

I have traced some bug of using fputs and fflush in multi processes environment. So I coded simple code like below :

#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include "windows.h"
#define FILENAME "test.txt"

int main(int argc , char *argv[]) 
{ 
    FILE *file; 
    file = fopen(FILENAME , "a"); 

    char buf[10240 + 100];
    int pid = GetCurrentProcessId();
    char buf_pid[20];
    itoa(pid, buf_pid, 10);

    int ii = 0;
    while(ii < 10000)
    {
        time_t     now = time(0);
        struct tm  tstruct;
        char       tt[80];
        tstruct = *localtime(&now);
        strftime(tt, sizeof(tt), "%Y/%m/%d %X", &tstruct);

        strcpy(buf, tt); 
        strcpy(buf + strlen(tt), " ");
        strcpy(buf + strlen(tt) + 1, buf_pid);
        strcpy(buf + strlen(tt) + strlen(buf_pid), " ");

        int len = strlen(tt) + strlen(buf_pid) + 1;

        int a = rand()%10240;
        char temp;
        for(int i=0;i<a;i++)
        {
            temp = 'a' + rand()%25;
            memset(buf + len, temp, 1);
            len++;
        }
        buf[len++] = '\n';
        buf[len++] = 0;

        //Sleep(1);
        printf("%d %s %d %d\n", ii, tt, pid, a);
        fputs(buf, file);
        fflush(file);**

        ii++;
    }

And I runned with some batch file like below :

start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe
start filewrite.exe

Then the result show like below , of course.

2019/11/08 22:57:14 261 rjatydimofugclqurclqecdrhvqsuwbvntmryktdl..
2019/11/08 22:57:14 261 ixoqldswtmhmjxqedqkprngproxvfpeuagbsxexje..
2019/11/08 22:57:14 261 skrhmutgpnkpwopudobptikjrfogdeairwyglcrqe..
2019/11/08 22:57:14 261 sqytmhtrnqkrtnfigwgcvvacgklmcjxinhehaqish..
2019/11/08 22:57:14 261 pvdvsfksuonldbwstvgflwjpecfymtfdrsnxlqqod..
2019/11/08 22:57:14 261 oqatxghvkhhjyvgnlymcjapedmpdakuuphbxnuqtd..
2019/11/08 22:57:14 261 ufrjkikbwvlvjejhpnndyebmgfjbmwvldsrlephow..
..

There is no explicit mutex, so fputs and fflush are not automic. I know fputs is to write the text into the file's buffer with it's handle in os kernel and fflush is to flush the buffer into the disk. And although there isn't synchronization objects, the order of accesses to the each buffer or flush code would be kept.

Is it possible to happen lock with above code? Or if there are 2 file that are being written by many processes, is it possible lock?

I found related good article but I don't convince it really happen. Is a sync/flush needed before writes to a locked file from multiple threads/processes in fopen a+ mode? fputs and fflush, writing and buffer process

Thanks!

semenbari
  • 725
  • 1
  • 8
  • 22
  • 1
    Are you asking why your output appears as expected despite having no locking ? Or are you asking whether locking is needed ? Or are you asking how to properly write to the same file from multiple concurrent processes ? Or something else ? – Sander De Dycker Nov 08 '19 at 14:24
  • Also, please add your environment (specifically OS and file system), since that'll help to provide specific answers. – Sander De Dycker Nov 08 '19 at 14:30
  • My question is that above code would have happen some times. of course, I know it is necessary to use synchronization objects and how to improve it. But the code with the problem doesn't already have them. I just want to know what makes the program hang. The environment is HP unix and maybe it is newest release in 2 year. Thanks! – semenbari Nov 08 '19 at 14:35
  • 2
    You indicate HP unix, buy code execute with 'start filewrite.exe', and is calling GetCurrentProcessId. Sound like Windows. – dash-o Nov 08 '19 at 14:45
  • @dash-o, Hi. yes. The environment is HP unix. And above code is a example I made at home. – semenbari Nov 08 '19 at 14:47
  • Is it possible to happen some hung cases with compile option or C++ version? – semenbari Nov 08 '19 at 14:48
  • You seem to be asking about an issue with code running on HP Unix, but then proceed to show us different code running on Windows that has different behavior. I'm probably still not understanding what you're asking, but either way, it's a good idea to clarify your question. – Sander De Dycker Nov 08 '19 at 14:56
  • @Sander De Dycker, yes. If a program has some code sequentially fputs and fflush without any synchronization objects and there are many instances of it, it could have blocking issue? That is my question. Thank. – semenbari Nov 08 '19 at 15:04
  • Sequential ? Not concurrent ? If there are no concurrent writes, synchronization is not needed. And what do you mean by "blocking" ? What is your exact problem on HP Unix ? What code are you running there ? How are you running it ? What behavior do you observe ? Which part of that behavior are you asking about ? Etc. – Sander De Dycker Nov 08 '19 at 15:10
  • @Sander De Dycker. This is a part of what I have seen. There are 20 program instances that getting messages from other agents and make the result and remain log in one log. But last week, the 20 program were locked almost same time for 10 minutes and begin work again without any additional action. I am finding all possiblity of the locking. The 20 programs remain their result in same two file. The 'sequential' mean the code' order was sequential. Of course they will be called by multi programs. once one process preoccupy fputs, then others will wait so on. – semenbari Nov 08 '19 at 15:17

1 Answers1

1

There is a 'hidden' lock in the program - when the code opens the log file, it uses APPEND mode ("a" to open). This will result in the flag O_APPEND set on the open system calls.

Quoting from man open:

O_APPEND The file is opened in append mode. Before each write(2), the file offset is positioned at the end of the file, as if with lseek(2). The modification of the file offset and the write operation are performed as a single atomic step.

The flag guarantee that each 'write' operation (which is executed whenever printf is executed to flush the lines to the file), will be atomic, and will not be mixed with output of other processes. The sync and the lock occurs within the operating system kernel, no user action is needed.

dash-o
  • 13,723
  • 1
  • 10
  • 37
  • 1
    `puts()` is not guaranteed to use a single `write()` call to actually write the data to the file. Nor is any other `FILE *`-based function. – Andrew Henle Nov 08 '19 at 16:49
  • 1
    Write to a **file descriptor** from `open( ..., O_WRONLY | O_APPEND ... );` using `write( fd, buf, strlen( buf ) );` Assuming you don't get a partial `write()` (technically allowed to happen, but I've **never** seen one on regular files), that is almost certainly going to be atomic - it pretty much has to be else the system is not POSIX compliant, but corner cases might expose issues. – Andrew Henle Nov 08 '19 at 16:53
  • @AndrewHenle agree with your comment. But see comment following comment about actual implementation. Reality is most modern systems will do a buffer of 512 (minimum, even in embedded) to 4K by default. Any message up to this side. terminate with new line, will be send with a single write – dash-o Nov 14 '19 at 13:04
  • The output is file buffered, not tty, so random interleaving is almost guaranteed. Each fputs is between about 40 and 10280 characters (rand()-controlled). – mevets Nov 14 '19 at 17:52