Sharing Variable between C and Python

Question

I'm using a named pipe FIFO to communicate between a C ( a loop generating numbers is C) and python (listening to the number generated by C and read them and process them). The code working well except for each generated number in C i need to close the fifo to be able to see it in Python, otherwise it won't be shown in python till a close command is called. I don't know if opening and closing FIFO is a good idea to be able to read them correctly in python code or not. I need to note that the C code generates number every 50 millisecond. This is why I doubt opening and closing is a good idea or not. Here are my codes in C and Python:

C as Server:

while (1){
            t=time_in_ms();
            if (t-t0>=50){
                    t0=t;
                    flag=1;
            }
            else{
                    flag=0;
            }
            if (flag==1){
                    flag=0;
                    printf("%lld %lld\n",count,t);
                    count+=1;
                    fprintf(f,"%lld\r",t);
                    fflush(f);
                    fclose(f);
                    f=fopen("fifo","w");

            }
    }

And Code in Python as Client:

with open(FIFO) as fifo:
print("FIFO opened")
while True:
    data = fifo.read()
    if len(data) == 0:
            count=count+1
    else:
            count=0
    if count>2000:
        print("Writer closed")
        break
    print data
    x=x+1

Possible duplicate of [Integrate Python And C++](https://stackoverflow.com/questions/1153577/integrate-python-and-c) — wwii, Nov 12 '17 at 16:09
The problem is that on *both ends of the fifo* you're using *buffered IO*. You should use `open/write` on the C side and `os.open/os.read` on the python side! — Antti Haapala -- Слава Україні, Nov 12 '17 at 16:25
May I know what you mean by use open/write on the C side and os.open/os.read on the python side? Could you please let me know how I should change the code to address your comments? — user2684657, Nov 12 '17 at 16:45
Regarding the C-code: `man 2 open` `man 2 write` Also have a look at: `man 3 fflush` When working with C, the man-pages in Linux is awesome and great resource — David Bern, Nov 12 '17 at 16:48
What I think @AnttiHaapala is saying; don't treat FIFO as STREAM, use the File descriptor instead. Like the `write` function does or dprintf. I should add, that I have never used dprintf before and just speculating on the behaviour — David Bern, Nov 12 '17 at 16:52
Notice that in Python, the buffered `file.read` *will* try to read until EOF. One option would be to use `readline` but it is not to say that it would be completely free of problems. — Antti Haapala -- Слава Україні, Nov 12 '17 at 17:01

score 0 · Accepted Answer · answered Nov 12 '17 at 17:11

0

Here is a small working example

Python side:

with open('./test_out.fifo', 'r') as fo:
    message = fo.readline()
    print(message)

On the "server" C-side

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

int main()
{
    int fifo_fd;
    fifo_fd = open("./test_out.fifo", O_WRONLY);
    dprintf(fifo_fd, "testing123\n");

    while(1)
    {
        sleep(1);
    }
    return 0;
}

The endless looping in the end of the C-program is just to demonstrate that we do not need to close the file before the data is readable in the python program

I should also say that I haven't done C-code in a while.

answered Nov 12 '17 at 17:11

David Bern

778
6
16

Do you know if there is a way that in python side I can check the fifo if the data is ready catch it otherwise be able to do other thinks? It seems the right now I'm in clock mode that will wait to a new data to come. I'm asking because I need to process the data after getting it in python, I don't want to lose the process power of system being blocked by reading FIFO. – user2684657 Nov 12 '17 at 21:00
let me think about this a moment. I'll get back to you here. – David Bern Nov 12 '17 at 21:03
1

let me explain my case, my sensor in C gets data every 50 millisecond and puts in FIFO, in python side when the FIFO has a data, I read it and analyze it once analysis is finished I should go back and check if there is any new data in FIFO and read it and do it again. My only concern is that if I'm analyzing data takes more than 50msec, I won't lose any data from FIFO. And I don't know what is the length of the fifo. – user2684657 Nov 12 '17 at 21:08
To answer the size. Try this command `sysctl fs.pipe-max-size` Regarding the other problem, there are solutions to this, im sure of it. Think of the problem for a while. Formulate a question and I will try my best to help you solve it. Why dont I help you here in the comments? Simple, perhaps someone else has the same problem, he/she wont find that answer here & if you think about the problem for a while, perhaps you find someone else that already had that exact question already. Feel free to give an url to your future question in this comment – David Bern Nov 12 '17 at 21:15
Of course, I will think about it and let you know about my findings. – user2684657 Nov 12 '17 at 21:19
One thing to think about here is. If you produce data in a rate that is greater then you are able to handle it, for how long can you do that before you run out of RAM? Is the rate just periodic and the consumer will be able to catch up? In that case, just increase the allowed FIFO size with the command I gave you. If the concern is that you dont want to waste time waiting for data to arrive on the fifo or perhaps a combination of them. Search around for blocking/non-blocking FIFO read. This starts to sound more and more like a question about technology then "programming" problems. – David Bern Nov 12 '17 at 21:25
exactly the second case, I mean depending of the data we are getting in Python, the processing it might take longer but I can say this happens just in 10% of data, for 90% the data analysis will finish before new data come to buffer. Then I think if the size of buffer is large enough then I should be okay. ' – user2684657 Nov 12 '17 at 21:51
In that case, just increase the "fifo" buffer size with the sysctl command I gave you in a earlier comment and the time you will have to wait for a complete line to appear in the fifo shouldn't be a problem. – David Bern Nov 14 '17 at 12:51

Sharing Variable between C and Python

1 Answers1