Share data between bash, python, and C

Question

I am working on a project that has a bunch of c and python files. I am using a menu function using a bash script on a Linux terminal. The bash terminal has some menu options and also puts the files in a loop. I.e. the files are executed cyclically. So, I am doing something like the following in the bash script

 #!/bin/bash

PS3='Please enter your choice: '
options=("Loop" "Quit")
select opt in "${options[@]}"
do
    case $opt in
        "Loop")
            echo "Looping the programs"
            while [ <some test> ]
            do
                  ./cprogram1 arg1 arg2 arg3 ... #calling the c program
                   python3 pythonprogram1.py     #calling the python program
                   sleep .5 # Waits 0.5 second.
            done
            ;;
        "Quit")
            break
            ;;
        *) echo "invalid option $REPLY";;
    esac
done

say cprogram1 spits out some data cprog1data. How can that data be accessed by the pythonprogram1.py?

One way I thought of was to have a log.txt file from which the C program can write and the python program can read. Is this correct? Are there better and faster ways to do this?

The file method is the simplest. If pythonprogram1.py can handle a stream from cprogram1 you could try a pipe. If cprogram1 writes the data as it's generated (rather than storing in memory and writing upon completion) then pythonprogram1.py can begin processing as soon as cprogram1 begins producing output. — Michael Ruth, Jul 27 '22 at 19:35
You can use a pipe: `./cprogram1 arg1 arg2 arg3 | python3 pythonprogram1.py`. The C program outputs to `stdout` and the python program reads from `stdin` — Craig Estey, Jul 27 '22 at 19:37
@MichaelRuth, do you prefer a text file or a binary file, or does it matter? — darthMaul, Jul 27 '22 at 19:43
I need more information about cprogram1 and pythonprogram1.py to have a preference. If the data are text, then use text. If the data are binary then use binary. That is, unless, there's some limitation which hasn't been disclosed in the question. — Michael Ruth, Jul 27 '22 at 19:50
@CraigEstey is the command you wrote not the other way around? — darthMaul, Jul 27 '22 at 19:53
No. You want the output of the C program to be the input of the python program. Pipe syntax is _different_ than file redirection syntax. The pipe command is correct. The corresponding redirection would be: `./cprogram1 arg1 arg2 arg3 > mytmp` and then `python3 pythonprogram1.py < mytmp` Perhaps that helps explain it — Craig Estey, Jul 27 '22 at 19:57
I have never used stdout in C so far. I see that it is just as simple as using a printf command. But, what if I don't want any data to be printed on the terminal. How do I tell it this? @CraigEstey — darthMaul, Jul 27 '22 at 19:58
Yes, using `stdout` [in C] _is_ just `printf`. You can redirect any program's standard output to a pipe or file as in the above examples. If you want to run a program and just ignore/skip/trash its output, you can redirect it to a special file/device: `/dev/null` That is, `./cprogram1 > /dev/null` — Craig Estey, Jul 27 '22 at 20:01
@CraigEstey, `./cprogram1 arg1 arg2 arg3 | python3 pythonprogram1.py` if you do this, will it automatically mean that the printf that I will use in my C program be directed to the python program without it being printed out on the terminal? — darthMaul, Jul 27 '22 at 20:04
Yes--No terminal output. To further clarify, `printf("hello\n")` [or `fprintf(stdout,"hello\n")`] usually doesn't know or care what its `stdout` stream is connected to. `printf` is _not_ "output to TTY" It _is_ "output to standard output stream [for my program]" You control that with the shell command you use to invoke it with. If you do _not_ specify pipe syntax or output redirection, the _default_ is the controlling TTY screen/window. But, it goes where you tell it to. You can even force the TTY: `./cprogram1 > /dev/tty` — Craig Estey, Jul 27 '22 at 20:12
One more question. @CraigEstey. In my process, the cprogram1 has to wait for data to be received before it can actually hand over the data. When I executed `./cprogram1 arg1 arg2 arg3 | python3 pythonprogram1.py`, it seemed like the python program wasn't waiting on the cprogram to complete and was erroring out that it didn't have any data. Is this normal? — darthMaul, Jul 27 '22 at 21:33
To use a pipe the program(s) have to behave. But, from what you describe, if all the python script did was read from `stdin` (e.g.) https://stackoverflow.com/q/1450393/5382650 it _should_ just wait until it receives some input. For a pipe, the python script will _not_ see an EOF just because the C program has not provided any output yet. As long as the pipe is intact the python stage will wait forever for input. So, I might check the python program to be sure it is using `stdin` correctly. — Craig Estey, Jul 27 '22 at 21:43

score -1 · Accepted Answer · answered Jul 27 '22 at 19:46

I think your best option is to use a pipe tp send data from the cprogram1 directly into pythonprogram1 as it is generated. If cprogram1 is designed to finish running before pythonprogram1 even begins, then you could store the output of cprogram1 in a variable in your bash script and use it as an argument for pythonprogram1.

Depending on how cprogram1 and pythonprogram1 run, having them share a file such as log.txt could introduce issues when one program is has the file open and the other is trying to access it.

Share data between bash, python, and C

1 Answers1