Link data from a running C code to a running Python code

Question

I am receiving data in my C code, and currently the only thing I can do is to just print the data on the screen. What I want to know if there's a possibility to instead of just printing the data on the screen, is it doable to send the data to a running Python code that will be able to further process the data? The data is in an array form. The device that will have the data printed on the screen will be running the Python code.

I am doing this to avoid saving the data in a file and then accessing the file from my python script, which will increase the delay for my time dependent system.

The C code is running on a low processing device that is just able to receive and the data without any processing.

(EDIT) A demonstration of what I mean has been included:

C side:

// code that receives the data periodically as an array 
// when received do 
printf(" x = %d y = %d z = %d ", data[0],data[1],data[2]); 
// what I am looking for is something like this 
sendtoPy("filename.py",data)
// repeat 

Python side:

# this code just sits to wait for anything coming from C 
# wait for any code to be received from datafromCexample.c
def waitforC("datafromCexample.c",data):
      print "data has been received"
      # process the data received into a list
      # further processing 
waitforC(data)

Of course this is just raw code for demonstration. If I made mistakes in my demonstration then please teach me and correct me, I simply want to learn. If this is not available, could you please include alternatives that can be used in this situation?

Thank you.

Answer 1

If the C program and the Python program/script can run on the same device, you have several options:

• Let the C code output the data on standard output (this is probably what it's already doing), pipe it to the Python program and let the Python program read in on standard input.
  or
• Let the C code output the data on standard output and call the C program from your Python code (e.g. with subprocess.check_output, processing the the produced output.
  or
• In the C code, have a function that will return the data chunk by chunk (one chunk per call, where "chunk" is a quantity of a size you can easily produce and process). Use Cyton (see Tony's answer) or CFFI or any of the other available ways to call it from Python repeatedly to obtain the data to process.
  or
• Write a Python script that can be invoked with a single chunk of data and process it. Call that Python script repeatedly for all chunks of data from your C code, using a system call to invoke it.
  or
• Use Cyton to invoke a python function from C.

Depending on the operating system, you can get even fancier with named pipes, sockets, bus messages and whatnot, but the options above are probably already enough to choose from and should suffice for most use cases. Which one to choose depends on the nature of your data and of the wanted processing of it, as well as some non-functional requirements (e.g., what, if one of the processes is stalling, ...) so cannot be answered in general.

Pipe

In comments to this answer, you inquire

I am looking to go with the first option […]. I would appreciate if you could recommend me a library or procedure that can work well with arrays.

and

For the C side, I will need the library that through it I will pipe the data to the python code. In the Python side, I suppose I will need to use multi-threading that will allow me to 1- run forever (to anticipate any incoming data) 2- receive with the incoming data from the C code, hence a library that will be able to do so

The beauty of pipes is that the involved processes don't need any special libraries in most programming languages. The processes just do something that almost every general purpose programming language allows in a (more or less) easy way: Reading from standard input and/or writing to standard output. That is also what command line applications do when they read what you type in on your keyboard and/or print text to the (today often virtual) terminal (a.k.a. a text console).

Forwarding one process' output to the other process' input (as if a user was reading what the first process prints and typing that in to the second process) is the operating system's task, so the processes don't have to be concerned with that. The first process just needs to output a format the second process can understand.

For example, the unix command ls will list the files and directories in the current working directory. Let's assume it aways prints one name per line. The unix command shuf reads lines from standard input until it detects an end-of-file character in them, then prints them in a random order. (It shuffles them.)

You can invoke shuf in a shell (e.g. sh or bash) and type in Hello World![Enter]I like shuffled text.[Enter]Do you, too?[Enter]No? What a pity.[Enter][Ctrl+d] and might get an output like:

Do you, too?
Hello World!
No? What a pity.
I like shuffled text.

If you want to list the files and directories in the current working directory, but in a different random order than returned by ls, you can pipe the output of ls through shuf. E.g. in sh or bash:

ls | shuf

would take the output of ls and forward it to shuf as-if it was typed in manually. (Only much quicker than you can type.)

Thus, if your C program produces machine-parseable output on the terminal (which might already be the case; You might want to include an example of its current terminal output in your question.), you can forward that to your python program with something like

./binary_of_your_c_program | python ./your_python_script.py

Your python program then just has to read from standard input and process what it gets there however you want it to. Most straight-forward approaches for reading standard input will loop over all lines received until an end-of-file occurs or they receive a signal that terminates the process, so keeping the python script perpetually running shouldn't be too hard. (Unless you also care about having it automatically restarted when it unexpectedly terminates, e.g. due to a runtime error.)

Answer 2

You may consider invoke C function from python code, read this example https://csl.name/post/c-functions-python/

Answer 3

If You care about time efficiency of Your system, I would use Cython for wrapping the c part of the program that You have. You can read the full tutorial on that here. You may also use the Cython for speeding up the python part of the code.