1

I have a threaded C program that I want to launch using a Python GUI in a Unix environment. I want to be able to change the state of the GUI by gathering output from the C program.

For instance, this is output from my C program using printf:

Thread on tile 1: On
Thread on tile 2: OFF
Thread on tile 3: Disable
...
Thread on tile 61: ON

I would update my GUI based on the output. What makes the problem difficult is both my GUI and the C program need to run simultaneously and updates happening in realtime. I also need to be able to send commands to the C program from my GUI.

I'm new to Python, C, and Unix (I know, complete rookie status). I've read up on subprocess, Popen, and pexpect, but not sure how to put it all together of if this is even possible at all.

Thanks in advance

NASA Intern
  • 823
  • 3
  • 11
  • 19

3 Answers3

1

The basic outline of an approach would be to have your python GUI create a new process with the C program and then have the python GUI reading from one end of a pipe while the C program is writing to the other end of the pipe. The python GUI will read the output from the C program, interpret the output, and then do something based on what it has read.

Multiprocessing with Python.

How to fork and return text with Python

Recipe to fork a daemon process on Unix

The recipe article has comments about doing redirection of standard in and standard out which you may need to do.

Richard Chambers
  • 16,643
  • 4
  • 81
  • 106
  • Also take a look at this article. [how to inherit handles](http://stackoverflow.com/questions/8500047/how-to-inherit-stdin-and-stdout-in-python-by-using-os-execv) – Richard Chambers Jul 10 '12 at 00:05
  • An article that goes into more detail about using pipes in python [Problem with Python program using os.pipe and os.fork()](http://stackoverflow.com/questions/871447/problem-with-a-python-program-using-os-pipe-and-os-fork). – Richard Chambers Jul 10 '12 at 00:25
  • Thanks, I'm going to look into this now. – NASA Intern Jul 10 '12 at 15:40
  • I'm reading, but I'm not quite sure i understand why I need to fork/thread the python program? The C program is threaded, and I'm just reading the output of the C program. Or maybe I'm not understanding at all? – NASA Intern Jul 10 '12 at 16:21
  • The idea is to have two cooperating processes running which are communicating with each other by sending text messages to each other. If you have the Python GUI start up the C program as a process, it allows you to do what is necessary to synchronize the two processes and to setup the communication between the two by using shared I/O handles. – Richard Chambers Jul 10 '12 at 22:28
  • However your question brings up an alternative architecture. Could you have the Python GUI running as a front end to a web server and the C Program would connect to the Python GUI web server and camp on the connection sending and receiving messages? The C Program would need to use sockets. And this does open up the architecture for remote access as well so the C program and Python GUI need not be on the same computer. – Richard Chambers Jul 10 '12 at 22:38
  • [really simple web server in python](http://www.linuxjournal.com/content/tech-tip-really-simple-http-server-python). [simple web server in C](http://www.paulgriffiths.net/program/c/webserv.php). [creating a web server in pure C](http://stackoverflow.com/questions/409087/creating-a-web-server-in-pure-c). – Richard Chambers Jul 10 '12 at 22:40
1

There's a toughie. I've run into this problem in the past (with no truly satisfactory solution):

https://groups.google.com/forum/?fromgroups#!topic/comp.lang.python/79uoHgAbg18

As suggested there, take a look at this custom module:

http://pypi.python.org/pypi/sarge/0.1
http://sarge.readthedocs.org/en/latest/

Edit @Richard (not enough rep to comment): The problem with pipes is that unless they are attached to an interactive terminal, they are fully buffered -- meaning that none of the output is passed through the pipe to the Python until the C prog is done running, which certainly doesn't qualify as a real time.

Edit 2: Based on Richard's link and some earlier thinking I had done, it occurred to me that it might be possible to manually loop over the pipe by treating it as a file object and only reading one line at a time:

from time import sleep
# Assume proc is the Popen object
wait_time = 1 # 1 second, same delay as `tail -f`
while True: # Or whatever condition you need
    line = proc.stdout.readline()
    if line != '' and line != '\n':
        parse_line_do_stuff()

    sleep(wait_time)

This assumes that readline() is non-blocking, and further assumes that the pipe is at most line buffered, and even then it might not work. I've never tried it.

Dubslow
  • 553
  • 2
  • 6
  • 15
  • [Problem with a Python program using os.pipe and os.fork()](http://stackoverflow.com/questions/871447/problem-with-a-python-program-using-os-pipe-and-os-fork). With standard C I/O library, fflush(). – Richard Chambers Jul 10 '12 at 00:27
  • 1
    As you note, the Really Big Problem (in many cases anyway—those where the sub-process that you intend to run and control and interact with, is written in a way that you can't easily rewrite or "trick" it into working right) lies in programs doing things in a "fully buffered" manner unless they are connected directly to an interactive terminal. The solution is to provide an interactive terminal: [pexpect](http://pypi.python.org/pypi/pexpect/), on Unix. – torek Jul 10 '12 at 01:53
  • I looked into pexpect (its really complicated to me though!) I guess I just need to try out some of the advice given here and see what will work the best! I just needed to be pointed in the right direction! – NASA Intern Jul 10 '12 at 15:42
  • Can you give me some background on this "buffered" concept? – NASA Intern Jul 10 '12 at 16:24
  • Sarge looks promising! and is much simpler that pexpect! – NASA Intern Jul 10 '12 at 16:31
  • @NASAIntern: http://www.gnu.org/software/libc/manual/html_node/Stream-Buffering.html – Dubslow Jul 10 '12 at 18:55
  • For example: In a terminal, such as bash or the Python interpreter, your input to it is line buffered, meaning no characters are actually transmitted to the thing that runs code until it sees a '\n', and the only way to make that is to hit Enter. If it were unbuffered, as soon as you type 'p' the shell would try to execute 'p' as a command, which certainly isn't what you want. – Dubslow Jul 10 '12 at 18:58
0

You have two processes, and you want them to communicate. This is known as "interprocess communication" or even IPC. If you Google for "interprocess communication" you can find some information, like this:

http://beej.us/guide/bgipc/output/html/singlepage/bgipc.html

You might want to try a "domain socket" for communicating between the C program and the Python GUI. The guide linked above explains how to talk through a domain socket in C; here is a tutorial on talking through a domain socket with Python.

http://www.doughellmann.com/PyMOTW/socket/uds.html

steveha
  • 74,789
  • 21
  • 92
  • 117
  • So you are saying I can either use a C socket or a Python socket? either or but not both correct? – NASA Intern Jul 10 '12 at 15:43
  • No, I am saying that if you want a C program to talk to a Python program, both sides can use a socket. The C program will have to use C to talk to the socket, and the Python program will have to use Python to talk to the socket. That "Beej" guide has example C code showing how to talk to a socket, and the second link is a tutorial on using Python code to talk to a socket. – steveha Jul 10 '12 at 16:24
  • Each program will read data from the socket and write data to the socket. It doesn't matter what kind of program is on the other end of the socket. It could be a C program on the other end, a Python program, whatever. – steveha Jul 10 '12 at 16:26
  • I see! Thanks for the clarification! So I assume that by using these sockets I wouldn't have to get into using python' subprocess, piping, ect? – NASA Intern Jul 10 '12 at 16:41
  • Using a domain socket is an alternative to using a named pipe. The advantage of the socket is that you won't need to worry about buffering; the data should arrive quickly. I haven't personally done much with domain sockets so I'm not an expert, but I think it will be a better solution for you than trying to use a named pipe and trying to flush it constantly to force data to be pushed past the buffering. Please read that "Beej" document and see if you think the domain sockets example would help you. That document lists other alternatives as well that might be even better for you. – steveha Jul 10 '12 at 17:20