0

Python program is doing a front end-work and C program is doing a back-end work. Each of them is a separate process.

Python process acts as a master process which sends signals and events to the C process.

The C process generates stats, counters and information which is sent back to the python process which displays it to the user.

There will be a duplex communication between them

After searching for different techniques, I found these are some of the ways I could do it:

1. Create a UDP/TCP socket between the 2 processes.
2. Use a memory mapped file.

What should be the format of the data passed? 

1. Should I use variable size structures?
2. Should I use Tag Length Value Format?
3. Should I use XML?

Can I have a common file which both the process understand?

For example, a common header file which has the format of the data to passed.

Different kinds of data messages are

1. Signal messages from python to c.
2. Signal messages from c to python.
3. Heartbeat messages from python to c.
4. Stats messages from c to python.
5. Log messages from c to python.
6. Progress messages from c to python.
7. Update messages from python to c

In terms of C, each message is C structure.

Nikhil
  • 2,168
  • 6
  • 33
  • 51
  • Please provide details of your environment, e.g. allowed latency (realtime or slow?), typical message size (few bytes or MB or TB?), desired throughput (occasional messages or 1Gtps?), how these processes are spawned (one is a child of the other, or fully independent?), are you in control of source code for both processes? What are reliability requirements (if one process is slow, should the other block or discard message?) – Dima Tisnek Nov 18 '14 at 13:15

3 Answers3

3

Not allowed to comment, so: Why is a simple pipe | not enough?

JdeHaan
  • 332
  • 6
  • 19
  • what is "Not allowed to comment" ? please explain your answer – Riad Nov 18 '14 at 10:30
  • 1
    @Riad: as in "I am not allowed to comment on the question since I dont have a 50 reputation so I ask my question this way." – JdeHaan Nov 18 '14 at 10:46
  • @JdeHaan how do you send messages back through a shell pipe? – deets Nov 18 '14 at 11:25
  • @deets: Ok, you have to cheat a bit with a fifo: mkfifo fifo ./process1 argument1 < fifo | ./process2 argument1 > fifo ; rm fifo see http://stackoverflow.com/questions/5129276/how-to-redirect-stdout-of-2nd-process-back-to-stdin-of-1st-process – JdeHaan Nov 19 '14 at 07:48
2

Simplest of all: text files.

2nd simplest: json files.

More interesting, most flexible: wrap your C program in a Python runner, use ctypes to communicate within this process.

Dima Tisnek
  • 11,241
  • 4
  • 68
  • 120
1

I would consider using nanomsg for this. http://nanomsg.org/ It's a simple-to use C-only IPC library, with good python bindings also available.

Alternatively, use it's older, but also awesome ZeroMQ if you like.

Last but not least, while multiprocess-approaches are a good thing (tm), you could consider using ctypes and build a python wrapper around it. The advantage of this approach is that you can leverage the power of python, whilst not losing the performance. E.g. you could become multiprocessing with a few lines of python-code only.

deets
  • 6,285
  • 29
  • 28
  • To extend on the python wrapper approach: You can also use [SWIG](http://www.swig.org) or [cython](http://www.cython.org/) in order to connect your c-code with python. – tssch Nov 18 '14 at 12:08