Communicate between two python app

Question

I have done two app's :

The first one is a spider that extract all links from a website.
The second one do some checks on each link sent by the first app.

When the first app find a link, how can I send a notification or something else to the second app ?
The second app must listen continuously the data sent by the first app.

I found few post speaking of Queue but I don't really understand how that work.

Can someone explain me with a simple exemple how to communicate between the two app ?

Thank's

score 1 · Answer 1 · answered Oct 05 '16 at 23:12

There are all sorts of ways to accomplish inter-process communication, but by far the simplest is to use the filesystem. Have your spider write it's output to a temp file. When it's finished, move it into a folder that your second process polls periodically and when it finds work, then process it.

The spider could like something like:

import tempfile, os
tmpname = ''
with tempfile.NamedTemporaryFile(delete=False) as tmp:
    tmpname = tmp.name
    tmp.write("spider output....\n")
tgt = os.path.join('incoming', os.path.basename(tmpname))
os.rename(tmpname, tgt)

The second process could look something like this:

import time, os
while 1:
    time.sleep(5)
    for item in os.listdir('incoming'):
        work_item = os.path.join('incoming', item)
        with open(work_item) as fin:
            # do something with item
        os.unlink(work_item)

score 0 · Answer 2 · answered Oct 05 '16 at 22:49

0

You want to save one file as a "module" to be imported by the other file. Here this can be implemented with the the import keyword. For example, if you name the second part of your application listener.py, you can type import listener in your other file (remember to put them in the same folder!) and call any method from the second file. You can read more on Python modules here.

answered Oct 05 '16 at 22:49

jxmorris12

1,262
4
15
27

If I do that the first app will be slowed during the checks. I need to do these checks on an other app – Pixel Oct 05 '16 at 22:52
https://www.youtube.com/watch?v=0oTh1CXRaQ0 This helped me understand modules and packages very well..take your time and follow the code on your computer its worth it. – tesseract Oct 05 '16 at 22:56
I want to separate the two app's on 2 process not on the same – Pixel Oct 05 '16 at 22:58
@Pixel in that case, you're trying to implement something called multithreading in Python. [Here's a good answer explaining how to call a function asynchronously in Python.](http://stackoverflow.com/a/1239252/2287177) – jxmorris12 Oct 05 '16 at 22:58

score 0 · Answer 3 · answered Oct 05 '16 at 23:13

A Queue is just a container into which items may be put and retrieved, often in FIFO order. The Queue module in Python 2 is just an implementation of one that supports synchronized access, meaning that it supports multiple threads using it (putting and getting things) at the same time.

Communicate between two python app

3 Answers3