0

I am aware that this question has already been asked, but I have done extensive research and am unable to figure my problem out.

My question has to do with running complex functions in parallel.

I'm using selenium webdriver to automatically download files from one website and upload them to another, and I want these functions to run at the same time. They both use while loops and have layered conditional statements. I cannot get the functions to run at the same time and would appreciate help. My code is as follows: 

import multiprocessing

def auto_download():

    # function logic here 

def auto_upload():

    # function logic here 

if __name__ == '__main__':

p1 = multiprocessing.Process(name='auto download', target=auto_download())
p2 = multiprocessing.Process(name='auto upload', target=auto_upload())
p1.start()
p2.start()

This code runs the first function auto_download() but never starts the second.

However, if I run the following code from here, which is the same idea but with much simpler functions, it works fine. 

import multiprocessing
import time

def add():
    while True:
        print (1)
        time.sleep(3)

def sud():
    while True:
        print(0)
        time.sleep(3)

if __name__ == '__main__':
    p1 = multiprocessing.Process(name='p1', target=add)
    p = multiprocessing.Process(name='p', target=sud)
    p1.start()
    p.start()

Does my problem stem from the complexity of the functions I am trying to run simultaneously? Thanks ahead of time for your help!

EDIT: The solution (thanks to Raw Dawg) is that I was calling the function directly instead of in the process object. This is different than the solutions for this question. The following code fixes the problem: 

p1 = multiprocessing.Process(name='auto download', target=auto_download)
p2 = multiprocessing.Process(name='auto upload', target=auto_upload)
p1.start()
p2.start()
Jared Forth
  • 1,577
  • 6
  • 17
  • 32
  • 1
    It may have something to do with the content of your functions blocking each other. I had a similar issue some years ago. Is it possible to publish the function code? – Doralitze Mar 07 '18 at 17:14
  • Thanks for your reply! There is confidential information contained in the functions (I'm writing this program for a company), but I'll edit that out so I can try to express the function logic. I'll edit and update my post. – Jared Forth Mar 07 '18 at 17:19
  • Possible duplicate of [Python multiprocessing : processes do not start](https://stackoverflow.com/questions/45237326/python-multiprocessing-processes-do-not-start) – JohanL Mar 07 '18 at 17:41

1 Answers1

1

I think I found the problem here - when you define your process with 

p1 = multiprocessing.Process(name='auto download', target=auto_download())

You are running the function auto_download(). You are essentially starting this function in the main process, not in a Process object as you want to. For this reason p1.start() does not have the effect you want, as you aren't running these functions concurrently. Notice that in the second example in the Process signature you define it as 

p1 = multiprocessing.Process(name='p1', target=add)

with no parenthesis after add. Try it again with the following code:

p1 = multiprocessing.Process(name='auto download', target=auto_download)
p2 = multiprocessing.Process(name='auto upload', target=auto_upload)
p1.start()
p2.start()
Sevy
  • 688
  • 4
  • 11