0

I have a bunch of .py scripts as part of a project. Some of them i want to start and have running in the background whilst the others run through what they need to do.
For example, I have a script which takes a Screenshot every 10 seconds until the script is closed and i wish to have this running in the background whilst the other scripts get called and run through till finish.

Another example is a script which calculates the hash of every file in a designated folder. This has the potential to run for a fair amount of time so it would be good if the rest of the scripts could be kicked off at the same time so they do not have to wait for the Hash script to finish what it is doing before they are invoked.

Is Multiprocessor the right method for this kind of processing, or is there another way to achieve these results which would be better such as this answer: Run multiple python scripts concurrently

Community
  • 1
  • 1
Dr.Pepper
  • 559
  • 4
  • 11
  • 27
  • 1
    I think you already answered your own question... If you already have the scripts written as you want them, the simplest option is likely to call them all using a shell script. With bash, terminating a command with an `&` will start the process in the background so many can be started at once. In windows you call `start /b python script.py` – Aaron Jan 05 '17 at 19:34
  • How would this work if i wanted the program to be an EXE? – Dr.Pepper Jan 05 '17 at 19:53
  • within your batch file... `start /b someprogram.exe` – Aaron Jan 05 '17 at 19:57
  • Thanks, will give this a shot! – Dr.Pepper Jan 05 '17 at 20:00

2 Answers2

0

You could also use something like Celery to run the tasks async and you'll be able to call tasks from within your python code instead of through the shell.

Sean Parsons
  • 724
  • 4
  • 12
0

It depends. With multiprocessing you can create a process manager, so it can spawn the processes the way you want, but there are more flexible ways to do it without coding. Multiprocessing is usually hard.

Check out circus, it's a process manager written in Python that you can use as a library, standalone or via remote API. You can define hooks to model dependencies between processes, see docs.

A simple configuration could be:

[watcher:one-shot-script]
cmd = python script.py
numprocesses = 1
warmup_delay = 30

[watcher:snapshots]
cmd = python snapshots.py
numprocesses = 1
warmup_delay = 30

[watcher:hash]
cmd = python hashing.py
numprocesses = 1
charli
  • 1,700
  • 1
  • 13
  • 21
  • How would this work if i wanted the program to be an EXE? Would circus work in this way? – Dr.Pepper Jan 05 '17 at 19:54
  • Nope, circus it's a linux daemon. If you want an EXE, I'm afraid that you will have to write it with multiprocessing. – charli Jan 05 '17 at 19:57