Running two python scripts with bash file

Question

I would like to run two python scripts at the same time on my lap top without any decreasing in their calculation's speed.

I have searched and saw this question saying that we should use bash file. I have searched but I did not understand what should I do and how to run those scrips with this way called bash.

python script1.py &
python script2.py &

I am inexperienced in it and I need your professional advice. I do not understand how to do that, where and how. I am using Windows 64bit.

Best

PS: The answer I checked the mark is a way to run in parallel two tasks, but it does not decrease the calculation time for two parallel tasks at all.

open a terminal and run the command `python script1.py &` `python script2.py &` ( ubuntu /linux based system ) you will be able to run the script without affecting on other performance — sahasrara62, Feb 28 '19 at 11:32
@prashantrana I am using windows. so just needs to write `python script1.py & python script2.py &` in command prompt? and every thing will be right? — Ma Y, Feb 28 '19 at 11:33
@prashantrana well I want two proces at same time without decreasing the speed — Ma Y, Feb 28 '19 at 11:37
@MaY yeah it will do it , what i meant was run `python script1.py &` click enter and then run `python script2.py &` in terminal — sahasrara62, Feb 28 '19 at 11:38
@MaY, I see your update and it is absolutely not correct that GNU parallel does not decrease overall computation time. While I don't have any visibility of what you are doing, here's a demonstration of how GNU parallel increase computation time https://www.youtube.com/watch?v=kl8LO2jcvMc — Alex Harvey, Mar 02 '19 at 12:59
And this one is by the author of GNU parallel https://www.youtube.com/watch?v=OpaiGYxkSuQ — Alex Harvey, Mar 02 '19 at 13:00

score 2 · Answer 1 · answered Feb 28 '19 at 11:37

2

I use a batch file which contains these lines:

start python script1.py
start python script2.py

This opens a new window for each start statement.

answered Feb 28 '19 at 11:37

quamrana

37,849
12
53
71

won't `call python script1.py &` be better choice – sahasrara62 Feb 28 '19 at 11:39
Doesn't work for me. The two scripts run sequentially, not concurrently. – quamrana Feb 28 '19 at 11:39
I have two question here. should I write it on `command prompt` . and another one this run both them in parallel? I mean if you open just two Python scripts with IDL your running time will increase. But I do not want to have longer time. Is it possible? – Ma Y Feb 28 '19 at 11:40
1

You can open two command prompts yourself and run `script1` in one and `script2` in the other, or, type the above lines in one command prompt, or use a batch file to start them. – quamrana Feb 28 '19 at 11:43
So far so good. The let me finish my scripts that I have ran them with IDLE, I think they will be finished within 6 hours. Then I will run it. Each of them lasts for 21 hours, and when I run them both together the time is 26 hours. so I should try to see is it possible to have that 21 hours with this way or not. – Ma Y Feb 28 '19 at 11:48

Alex Harvey · Accepted Answer · 2019-03-04T08:32:30.680

2

If you can install GNU Parallel on Windows under Git Bash (ref), then you can run the two scripts on separate CPUs this way:

▶ (cat <<EOF) | parallel --jobs 2
python script1.py
python script2.py
EOF

Note from the parallel man page:

   --jobs N
       Number of jobslots on each machine. Run up to N jobs in parallel.
       0 means as many as possible. Default is 100% which will run one job per
       CPU on each machine.

Note that the question has been updated to state that parallelisation does not improve calculation time, which is not generally a correct statement.

While the benefits of parallelisation are highly machine- and workload-dependent, parallelisation significantly improves the processing time of CPU-bound processes on multi-core computers.

Here is a demonstration based on calculating 50,000 digits of Pi using Spigot's algorithm (code) on my quad-core MacBook Pro:

Single task (52s):

▶ time python3 spigot.py
...
python3 spigot.py 52.73s user 0.32s system 98% cpu 53.857 total

Running the same computation twice in GNU parallel (74s):

▶ (cat <<EOF) | time parallel --jobs 2                                                                                                                                   
python3 spigot.py                                                                                                                                                      
python3 spigot.py                                                                                                                                                      
EOF        
...
parallel --jobs 2  74.19s user 0.48s system 196% cpu 37.923 total

Of course this is on a system that is busy running an operating system and all my other apps, so it doesn't halve the processing time, but it is a big improvement all the same.

See also this related Stack Overflow answer.

edited Mar 04 '19 at 08:32

answered Feb 28 '19 at 11:52

Alex Harvey

14,494
5
61
97

Thank you, I have git-bash on my window. I have this one in git bash command windows `YMa@Yong21-PC MINGW64 ~ $` so shoul I write your code after $? – Ma Y Feb 28 '19 at 12:16
what is `▶ cat` and etc. sorry for such a questions. – Ma Y Feb 28 '19 at 12:17
and could I ask you how to install gnu parallel on windoes? – Ma Y Feb 28 '19 at 12:23
The ▶ is my prompt. Ignore that. Install Git Bash, and then type everything exactly as you see it in my answer, with the ▶ character. As for installing Git Bash, did you follow the link to the related Stack Overflow question where there are instructions? – Alex Harvey Feb 28 '19 at 12:35
no because it does not open for me. I just had git bash before. but I do not know how to insall gnu plot using that – Ma Y Feb 28 '19 at 12:42
Sorry. Here are some instructions on installing Git Bash https://www.youtube.com/watch?v=rWboGsc6CqI – Alex Harvey Feb 28 '19 at 12:44
And then follow the instructions here to install GNU parallel https://stackoverflow.com/a/52451011/3787051 – Alex Harvey Feb 28 '19 at 12:46
yes I saw it before and installed git bash. Now I am asking how to install Gnu parallel. because I found one using sudo apt – Ma Y Feb 28 '19 at 12:46
`bash: 10seconds_install: No such file or directory ` – Ma Y Feb 28 '19 at 12:47
`$ wget https://git.savannah.gnu.org/cgit/parallel.git/plain/10seconds_install bash: wget: command not found` – Ma Y Feb 28 '19 at 12:48
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/189202/discussion-between-alex-harvey-and-ma-y). – Alex Harvey Feb 28 '19 at 12:51

score 1 · Answer 3 · answered Feb 28 '19 at 13:03

A quite easy way to run parallel jobs of every kind is using nohup. This redirect the output to a file call nohup.out (by default). In your case you should just write:

nohup python script1.py > output_script1 &
nohup python script2.py > output_script2 &

That's it. With nohup you can also logout and the script will be continuing until they have finished

Running two python scripts with bash file

3 Answers3