How to run several scripts in differents folders on Python

Question

I have to run a lot of scripts for paper, and I would like to make this automatic. I have several folders (P1, P2,..., PN) in which I have a script ( test1, test2, ... testN) and I need to run all this, but doing on by one by myself I waste a lot of time that I don't have!

I tried subprocess:

where P1_T1 is:

for i in range(5):
    x = i+2*i
   
    print(x)

and P1_T2 is:

for i in range(5):
    x = i+3*i
    print(x)

But it didn't work.

you need them to run simultaneously is what you mean by it didn’t work? — gold_cy, Sep 15 '19 at 15:39
What do you mean by "didn't work"? Did you get an error? If so, what was it? Thanks for clarifying. — ggorlen, Sep 15 '19 at 15:40
Also, instead of writing each `Popen` by hand, you should use a loop instead. — Arnav Borborah, Sep 15 '19 at 15:41
@aws_apprentice I don't need to run simultaneously, I just need to run one by one, but this way is not showing any result on the screen, this was just a test, in my real code I have a .txt file as output, and has not been produced with this subprocess code — JCV, Sep 16 '19 at 14:29

Enthus3d · Accepted Answer · 2019-09-16T15:14:15.047

1

If you want to recurse through a set of directories, I recommend using os.walk. This implementation should attempt to run POpen 'python [filename]' on every file in your root directory:

import os
import importlib.util


path = "C:\\SO\\testfolder" # <--- replace this with the path to the folder containing all of your p1, p2, p3, p4 folders.

for root, subdirs, files in os.walk(path):
    for file in files:
        file_path = os.path.join(root, file)
        filename, file_extension = os.path.splitext(file_path)
        if file_extension == ".py":
            print("Now Executing: " + filename + "-----------")
            spec = importlib.util.spec_from_file_location(file, file_path)
            module = importlib.util.module_from_spec(spec)
            spec.loader.exec_module(module)

Edit: Added in use of import library+exec_module to run the python files. Import method referenced from here.

edited Sep 16 '19 at 15:14

answered Sep 15 '19 at 15:56

Enthus3d

1,727
12
26

Is there another way to run a script from a folder without POpen? – JCV Sep 16 '19 at 14:47
This code didn't work, I got the error: raise TypeError("bufsize must be an integer") TypeError: bufsize must be an integer – JCV Sep 16 '19 at 14:49
I am looking into using importlib to run it, give me a moment to test it – Enthus3d Sep 16 '19 at 15:03
I figure out what is, it was missing [ ]; This way run without erros but doesnt shows the results : import subprocess import os path = 'C:/Users/Familia/OneDrive - Università di Cagliari/PhD/simpegmaster/Notebooks/Testes_Jenny/TEMFAST/TEMFAST Work/Castanheira/Teste' for root, subdirs, files in os.walk(path): for file in files: subprocess.run(['python',os.path.join(root,file)]) – JCV Sep 16 '19 at 15:04
That's good, I found a way to run so that print statements are still done, give me a moment to finish testing it and I'll update my answer – Enthus3d Sep 16 '19 at 15:11
I made an improved version, by using some code for import lib + exec. Try running this version, it should hopefully allow you to run and get the proper outputs – Enthus3d Sep 16 '19 at 15:14
1

Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/199530/discussion-between-jeniffer-barreto-and-enthus3d). – JCV Sep 16 '19 at 15:58
Hopefully the rest of your paper/project goes well! – Enthus3d Sep 16 '19 at 16:17
No problem :). Glad I could help you out. If you ever need me to go into more detail about how the implementation works, feel free to comment/use the chat. – Enthus3d Sep 16 '19 at 18:11

score 0 · Answer 2 · answered Sep 15 '19 at 15:40

0

Popen expects list of arguments, so what about

import subprocess
from subprocess import Popen

Popen(['python', 'p1/P1_T1.py'])
time.sleep(1)
Popen(['python','p2/P2_T1.py'])

Does this work for you? There is nice tutorial - Python 3 Subprocess Examples

answered Sep 15 '19 at 15:40

Kryštof Řeháček

1,965
1
16
28

No, this doesn't work. No error, but no result, doesn't show anything on the screen. Like in the photo that I have add – JCV Sep 16 '19 at 14:24

How to run several scripts in differents folders on Python

2 Answers2