Run all Python scripts in a folder - from Python

Question

How can I write a Python program that runs all Python scripts in the current folder? The program should run in Linux, Windows and any other OS in which python is installed.

Here is what I tried:

import glob, importlib

for file in glob.iglob("*.py"):
    importlib.import_module(file)

This returns an error: ModuleNotFoundError: No module named 'agents.py'; 'agents' is not a package (here agents.py is one of the files in the folder; it is indeed not a package and not intended to be a package - it is just a script).

If I change the last line to:

    importlib.import_module(file.replace(".py",""))

then I get no error, but also the scripts do not run.

Another attempt:

import glob, os

for file in glob.iglob("*.py"):
    os.system(file)

This does not work on Windows - it tries to open each file in Notepad.

score 0 · Accepted Answer · answered May 30 '19 at 12:34

0

You need to specify that you are running the script through the command line. To do this you need to add python3 plus the name of the file that you are running. The following code should work

import os
import glob

for file in glob.iglob("*.py"):
    os.system("python3 " + file)

If you are using a version other than python3, just change the argument from python3 to python

answered May 30 '19 at 12:34

Thomas Hayes

102
6

What if one platform has python3 while another platform has python.exe? Is it possible to get the executable of the current python interpreter? – Erel Segal-Halevi May 30 '19 at 12:54
1

That is possible with `sys.executable`. – SourceSimian May 30 '19 at 12:57
I believe that you can do that with `sys.executable`. For more information, see https://docs.python.org/3/library/sys.html#sys.executable – Thomas Hayes May 30 '19 at 13:00
@ErelSegal-Halevi you can just call "python", '/usr/bin/python' is the symbolic link to installed version of python – gjois Aug 05 '21 at 10:47

score 0 · Answer 2 · answered May 30 '19 at 12:39

Maybe you can make use of the subprocess module; this question shows a few options.

Your code could look like this:

import os
import subprocess

base_path = os.getcwd()
print('base_path', base_path)

# TODO: this might need to be 'python3' in some cases
python_executable = 'python'
print('python_executable', python_executable)

py_file_list = []
for dir_path, _, file_name_list in os.walk(base_path):
    for file_name in file_name_list:
        if file_name.endswith('.csv'):
            # add full path, not just file_name
            py_file_list.append(
                os.path.join(dir_path, file_name))

print('PY files that were found:')
for i, file_path in enumerate(py_file_list):
    print('   {:3d} {}'.format(i, file_path))

    # call script
    subprocess.run([python_executable, file_path])

Does that work for you?

Note that the docs for os.system() even suggest using subprocess instead:

The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function.

score 0 · Answer 3 · answered May 30 '19 at 12:49

If you have control over the content of the scripts, perhaps you might consider using a plugin technique, this would bring the problem more into the Python domain and thus makes it less platform dependent. Take a look at pyPlugin as an example.

This way you could run each "plugin" from within the original process, or using the Python::multiprocessing library you could still seamlessly use sub-processes.

Run all Python scripts in a folder - from Python

3 Answers3