427

I want to run a Python script from another Python script. I want to pass variables like I would using the command line.

For example, I would run my first script that would iterate through a list of values (0,1,2,3) and pass those to the 2nd script script2.py 0 then script2.py 1, etc.

I found Stack Overflow question 1186789 which is a similar question, but ars's answer calls a function, where as I want to run the whole script, not just a function, and balpha's answer calls the script but with no arguments. I changed this to something like the below as a test:

execfile("script2.py 1")

But it is not accepting variables properly. When I print out the sys.argv in script2.py it is the original command call to first script "['C:\script1.py'].

I don't really want to change the original script (i.e. script2.py in my example) since I don't own it.

I figure there must be a way to do this; I am just confused how you do it.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Gern Blanston
  • 42,482
  • 19
  • 50
  • 64
  • 1
    The question is if you know the name of the script (then import it) or if you do not know the name of the script at programming time (then use subprocess.call). In the second case this question also would not be a duplicate. Because the question doesn't make it clear, it's also not really a good one. – NoDataDumpNoContribution Sep 03 '15 at 11:37
  • @Trilarion: wrong. You can import a python module even if its name is generated at runtime. – jfs Nov 29 '15 at 08:33
  • @J.F.Sebastian Okay. As a side remark: This way is not really covered well by any answer here or in the linked question, except partly in http://stackoverflow.com/a/1186840/1536976. – NoDataDumpNoContribution Nov 29 '15 at 10:05
  • 1
    @Trilarion: why should it be covered at all? The names are fixed in both questions. Anyway, the statement *"if you do not know the name of the script at programming time (then use subprocess.call)."* is wrong regardless. If you have a new question; [ask](http://stackoverflow.com/questions/ask). – jfs Nov 29 '15 at 10:23
  • @J.F.Sebastian Agreed again. I don't have any question right now. If I come up with one I will surely ask. – NoDataDumpNoContribution Nov 29 '15 at 10:24
  • 1
    @Oli4 "definitely" is a strong word. Care to elaborate? I see subprocess.call() solution that accepts passing multiple command line arguments. I see `import` being mentioned for cases where main() function is defined (it won't help OP but it is the right way for many other people with a similar problem). I see execfile() for Python 2 that uses whatever you put into `sys.argv` (admittedly that last bit is not mentioned explicitly) — this option should be ignored by beginners. There is even an explicit os.system() answer with multiple arguments (the answer that is accepted here). – jfs Aug 10 '17 at 16:57
  • @J.F.Sebastian After your elaboration, I am tempted to agree with you. Even though the other question does not explicitly specify how to send arguments or that it is required, the questions are indeed similar. Sometimes by asking a question differently different solutions appear as ChrisAdams's answer solved my problem. I apologize for my earlier statement saying that this is indeed not a duplicate, and thank you. – Oli4 Aug 10 '17 at 20:10
  • 2
    If os.system isn't powerful enough for you, there's [the subprocess module](http://docs.python.org/library/subprocess.html#module-subprocess). – nmichaels Sep 23 '10 at 19:37

6 Answers6

415

Try using os.system:

os.system("script2.py 1")

execfile is different because it is designed to run a sequence of Python statements in the current execution context. That's why sys.argv didn't change for you.

Greg Hewgill
  • 951,095
  • 183
  • 1,149
  • 1,285
  • 67
    I believe it's generally preferable to use `subprocess.Popen` over `os.system`: http://docs.python.org/library/subprocess.html#replacing-os-system. – Katriel Sep 23 '10 at 20:15
  • 14
    Yes, that's what the help for `os.system` says. However, for simple uses `os.system` is the simplest way to get the job done. It depends on what your needs are, of course. – Greg Hewgill Sep 23 '10 at 20:39
  • 2
    You can also use subprocess.check_call(). – MarioVilas Mar 13 '13 at 11:01
  • 39
    You have to add python to run the script: os.system("python script2.py 1") – Andromida Jan 02 '14 at 13:10
  • So if I wanted to start 5 threads, I'd just paste that line 5 times into a file, then call that file from cron? – User Jan 27 '14 at 01:51
  • 13
    @macdonjo: No, the `os.system()` call waits until the thing you called finishes before continuing. You could use `subprocess.Popen()` and manage the new processes yourself, or use the `multiprocessing` module, or various other solutions. – Greg Hewgill Jan 27 '14 at 02:58
  • This waits for the callback though, I think you need to use subprocesss – radtek Jan 27 '15 at 19:17
  • @GregHewgill - Is there any dynamic way to pass variable's `value` in `os.system("script2.py myvariable")` ? (Note: myvariable keeps changing, so can't pass static value) – H. U. Jan 26 '16 at 00:42
  • @H.U.: Please see [Passing a variable, set by the user, as a parameter for a os.system() command](https://stackoverflow.com/questions/8715419/passing-a-variable-set-by-the-user-as-a-parameter-for-a-os-system-command) for specific help with that. – Greg Hewgill Jan 26 '16 at 00:45
  • 13
    I encounter the issue `sh: 1: *.py: not found`. Add **python** and then it works, i.e., `os.system("python script2.py 1")`. – SparkAndShine Apr 21 '16 at 21:48
  • 2
    @SparkandShine: In the original question, `script2.py` is executable. If it's not executable, you will need to do what you say. Better would be to use `sys.executable`, which is the name of the *current* Python executable (in case it is different from the default one). So something like: `os.system(sys.executable + " script2.py 1")` – Greg Hewgill Apr 21 '16 at 21:54
  • @GregHewgill thx, for me, the permission of the python script is `755` and `script2.py` begins with `#!/usr/bin/env python`. BTW, `os.system(sys.executable + " script2.py 1")` also occurs `sh: 1: *.py: not found`. – SparkAndShine Apr 21 '16 at 22:03
  • how can I pass location as an argument in my case the file has to be executed with location like this `CapGenVoice.py -i "C:\Users\Ajax\proDem1\a (4).jpg"` ? – Ajax Mar 14 '21 at 13:38
  • `os.system("python3 script2.py")` actually worked for me, `os.system("script2.py 1")` was giving me weird errors – kiradotee Mar 15 '21 at 11:44
  • Do you know more about reentrance of python? How much instances could I run?. I oberseved crashes in 3.5.2_32bit. – Nikolai Ehrhardt Mar 09 '22 at 07:42
  • 1
    but will this still work if we convert our program to `.exe` and run on a machine which does not have `python` installed ? – Belal Ahmed Aug 02 '22 at 16:06
136

This is inherently the wrong thing to do. If you are running a Python script from another Python script, you should communicate through Python instead of through the OS:

import script1

In an ideal world, you will be able to call a function inside script1 directly:

for i in range(whatever):
    script1.some_function(i)

If necessary, you can hack sys.argv. There's a neat way of doing this using a context manager to ensure that you don't make any permanent changes.

import contextlib
@contextlib.contextmanager
def redirect_argv(num):
    sys._argv = sys.argv[:]
    sys.argv=[str(num)]
    yield
    sys.argv = sys._argv

with redirect_argv(1):
    print(sys.argv)

I think this is preferable to passing all your data to the OS and back; that's just silly.

Katriel
  • 120,462
  • 19
  • 136
  • 170
  • 27
    It might be the "wrong" thing to do but what if the script you need to reference has no main or functions.... an import would execute the script at the time of import, which probably isn't what you want (and you don't want to refactor it because people are also using that script as is). os/subprocess could deal with such a case – Dan S Aug 21 '12 at 15:33
  • True... but that's a different question IMO! – Katriel Aug 21 '12 at 16:08
  • 4
    Won't that fail for multithreaded scripts? – MarioVilas Mar 13 '13 at 11:01
  • @MarioVilas probably, yes. – Katriel Mar 13 '13 at 11:42
  • Your method doesn't work for multithreading. I'm trying to run the same file 10 times at the same time. – User Jan 27 '14 at 01:43
  • @DanS: it does work. You could [use `multiprocessing`](http://stackoverflow.com/q/30076185/4279) – jfs Nov 29 '15 at 08:36
  • 14
    From a higher-level perspective, it's "wrong" to call this strategy "wrong". When a system scales up, having detailed knowledge of the inner workings of components becomes a larger and larger problem -- a program written in Python might get redesigned in another language, and using the generic process communication facilities of the operating system has much to offer as a strategy. – Stabledog Jul 28 '18 at 21:43
  • 1
    It's worth to note that you may import right in the middle of the code, it doesn't have to be placed at the top, @DanS – user Oct 11 '18 at 01:29
  • 2
    ... and what about dependencies? If each script has its own dependencies why the first script should to know all the sencond's script dependencies? From the point of view of architecture, what is wrong is to call directly importing in python (IMHO) – 4lberto Feb 09 '19 at 21:25
  • Also, for neatness, I like to have my scripts in a `bin` directory instead of strewn all around the root directory of the project. In order to import scripts from `bin`, I'd have to add an `__init__.py` to the `bin` directory which seems wrong. – Shankari Mar 20 '20 at 05:10
  • 1
    What do you mean by `permanent changes`? – alper Aug 07 '21 at 21:45
136

Ideally, the Python script you want to run will be set up with code like this near the end:

def main(arg1, arg2, etc):
    # do whatever the script does


if __name__ == "__main__":
    main(sys.argv[1], sys.argv[2], sys.argv[3])

In other words, if the module is called from the command line, it parses the command line options and then calls another function, main(), to do the actual work. (The actual arguments will vary, and the parsing may be more involved.)

If you want to call such a script from another Python script, however, you can simply import it and call modulename.main() directly, rather than going through the operating system.

os.system will work, but it is the roundabout (read "slow") way to do it, as you are starting a whole new Python interpreter process each time for no raisin.

kindall
  • 178,883
  • 35
  • 278
  • 309
  • 22
    Re: "no raisin." This is not an error. However, it was interesting to see how long it would take for someone unfamiliar with *Futurama* to "correct" it in a random Stack Overflow question: two years and three months. :-) – kindall Dec 29 '12 at 18:30
  • I laughed at "no raisin" simply because it was a ridiculous typo, then saw your comment and found a clip on YouTube. Even funnier. – armani May 20 '13 at 15:51
  • I had a script with no main, but was able to introduce like so: 1. Separate function defs and standalone statements 2. Do the argument parsing under the "if __name__..." section 3. indent the rest of statements under def main(...) and have that logic operate on the method parameters. Then I was able to call the main method from another script (ex: A unit test) – Stan Kurdziel May 28 '13 at 18:00
  • Yeah, Python really makes that kind of refactoring pretty easy. I have done similar things more than once myself! – kindall May 28 '13 at 18:06
  • This approach doesn't work for me if the main program has optional parameters. Here I get "list index out of range". Better have a `Main(sys.argv)` and handle the list of arguments in Main – 576i Nov 11 '14 at 08:48
  • "This approach doesn't work for me if the main program has optional parameters" -- if your parsing is at all complicated you should be using the `argparse` module anyway. – kindall Mar 10 '15 at 17:56
  • 2
    Here we are again nearly 5 years later, and I was delighted to find your quote from that Fry-befuddled brain from Futurama. – Jimmy Sep 04 '15 at 21:29
  • 1
    whoa, 2 years...an unraisinable length of time! – MarcTheSpark Sep 23 '21 at 17:33
  • 1
    I just saw this in 2023 and it is still riddled with plot holes and spelling errors. – dmedine Apr 17 '23 at 01:09
62

I think the good practice may be something like this; 

import subprocess
cmd = 'python script.py'

p = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)
out, err = p.communicate() 
result = out.split('\n')
for lin in result:
    if not lin.startswith('#'):
        print(lin)

according to documentation The subprocess module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module intends to replace several older modules and functions:

os.system
os.spawn*
os.popen*
popen2.*
commands.*

Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process. Read Here

Medhat
  • 1,622
  • 16
  • 31
52

SubProcess module:
http://docs.python.org/dev/library/subprocess.html#using-the-subprocess-module

import subprocess
subprocess.Popen("script2.py 1", shell=True)

With this, you can also redirect stdin, stdout, and stderr.

Chris Adams
  • 1,067
  • 8
  • 15
  • 13
    Do not use `shell=True` unless necessary. – Piotr Dobrogost Nov 03 '13 at 10:39
  • 1
    @PiotrDobrogost could you elaborate on that? Edit: For anyone curious, here's an explanation on why: https://stackoverflow.com/questions/3172470/actual-meaning-of-shell-true-in-subprocess – emilaz Feb 13 '20 at 17:10
41
import subprocess
subprocess.call(" python script2.py 1", shell=True)
Duncan Jones
  • 67,400
  • 29
  • 193
  • 254
Nikos
  • 427
  • 4
  • 2