Python's subprocess.Popen() and source

Question

A third-party Python 2.5 script I'm trying to debug has got me stymied. The relevant part of the script is:

       proc = subprocess.Popen(
               "ls && source houdini_setup",
               shell = True,
               executable = "/bin/bash",
               )

There is a daemon that listens to port 5001 and runs the above script. When the script runs, it fails with the following error:

_cygwin.py
houdini_setup
... (more files) ...
/bin/sh: line 0: source: houdini_setup: file not found

There very much exists a file houdini_setup, as illustrated by the ls, and in fact if I change "source" to "cat" in the script above, the script prints the contents of houdini_setup as expected. Moreover, running the exact above command in a bona-fide bash shell also sources the file with no complaints.

Does anyone have any idea what's going on here?

I copy-pasted it from the code I'm trying to debug; deleting the line `stdin = null;` unfortunately makes no difference. — user168715, Jan 10 '10 at 21:04

score 8 · Accepted Answer · answered Jan 10 '10 at 21:03

8

source uses $PATH to find what you pass to it, if you don't specify a directory. Try source ./houdini_setup.

answered Jan 10 '10 at 21:03

Ignacio Vazquez-Abrams

776,304
153
1,341
1,358

That did the trick, thanks! In a real bash shell, source doesn't need a ./ for files in the current directory, even when $PATH does not include said directory. Any idea why it's needed in Popen()? – user168715 Jan 10 '10 at 21:09
It may have to do with the fact that the shell executing was actually sh, not bash, despite the line passed to the function call. – LeafStorm Jan 10 '10 at 21:14
It has been my experience that it *is* needed. Always. – Ignacio Vazquez-Abrams Jan 10 '10 at 21:15
Even if it isn't needed, it should be. Having . in the path is a configuration problem, and can lead to all kinds of badness and confusion. – Andrew McGregor Jan 10 '10 at 21:49
@Andrew That argument only really applies to the executable PATH. There is no standard search path used by the source command any more than there is for e.g. ls. But while no ./ is needed in bash, it is needed by dash, which is these days commonly used as the implementation of the sh alias. Not sure why sh is being used despite the executable argument being passed in the OP question, though. – andybuckley Sep 19 '13 at 13:56

score 2 · Answer 2 · answered Jan 10 '10 at 21:43

You can use strace to help diagnose problems like this. In this case running it with the -ff flag to follow subprocess would have revealed that your code was looking for "houdini_setup" in the wrong place:

stat64("/usr/local/bin/houdini_setup", 0xbf989e60) = -1 ENOENT (No such file or directory)
stat64("/usr/bin/houdini_setup", 0xbf989e60) = -1 ENOENT (No such file or directory)
stat64("/bin/houdini_setup", 0xbf989e60) = -1 ENOENT (No such file or directory)

That could have led you to check the documentation for the source builtin.

score 0 · Answer 3 · answered Dec 06 '12 at 22:42

0

Maybe you can just use bash itself to do the job, if it's available on your system:

os.popen("/bin/bash -l -c 'source $HOME/.yourfiletobesourced'")

answered Dec 06 '12 at 22:42

RubenCaro

1,419
14
12

2

You should still use the `subprocess` module to do that (with the `shell=True` keyword set). – Martijn Pieters Dec 06 '12 at 23:01

Python's subprocess.Popen() and source

3 Answers3