Running shell script from python

Question

I have written a small python programme which writes a C-shell script, and then calls the script with subprocess. However, python fails to execute the script. I tested this in the terminal and discovered that the script will also not run from the terminal (i.e. not a python issue). Further testing revealed I could write an identical script in a text editor (vi) and then successfully execute it from the terminal (i.e. I don't think the contents of the script are at fault). Eventually I found that if I tried to run the output from python directly in the terminal with ./myscript.com it would not run. However, if I open with vi, make NO changes, save and then run from the terminal using ./myscript.com (i.e. the exact same command) it will execute. The script is included below, in case it is of use, however, most of the contents are very specific to the particular programme I am trying to run. I have no idea how to debug this further, or why the effect of opening, saving and closing in vi is sufficient to allow the script to be executed. (I have checked the permissions for example and these are unchanged after the vi step.) Any suggestions on how to proceed which be much appreciated.

#!/bin/csh
#
bin2pipe -in /full/path/to/this/script/recon.spc -bad 0.0 -noswap \
-xN 1714 \
-xT 1714 \
-xFT Freq \
-xMODE Real \
-xSW 4184.570312 \
-xCAR 10.929523 \
-xOBS 800.130005 \
-xLAB 1H \
-yN 2048 \
-yT 2048 \
-yFT Freq \
-yMODE Real \
-ySW 1700.680054 \
-yCAR 125.728023 \
-yOBS 81.075996 \
-yLAB 1H \
-ndim 2 \
|nmrPipe -fn TP -auto \
|nmrPipe -fn REV \
|nmrPipe -fn TP -auto \
-out recon.spc.pipe.ft2 -ov -verb

From python I call it as `subprocess.call(FileName)`, having previously set the file permissions so it is executible with `os.chmod(FileName, 0o755)` One thing I have noticed is that opening, saving and closing in vi adds 1 byte to the file size (492 to 493 bytes). I don't know if this is significant, or where this extra byte is being added. — 218, Aug 28 '16 at 15:21
You could compare the two files byte for byte and see what is different. Are the *newline*'s equivalent? Unicode versus ascii? — wwii, Aug 28 '16 at 15:46
Changing the shebang to `#!/bin/sh` might not help, but could be highly recommendable for other reasons. — tripleee, Aug 28 '16 at 17:00
You need to call the file properly `subprocess.call(["/bin/csh", "/full/path/to/file"])` — Burhan Khalid, Aug 28 '16 at 17:17
The software I'm trying to call has to use the C-shell hence not using /bin/sh. I have tried `subprocess.call(["/bin/csh", "/full/path/to/file"])` but the process still isn't run. How can I check whether the newlines are equivalent? — 218, Aug 28 '16 at 17:37
Perhaps it doesn't run because the file paths are relevant in your script, and the paths should be absolute; so that when its run from anywhere, it can find the `recon.spc` file. — Burhan Khalid, Aug 28 '16 at 19:02
I actually included the full file paths in the script I made. I just removed them for the purpose of posting - I'll edit the post. However, as in the question the script won't run even from the terminal in the directory where the files are present. But opening, saving and closing (:wq) is sufficient to allow me then to run it. I don't understand what difference this can be making, or how I can decipher what vi is doing. — 218, Aug 28 '16 at 22:07
Copy the file before editing it. Run `xxd` on the copy and the modified file. Compare. — tripleee, Aug 29 '16 at 14:26

score 0 · Answer 1 · edited May 23 '17 at 10:33

In case this is useful I used xxd (as suggested in the comments above, http://linuxcommand.org/man_pages/xxd1.html) to convert both script files, one created in python and one written manually, to hex

xxd script.txt output.txt

I then compared the output for the two files with diff and could see an additional character 0a in hex at the end of the file. This corresponds to a newline. As detailed here Why should text files end with a newline? and here VIM Disable Automatic Newline At End Of File, vi adds a newline character at the end of the file if there isn't one. This was enabling the script to run. Without the newline character at the end (which I hadn't included in the python script), the script wouldn't run.

Running shell script from python

1 Answers1