77

I'd like to embed the text of short python scripts inside of a bash script, for use in say, my .bash_profile. What's the best way to go about doing such a thing?

The solution I have so far is to call the python interpreter with the -c option, and tell the interpreter to exec whatever it reads from stdin. From there, I can build simple tools like the following, allowing me to process text for use in my interactive prompt:

function pyexec() {
    echo "$(/usr/bin/python -c 'import sys; exec sys.stdin.read()')"
}

function traildirs() {
    pyexec <<END
trail=int('${1:-3}')
import os
home = os.path.abspath(os.environ['HOME'])
cwd = os.environ['PWD']
if cwd.startswith(home):
    cwd = cwd.replace(home, '~', 1)
parts = cwd.split('/')
joined = os.path.join(*parts[-trail:])
if len(parts) <= trail and not joined.startswith('~'):
    joined = '/'+joined
print joined
END
}

export PS1="\h [\$(traildirs 2)] % "

This approach smells slightly funny to me though, and I'm wondering what alternatives to doing it this way might be.

My bash scripting skills are pretty rudimentary, so I'm particularly interested to hear if I'm doing something silly from the bash interpreter's perspective.

Matt Anderson
  • 19,311
  • 11
  • 41
  • 57
  • can you say more clearly what is it you are actually trying to do? from what i see , Python is not really needed. you can do most things with the shell. – ghostdog74 Feb 03 '10 at 01:55
  • 1
    @ghostdog74: nothing really deeper than I was saying; I'm just a much better python programmer than a bash programmer, and IMO python is more powerful, in general, than bash. It might be handy to implement functionality used in a bash script in python, and sometimes not depend on external files when doing so. I'm finally making the switch from tcsh to bash (after 15 years), and I'm trying to bend the shell to my will/preferences. – Matt Anderson Feb 03 '10 at 03:15
  • -1: Why not simply create a .py module file? Why force the Python into a shell script when a better solution is (usually) to stop using the shell entirely? – S.Lott Feb 03 '10 at 03:19
  • 3
    @S.Lott: In my case, I am running bash scripts as Alfred commands but need the split functionality of Python. Creating a .py file adds unnecessary overhead. – Chris Redford Mar 25 '13 at 21:24

9 Answers9

63

Why should you need to use -c? This works for me:

python << END
... code ...
END

without needing anything extra.

Ricardo Cárdenes
  • 9,004
  • 1
  • 21
  • 34
  • 4
    To collect output into a variable, I did `ABC=$(python << END` ... `END)` and it worked. – Evgeni Sergeev Feb 06 '14 at 05:53
  • 1
    What if we want to communicate to python a string parameter coming from bash script? – Augustin Riedinger Feb 04 '15 at 22:24
  • 2
    Then you need the `-`, as in Ned's answer. The arguments would follow the dash. – Ricardo Cárdenes Feb 04 '15 at 23:41
  • 8
    Be careful how you name the heredoc marker. When it's unquoted as in your example, shell expansion will occur within the heredoc string. For example, if your python code consisted of just `print '$SHELL'`, the output would be `/bin/bash` or whatever your shell happens to be. If you change the first line to `python - <<'END'`, the output would be `$SHELL`. If you are copying and pasting existing code to embed into a bash script, you almost certainly don't want shell substitutions -- you want the code to run the same way it does when it's not embedded in a bash script, right? – Chris Johnson Aug 17 '15 at 19:02
  • what you suggest in your comment does not work for me. Putting a=$(python << END b=1 END) gives me error: """./temp.sh: line 4: warning: here-document at line 4 delimited by end-of-file (wanted END')\n./temp.sh: line 3: warning: here-document at line 3 delimited by end-of-file (wanted END')./temp.sh: line 3: warning: here-document at line 3 delimited by end-of-file (wanted `END')\npython: can't open file 'b=1': [Errno 2] No such file or directory""" – user3342981 Jul 01 '19 at 20:43
  • 1
    @user3342981 If you have everything in a single line, as it looks like, that's not going to work: https://stackoverflow.com/questions/18660798/here-document-gives-unexpected-end-of-file-error – Ricardo Cárdenes Jul 04 '19 at 01:58
  • @RicardoCárdenes : Can you help on this question , posted on stackoverflow : https://stackoverflow.com/questions/68002623/how-to-pass-parameter-to-python-script-from-unix?noredirect=1 – codeholic24 Jun 19 '21 at 10:17
  • For all looking at how to pass arguments look at Ned's answer with `python - arg1 arg2 < – Error404 Jan 03 '23 at 15:18
41

The python interpreter accepts - on the command line as a synonym for stdin so you can replace the calls to pyexec with:

python - <<END

See command line reference here.

Ned Deily
  • 83,389
  • 16
  • 128
  • 151
  • 1
    Huh -- I had done a text search on the python man page, and the text `stdin` isn't in there. Looking more closely though, "standard input" is. Doh. – Matt Anderson Feb 03 '10 at 01:54
  • Agreed the heredoc approach can work. There are things to be careful about however: (1) if the heredoc marker (`END` in Ned's example) occurs in column 0 in the Python script, the heredoc will end early; (2) variations on heredoc marker (with or without leading `-`) affect whether tabs and spaces are preserved; and (3) syntactic overlap between Python and bash when variable expansion is turned on (heredoc marker not quoted), e.g. `${1}` could appear legitimately in a Python string format specifier, but would be replaced with a bash command line parameter if the heredoc marker isn't quoted. – Chris Johnson Mar 24 '14 at 03:09
  • 1
    Be careful how you name the heredoc marker. When it's unquoted as in your example, shell expansion will occur within the heredoc string. For example, if your python code consisted of just `print '$SHELL'`, the output would be `/bin/bash` or whatever your shell happens to be. If you change the first line to `python - <<'END'`, the output would be `$SHELL`. If you are copying and pasting existing code to embed into a bash script, you almost certainly don't want shell substitutions -- you want the code to run the same way it does when it's not embedded in a bash script, right? – Chris Johnson Aug 17 '15 at 19:01
  • 2
    A full example would be nice. – kev Jan 19 '18 at 07:14
  • Ricardo Cárdenes's answer below has a more full example, FYI: https://stackoverflow.com/a/2189116/1157440 – micseydel Dec 06 '19 at 17:39
  • @Ned Deily : Can you help on this question , posted on stackoverflow : https://stackoverflow.com/questions/68002623/how-to-pass-parameter-to-python-script-from-unix?noredirect=1 – codeholic24 Jun 19 '21 at 10:18
30

One problem with using bash here document is that the script is then passed to Python on stdin, so if you want to use the Python script as a filter, it becomes unwieldy. One alternative is to use the bash's process substitution, something like this:

... | python <( echo '
code here
' ) | ...

If the script it too long, you could also use here document inside the paren, like this:

... | python <(
cat << "END"
code here
END
 ) | ...

Inside the script, you can read/write as you normally would from/to standard i/o (e.g., sys.stdin.readlines to gobble up all the input).

Also, python -c can be used as mentioned in other answers, but here is how I like to do it to format nicely, while still respecting Python's indentation rules (credits):

read -r -d '' script <<-"EOF"
    code goes here prefixed by hard tab
EOF
python -c "$script"

Just make sure that the first character of each line inside here document is a hard tab. If you have to put this inside a function, then I use the below trick that I saw somewhere to make it look aligned:

function somefunc() {
    read -r -d '' script <<-"----EOF"
        code goes here prefixed by hard tab
----EOF
    python -c "$script"
}
haridsv
  • 9,065
  • 4
  • 62
  • 65
  • 3
    Why 0 upvotes? It it the only correct answer here, if you want to use sys.stdin in your program! – Igor Chubin Jun 16 '16 at 11:22
  • This is by far the best solution to date! I would also add that you probably want to quote the END in the second example: `cat <<'END'` to avoid having to escape $ chars and other special chars in the HERE document. – Michael Goldshteyn Aug 23 '18 at 14:14
  • @MichaelGoldshteyn Done! – haridsv Aug 23 '18 at 16:11
  • Can someone help me here. It's not working `#!/system/bin/bash function parse_plex() { read -r -d '' script <<-"----EOF" import requests requests.get('https://google.com') ----EOF python -c $script } parse_plex` File "", line 1 import ^ SyntaxError: invalid syntax – user5507535 May 11 '19 at 15:49
  • @user5507535 You need to put double-quotes around `$script`, e.g., `python -c "$script"`. I will update the answer. – haridsv May 17 '19 at 12:25
  • @haridsv : Can you help on this question , posted on stackoverflow : https://stackoverflow.com/questions/68002623/how-to-pass-parameter-to-python-script-from-unix?noredirect=1 – codeholic24 Jun 19 '21 at 10:17
  • To pass command line arguments into the script, update to: `python -c "$script" $@` – Contango Nov 25 '21 at 14:52
9

Sometimes it is not a good idea to use here document. Another alternative is to use python -c:

py_script="
import xml.etree.cElementTree as ET,sys
...
"

python -c "$py_script" arg1 arg2 ...
Don Li
  • 1,095
  • 2
  • 12
  • 17
  • Nice and simple, and lets you move the python code away from the middle of the rest of the bash code. Also lets your python code read from stdin. – dbort Jan 13 '16 at 01:35
  • @Don Li Can you help on this question , posted on stackoverflow : https://stackoverflow.com/questions/68002623/how-to-pass-parameter-to-python-script-from-unix?noredirect=1 – codeholic24 Jun 19 '21 at 10:17
7

If you need to use the output of the python in the bash script you can do something like this:

#!/bin/bash

ASDF="it didn't work"

ASDF=`python <<END
ASDF = 'it worked'
print ASDF
END`

echo $ASDF
desgua
  • 182
  • 1
  • 7
  • 1
    Be careful how you name the heredoc marker. When it's unquoted as in your example, shell expansion will occur within the heredoc string. For example, if your python code consisted of just `print '$SHELL'`, the output would be `/bin/bash` or whatever your shell happens to be. If you change the first line to `python - <<'END'`, the output would be `$SHELL`. If you are copying and pasting existing code to embed into a bash script, you almost certainly don't want shell substitutions -- you want the code to run the same way it does when it's not embedded in a bash script, right? – Chris Johnson Aug 17 '15 at 19:02
  • @desgua : Can you help on this question , posted on stackoverflow : https://stackoverflow.com/questions/68002623/how-to-pass-parameter-to-python-script-from-unix?noredirect=1 – codeholic24 Jun 19 '21 at 10:16
6

Ready to copy-paste example with input:

input="hello"
output=`python <<END
print "$input world";
END`

echo $output
AhHatem
  • 1,415
  • 3
  • 14
  • 23
Adrian Lopez
  • 2,601
  • 5
  • 31
  • 48
  • Can you help on this question , posted on stackoverflow : https://stackoverflow.com/questions/68002623/how-to-pass-parameter-to-python-script-from-unix?noredirect=1 – codeholic24 Jun 19 '21 at 10:16
  • @AhHatem : Can you help on this question , posted on stackoverflow : https://stackoverflow.com/questions/68002623/how-to-pass-parameter-to-python-script-from-unix?noredirect=1 – codeholic24 Jun 19 '21 at 10:16
2

Interesting... I want an answer now too ;-)

He is not asking how to execute python code within a bash script but actually have the python set environment variables.

Put this in a bash script and try to get it to say "it worked".

export ASDF="it didn't work"

python <<END
import os
os.environ['ASDF'] = 'it worked'
END

echo $ASDF

The problem is that the python gets executed in a copy of the environment. Any changes to that environment aren't seen after python exits.

If there is a solution for this, I'd love to see it too.

eric.frederich
  • 1,598
  • 4
  • 17
  • 30
  • 1
    That isn't really what I was getting at necessarily, and I'm pretty sure you can't get environment variables from a child process (directly) or alter a parent's environment variables (directly). You *can* however `eval` the output of the python script in the bash script, thus executing arbitrary statements, setting environment variables or doing anything else you might like. – Matt Anderson Feb 03 '10 at 03:12
  • 1
    @Eric: If you really want an answer, post your own question. – Ned Deily Feb 03 '10 at 03:29
  • Matt, sorry... I briefly looked at your python script and saw you were doing stuff with os.environ and assumed that was the kind of thing you were doing. My mistake. – eric.frederich Feb 03 '10 at 03:44
  • You can manage it by combining a bash variable and the python output. See my answer bellow. – desgua Dec 24 '13 at 02:25
  • @eric.frederich Can you help on this question , posted on stackoverflow : https://stackoverflow.com/questions/68002623/how-to-pass-parameter-to-python-script-from-unix?noredirect=1 – codeholic24 Jun 19 '21 at 10:15
1

If you are using zsh you can embed Python inside the script using zsh's join and python stdin option (python -):

py_cmd=${(j:\n:)"${(f)$(
    # You can also add comments, as long as you balance quotes
    <<<'if var == "quoted text":'
    <<<'  print "great!"')}"}

catch_output=$(echo $py_cmd | python -)

You can indent the Python snippet, so it looks nicer than the EOF or <<END solutions when inside functions.

fedeDev
  • 11
  • 1
1

Try this :

#!/bin/bash
a="test"
python -c "print  '$a this is a test'.title()"
kuro
  • 3,214
  • 3
  • 15
  • 31