Executing multi-line statements in the one-line command-line

Question

I'm using Python with -c to execute a one-liner loop, i.e.:

python -c "for r in range(10): print 'rob'"

This works fine. However, if I import a module before the for loop, I get a syntax error:

python -c "import sys; for r in range(10): print 'rob'"

  File "<string>", line 1
    import sys; for r in range(10): print 'rob'
              ^
SyntaxError: invalid syntax

How can this be fixed?

It's important to me to have this as a one-liner so that I can include it in a Makefile.

Answer 1

You could do

echo -e "import sys\nfor r in range(10): print 'rob'" | python

Or without pipes:

python -c "exec(\"import sys\nfor r in range(10): print 'rob'\")"

Or

(echo "import sys" ; echo "for r in range(10): print 'rob'") | python

Or SilentGhost's answer or Crast's answer.

Answer 2

This style can be used in makefiles too (and in fact it is used quite often).

python - <<EOF
import random
for r in range(3): print(random.randint(1, 42))
EOF

Or with hard tabs:

python - <<-EOF
    import random
    for r in range(3): print(random.randint(1, 42))
EOF
# Important: Replace the indentation above w/ hard tabs.

In above case, leading TAB characters are removed too (and some structured outlook can be achieved).

Instead of EOF can stand any marker word not appearing in the here document at a beginning of a line (see also here documents in the bash man page or here).

Answer 3

The issue is not actually with the import statement. It's with anything being before the for loop. Or more specifically, anything appearing before an inlined block.

For example, these all work:

python -c "import sys; print 'rob'"
python -c "import sys; sys.stdout.write('rob\n')"

If import being a statement were an issue, this would work, but it doesn't:

python -c "__import__('sys'); for r in range(10): print 'rob'"

For your very basic example, you could rewrite it as this:

python -c "import sys; map(lambda x: sys.stdout.write('rob%d\n' % x), range(10))"

However, lambdas can only execute expressions, not statements or multiple statements, so you may still be unable to do the thing you want to do. However, between generator expressions, list comprehension, lambdas, sys.stdout.write, the "map" builtin, and some creative string interpolation, you can do some powerful one-liners.

The question is, how far do you want to go, and at what point is it not better to write a small .py file which your makefile executes instead?

Answer 4

<sup>- To make this answer work with Python 3.x as well, print is called as a function: in 3.x, only print('foo') works, whereas 2.x also accepts print 'foo'.
- For a cross-platform perspective that includes Windows, see kxr's helpful answer.</sup>

In bash, ksh, or zsh:

Use an ANSI C-quoted string ($'...'), which allows using \n to represent newlines that are expanded to actual newlines before the string is passed to python:

python -c $'import sys\nfor r in range(10): print("rob")'

Note the \n between the import and for statements to effect a line break.

To pass shell-variable values to such a command, it is safest to use arguments and access them via sys.argv inside the Python script:

name='rob' # value to pass to the Python script
python -c $'import sys\nfor r in range(10): print(sys.argv[1])' "$name"

^{See below for a discussion of the pros and cons of using an (escape sequence-preprocessed) double-quoted command string with embedded shell-variable references.}

To work safely with $'...' strings:

• Double \ instances in your original source code.
  • \<char> sequences - such as \n in this case, but also the usual suspects such as \t, \r, \b - are expanded by $'...' (see man printf for the supported escapes)
• Escape ' instances as \'.

---

If you must remain POSIX-compliant:

Use printf with a command substitution:

python -c "$(printf %b 'import sys\nfor r in range(10): print("rob")')"

To work safely with this type of string:

• Double \ instances in your original source code.
  • \<char> sequences - such as \n in this case, but also the usual suspects such as \t, \r, \b - are expanded by printf (see man printf for the supported escape sequences).

• Pass a single-quoted string to printf %b and escape embedded single quotes as '\'' (sic).

  • Using single quotes protects the string's contents from interpretation by the shell.

    • That said, for short Python scripts (as in this case) you can use a double-quoted string to incorporate shell variable values into your scripts - as long as you're aware of the associated pitfalls (see next point); e.g., the shell expands $HOME to the current user's home dir. in the following command:

      • python -c "$(printf %b "import sys\nfor r in range(10): print('rob is $HOME')")"

    • However, the generally preferred approach is to pass values from the shell via arguments, and access them via sys.argv in Python; the equivalent of the above command is:

      • python -c "$(printf %b 'import sys\nfor r in range(10): print("rob is " + sys.argv[1])')" "$HOME"

  • While using a double-quoted string is more convenient - it allows you to use embedded single quotes unescaped and embedded double quotes as \" - it also makes the string subject to interpretation by the shell, which may or may not be the intent; $ and ` characters in your source code that are not meant for the shell may cause a syntax error or alter the string unexpectedly.

    • Additionally, the shell's own \ processing in double-quoted strings can get in the way; for instance, to get Python to produce literal output ro\b, you must pass ro\\b to it; with a '...' shell string and doubled \ instances, we get:
      python -c "$(printf %b 'import sys\nprint("ro\\\\bs")')" # ok: 'ro\bs'
      By contrast, this does not work as intended with a "..." shell string:
      python -c "$(printf %b "import sys\nprint('ro\\\\bs')")" # !! INCORRECT: 'rs'
      The shell interprets both "\b" and "\\b" as literal \b, requiring a dizzying number of additional \ instances to achieve the desired effect:
      python -c "$(printf %b "import sys\nprint('ro\\\\\\\\bs')")"

---

To pass the code via stdin rather than -c:

^{Note: I'm focusing on single-line solutions here; xorho's answer shows how to use a multi-line here-document - be sure to quote the delimiter, however; e.g., <<'EOF', unless you explicitly want the shell to expand the string up front (which comes with the caveats noted above).}

---

In bash, ksh, or zsh:

Combine an ANSI C-quoted string ($'...') with a here-string (<<<...):

python - <<<$'import sys\nfor r in range(10): print("rob")'

- tells python explicitly to read from stdin (which it does by default). - is optional in this case, but if you also want to pass arguments to the scripts, you do need it to disambiguate the argument from a script filename:

python - 'rob' <<<$'import sys\nfor r in range(10): print(sys.argv[1])'

---

If you must remain POSIX-compliant:

Use printf as above, but with a pipeline so as to pass its output via stdin:

printf %b 'import sys\nfor r in range(10): print("rob")' | python

With an argument:

printf %b 'import sys\nfor r in range(10): print(sys.argv[1])' | python - 'rob'

Answer 5

Any idea how this can be fixed?

Your problem is created by the fact that Python statements, separated by ;, are only allowed to be "small statements", which are all one-liners. From the grammar file in the Python docs:

stmt: simple_stmt | compound_stmt
simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE
small_stmt: (expr_stmt | del_stmt | pass_stmt | flow_stmt |
             import_stmt | global_stmt | nonlocal_stmt | assert_stmt)

Compound statements can't be included on the same line with other statements via semicolons - so doing this with the -c flag becomes very inconvenient.

When demonstrating Python while in a bash shell environment, I find it very useful to include compound statements. The only simple way of doing this reliably is with heredocs (a posix shell thing).

Heredocs

Use a heredoc (created with <<) and Python's command line interface option, -:

$ python - <<-"EOF"
        import sys                    # 1 tab indent
        for r in range(10):           # 1 tab indent
            print('rob')              # 1 tab indent and 4 spaces
EOF

Adding the - after << (the <<-) allows you to use tabs to indent (Stackoverflow converts tabs to spaces, so I've indented 8 spaces to emphasize this). The leading tabs will be stripped.

You can do it without the tabs with just <<:

$ python - << "EOF"
import sys
for r in range(10):
    print('rob')
EOF

Putting quotes around EOF prevents parameter and arithmetic expansion. This makes the heredoc more robust.

Bash multiline strings

If you use double-quotes, you'll get shell-expansion:

$ python -c "
> import sys
> for p in '$PATH'.split(':'):
>     print(p)
> "
/usr/sbin
/usr/bin
/sbin
/bin
...

To avoid shell expansion use single-quotes:

$ python -c '
> import sys
> for p in "$PATH".split(":"):
>     print(p)
> '
$PATH

Note that we need to swap the quote characters on the literals in Python - we basically can't use quote character being interpreted by BASH. We can alternate them though, like we can in Python - but this already looks quite confusing, which is why I don't recommend this:

$ python -c '
import sys
for p in "'"$PATH"'".split(":"):
    print(p)
'
/usr/sbin
/usr/bin
/sbin
/bin
...

Critique of the accepted answer (and others)

This is not very readable:

echo -e "import sys\nfor r in range(10): print 'rob'" | python

Not very readable, and additionally difficult to debug in the case of an error:

python -c "exec(\"import sys\\nfor r in range(10): print 'rob'\")"

Perhaps a bit more readable, but still quite ugly:

(echo "import sys" ; echo "for r in range(10): print 'rob'") | python

You'll have a bad time if you have "'s in your python:

$ python -c "import sys
> for r in range(10): print 'rob'"

Don't abuse map or list comprehensions to get for-loops:

python -c "import sys; map(lambda x: sys.stdout.write('rob%d\n' % x), range(10))"

These are all sad and bad. Don't do them.

Answer 6

Just use Return and type it on the next line:

python -c "import sys
for r in range(10): print 'rob'"

rob
rob
...

Answer 7

python2.6 -c "import sys; [sys.stdout.write('rob\n') for r in range(10)]"

works fine.

Use "[ ]" to inline your for loop.

Answer 8

The problem is not with the import statement. The problem is that the control flow statements don't work inlined in a Python interpreter command. Replace that import statement with any other statement, and you'll see the same problem.

Think about it: Python can't possibly inline everything. It uses indentation to group control-flow.

Answer 9

This variant is most portable for putting multi-line scripts on the command-line on Windows and Unix-like systems, Python 2 and Python 3, without pipes:

python -c "exec(\"import sys \nfor r in range(10): print('rob') \")"

(None of the other examples seen here so far did so.)

Neat on Windows is:

python -c exec"""import sys \nfor r in range(10): print 'rob' """
python -c exec("""import sys \nfor r in range(10): print('rob') """)

Neat on Bash on Unix-like systems is:

python -c $'import sys \nfor r in range(10): print("rob")'

---

This function turns any multiline-script into a portable command-one-liner:

def py2cmdline(script):
    exs = 'exec(%r)' % re.sub('\r\n|\r', '\n', script.rstrip())
    print('python -c "%s"' % exs.replace('"', r'\"'))

Usage:

>>> py2cmdline(getcliptext())
python -c "exec('print \'AA\tA\'\ntry:\n for i in 1, 2, 3:\n  print i / 0\nexcept:\n print \"\"\"longer\nmessage\"\"\"')"

The input was:

print 'AA    A'
try:
 for i in 1, 2, 3:
  print i / 0
except:
 print """longer
message"""

Answer 10

If your system is POSIX.2-compliant it should supply the printf utility:

printf "print 'zap'\nfor r in range(3): print 'rob'" | python

zap
rob
rob
rob

Answer 11

I'm not really a big Pythoner - but I found this syntax once, forgot where from, so I thought I'd document it:

If you use sys.stdout.write instead of print (the difference being, sys.stdout.write takes arguments as a function, in parentheses - whereas print doesn't), then for a one-liner, you can get away with inverting the order of the command and the for, removing the semicolon, and enclosing the command in square brackets, i.e.:

python -c "import sys; [sys.stdout.write('rob\n') for r in range(10)]"

I have no idea how this syntax would be called in Python :)

---

These square brackets in one-liners are "list comprehensions"; note this in Python 2.7:

STR=abc
echo $STR | python -c "import sys,re; a=(sys.stdout.write(line) for line in sys.stdin); print a"

Output:

<generator object <genexpr> at 0xb771461c>

So the command in round brackets/parenthesis is seen as a "generator object"; if we "iterate" through it by calling next() - then the command inside the parenthesis will be executed (note the "abc" in the output):

echo $STR | python -c "import sys,re; a=(sys.stdout.write(line) for line in sys.stdin); a.next() ; print a"

Output:

abc
<generator object <genexpr> at 0xb777b734>

If we now use square brackets - note that we don't need to call next() to have the command execute, it executes immediately upon assignment; however, later inspection reveals that a is None:

echo $STR | python -c "import sys,re; a=[sys.stdout.write(line) for line in sys.stdin]; print a"

Output:

abc
[None]

This doesn't leave much information to look for - for the square brackets case - but I stumbled upon this page which I think explains:

Python Tips And Tricks – First Edition - Python Tutorials | Dream.In.Code:

If you recall, the standard format of a single line generator is a kind of one line 'for' loop inside brackets. This will produce a 'one-shot' iterable object which is an object you can iterate over in only one direction and which you can't re-use once you reach the end.

A 'list comprehension' looks almost the same as a regular one-line generator, except that the regular brackets - ( ) - are replaced by square brackets - [ ]. The major advantage of alist comprehension is that produces a 'list', rather than a 'one-shot' iterable object, so that you can go back and forth through it, add elements, sort, etc.

And indeed it is a list - it's just its first element becomes none as soon as it is executed:

echo $STR | python -c "import sys,re; print [sys.stdout.write(line) for line in sys.stdin].__class__"

Output:

abc
<type 'list'>

---

echo $STR | python -c "import sys,re; print [sys.stdout.write(line) for line in sys.stdin][0]"

Output:

abc
None

List comprehensions are otherwise documented in 5. Data Structures: 5.1.4. List Comprehensions — Python v2.7.4 documentation as "List comprehensions provide a concise way to create lists"; presumably, that's where the limited "executability" of lists comes into play in one-liners.

Well, hope I'm not terribly too off the mark here ...

And here is a one-liner command line with two non-nested for-loops; both enclosed within "list comprehension" square brackets:

echo $STR | python -c "import sys,re; a=[sys.stdout.write(line) for line in sys.stdin]; b=[sys.stdout.write(str(x)) for x in range(2)] ; print a ; print b"

Output:

abc
01[None]
[None, None]

Notice that the second "list" b now has two elements, since its for loop explicitly ran twice; however, the result of sys.stdout.write() in both cases was (apparently) None.

Answer 12

single/double quotes and backslash everywhere:

python -c 'exec("import sys\nfor i in range(10): print \"bob\"")'

Much better:

python -c '
import sys
for i in range(10):
  print("bob")
'

Note: On some systems executable python is for Python 2 and may not be available. And executable python3 is for Python 3.

Answer 13

I wanted a solution with the following properties:

1. Readable
2. Read standard input for processing output of other tools

Both requirements were not provided in the other answers, so here's how to read standard input while doing everything on the command line:

grep special_string -r | sort | python3 <(cat <<EOF
import sys
for line in sys.stdin:
    tokens = line.split()
    if len(tokens) == 4:
        print("%-45s %7.3f    %s    %s" % (tokens[0], float(tokens[1]), tokens[2], tokens[3]))
EOF
)

Answer 14

Use python -c with triple quotes:

python -c """
import os
os.system('pwd')
os.system('ls -l')
print('Hello, World!')
for _ in range(5):
    print(_)
"""

Answer 15

This script provides a Perl-like command line interface:

Pyliner - Script to run arbitrary Python code on the command line (Python recipe)

Answer 16

If you don't want to touch standard input and simulate as if you had passed "python cmdfile.py", you can do the following from a Bash shell:

python <(printf "word=raw_input('Enter word: ')\nimport sys\nfor i in range(5):\n    print(word)")

As you can see, it allows you to use standard input for reading input data. Internally the shell creates the temporary file for the input command contents.

Answer 17

When I needed to do this, I used:

python -c "$(echo -e "import sys\nsys.stdout.write('Hello, World!\\\n')")"

Note the triple backslash for the newline in the sys.stdout.write statement.

Answer 18

There is one more option. sys.stdout.write returns None, which keeps the list empty:

cat somefile.log|python -c "import sys;[line for line in sys.stdin if sys.stdout.write(line*2)]"