5

I was playing around with Python's subprocess module, trying a few examples but I can't seem to get heredoc statements to work.

Here is the trivial example I was playing with:

import subprocess
a = "A String of Text"
p = subprocess.Popen(["cat", "<<DATA\n" + a + "\nDATA"])

I get the following error when I run the code above:

cat: <<DATA\nA String of Text\nDATA: No such file or directory

Am I doing it wrong? Is this even possible? If so how would I go about doing it?

Update

Just wanted to say that this should never be performed in a real python program because there are better ways of doing this.

MitMaro
  • 5,607
  • 6
  • 28
  • 52

4 Answers4

6

The shell "heredoc" support is a shell feature. subprocess.Popen does not run your command through the shell by default, so this syntax certainly won't work.

However, since you're using pipes anyway, there isn't any need to use the heredoc support of the shell. Just write your string a to the stdin pipe of the process you just started. This is exactly what the shell would do with the heredoc anyway.

You can do this with Popen.communicate():

p.communicate(a)

The return value of the communicate() function contains the output of the process (in two streams, see the docs).

Greg Hewgill
  • 951,095
  • 183
  • 1,149
  • 1,285
  • I didn't quite get what the Shell parameter did. The docs explain very well what happens when you set it to true but not so much when it is set to false. – MitMaro Sep 03 '09 at 22:22
  • 1
    When the `shell` paramter is `False`, the `subprocess` module directly executes the program you specify (in your case, probably `/bin/cat`) with exactly the arguments specified in the call to `Popen()`. No interpretation of shell characters such as redirection or pipes or anything is done, the program sees exactly what you send it. This is why `cat` said it couldn't find the file whose name started with `< – Greg Hewgill Sep 03 '09 at 22:42
  • Thanks for the explanation, clarified several things. – MitMaro Sep 03 '09 at 23:03
4

You're passing shell syntax as an arguments to cat program. You can try to do it like that:

p = subprocess.Popen(["sh", "-c", "cat <<DATA\n" + a + "\nDATA"])

But the concept itself is wrong. You should use Python features instead of calling shell scripts inside your python scripts.

And in this particular case you should that shell's heredoc syntax interpolates variables, so you'll need to escape all the text inside a and make sure there's no DATA line in it.

For Python equivalent, I think the closest idea to this (assuming you don't want just print(a) ;-)) is passing the value of the variable to stdin of a spawned process:

p = subprocess.Popen(["program", ...], stdin=subprocess.PIPE)
p.communicate(a)
Michał Górny
  • 18,713
  • 5
  • 53
  • 76
  • This is a helpful answer. It'd be even better if you could give code suggestions for how to do do this the Pythonic way. – Brian M. Hunt Sep 03 '09 at 22:10
  • As I stated in my question this was just playing around with subprocess and I would never do something like calling `cat` in Python. It was out of curiosity more so then for use in real code. – MitMaro Sep 03 '09 at 22:20
  • "You should use Python features instead of calling shell scripts inside your python scripts." Not all shell features are available in Python. A heredoc can be fed to many different kinds of programs that do not have a Python interface, and the heredoc contents can be dynamically generated with Python. – user5359531 Mar 17 '17 at 12:18
  • 1
    @user5359531, I've added a trivial example for that. – Michał Górny Mar 18 '17 at 12:25
4

As others have pointed out, you need to run it in a shell. Popen makes this easy with a shell=True argument. I get the following output:

>>> import subprocess
>>> a = "A String of Text"
>>> p = subprocess.Popen("cat <<DATA\n" + a + "\nDATA", shell=True)
>>> A String of Text

>>> p.wait()
0
bstpierre
  • 30,042
  • 15
  • 70
  • 103
1

As of Python 3.5 you can use subrocess.run as in:

subprocess.run(['cat'], input=b"A String of Text")
thejeremyjohn
  • 151
  • 5