0

So I have a shell script that requires something like

process_name --method POST --inputFile /path/to/file

Instead of the path to file, I can follow this answer and do something like:

process_name --method POST --inputFile <(echo "{\"param\":\"value\"}")

But when I try to do the same from python using:

shell_string = 'process_name --method POST --inputFile'
echo_val = '<(echo "{\"param\":\"value\"}")'
subprocess.Popen(shell_string.split() + [echo_val], stdout=subprocess.PIPE, stderr=subprocess.PIPE)

I get the following error: FileNotFoundError: [Errno 2] No such file or directory: \'<(echo "{\\\\"param\\\\":\\\\"value\\\\"}")\'\n'

I'm assuming this has to do with appropriately escaping the quotes and backslashes but I can't quite figure out what it needs from me. Help is appreciated.

Thanks.

doddy
  • 579
  • 5
  • 18
  • It's not an escaping problem. Instead, `<(...)` is a direction *to the shell* to replace that syntax construct with a filename (typically of the form `/dev/fd/##`) from which the output of the command in `...` can be read. If you don't have a shell (and specifically, a shell that has that extension; `/bin/sh` won't cut it), then the construct has no meaning. – Charles Duffy Oct 06 '20 at 00:02
  • BTW, of the two duplicates this is closed with, the answers on [How to execute '<(cat fileA fileB)' using python?](https://stackoverflow.com/questions/26593229/how-to-execute-cat-filea-fileb-using-python) are IMHO considerably better. – Charles Duffy Oct 06 '20 at 00:05
  • Also, `shell_string.split()` in the context of `subprocess.Popen` is an antipattern. Yes, a lot of people do it, but those people are creating bugs for themselves unnecessarily. It's much better practice to create an explicit list. – Charles Duffy Oct 06 '20 at 00:06
  • I don't feel like the linked questions are duplicates. I am also unable to use them to solve my issue. I have tried to write to stdin and then reading from there but I am unable to read stdout. For someone who is a novice at using the shell, this should not have been marked a duplicate. – doddy Oct 06 '20 at 02:17
  • It's a Python question, not a shell question. Your `<(...)` shell syntax needs to be replaced with a `/dev/fd/##` filename. The linked questions have answers that tell you how to create such a filename. – Charles Duffy Oct 06 '20 at 02:43
  • ...so: https://stackoverflow.com/a/26594338/14122 tells you **exactly** what you need to do. Replace the `["cat", "foo.txt", "bar.txt"]` in their answer with `["echo", '{"description":"prioritized for WBR"}']` and the `['wc', '/dev/fd/' + str(input_fd)]` with `['process_name', '--method', 'POST', '--inputfile']` and there you are. – Charles Duffy Oct 06 '20 at 02:43
  • ...the only thing about that answer that isn't still perfect in modern times is that in current versions of Python you need to pass a list of file descriptors to be exposed to the process when it's more than the default stdin/stdout/stderr trio, but for that we have other Q&A entries (f/e, I mention it in https://stackoverflow.com/questions/57098553/why-does-script-py-cat-gz-work-with-subprocess-popen-in-python-2-but-not; search for `pass_fds`) – Charles Duffy Oct 06 '20 at 02:48
  • ...see https://stackoverflow.com/questions/64158135/how-to-use-subprocess-in-python-to-run-a-bash-command-that-is-very-complicated for a modern, _complete_ answer, `pass_fds` included. – Charles Duffy Oct 06 '20 at 02:48
  • ...note, btw -- not _one_ of the answers I'm encouraging _involves any shell at all_; they're all doing the work internal to Python, without needing any shell to implement `<(...anything...)` magic. Thus, being a novice at shell is moot; if you're implementing the process substitution logic in Python, there _is_ no shell. – Charles Duffy Oct 06 '20 at 02:50
  • ...err, 4 comments above, that should have been `['process_name', '--method', 'POST', '--inputfile', f'/dev/fd/{input_fd}']` (or the original Python-2-compatible `'/dev/fd/' + str(fd)` as given in the linked question's preexisting answer, of course). – Charles Duffy Oct 06 '20 at 02:53

0 Answers0