1

I'm new to subprocess and I'm trying to use subprocess.call instead of os.system. The command would look like this in os.system:

os.system('cluster submit {} gs://{}/{}'.format(cluster_name, bucket, file))

How would that translate into subprocess? I tried the following but neither worked:

  1. subprocess.call(["cluster", "submit", "{}", "gs://{}/{}".format(cluster_name, bucket, file)]).strip()

  2. subprocess.call(["cluster", "submit", "{}", "gs://{}/{}".format(cluster_name, bucket_name, script)], shell=True).strip()

claudiadast
  • 419
  • 1
  • 9
  • 18
  • What is "cluster"? If it is an alias, you will need to fork it into a shell so that BASH can resolve it. See this answer: https://stackoverflow.com/questions/89228/calling-an-external-command-in-python/89243#89243 – Stuart Buckingham Sep 29 '17 at 16:14
  • 1
    @StuartBuckingham, noninteractive shells don't support aliases. They can be turned on in bash, but even then, an uninteractive shell doesn't run `.bashrc` or `.bash_profile`, so it won't see them. – Charles Duffy Sep 29 '17 at 16:21
  • @PM2Ring, the default `sh` has nothing to do with it. If you don't have `shell=True`, then it's exclusively the shebang used to decide which shell to execute, and executing a script with no shebang will fail; it's only if you have a shell that there's a fallback behavior of spawning another shell interpreter if the kernel's `exec` call fails. – Charles Duffy Sep 29 '17 at 16:24
  • @CharlesDuffy Thanks for the correction – Stuart Buckingham Sep 29 '17 at 16:26

2 Answers2

2

First, take a look at the docs for subprocess.call:

subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False)

Run the command described by args. Wait for command to complete, then return the returncode attribute.

The command returns the process return code, which is an integer, so your attempt to call subprocess.call(...).strip() is going to fail. If you want the output of the command, you need:

subprocess.check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)

Run command with arguments and return its output as a byte string.

This would give you:

output = subprocess.call(["cluster", "submit", "{}", 
    "gs://{}/{}".format(cluster_name, bucket, file)]).strip()

But there are some basic Python problems there, because you've got a bare "{}" on which you are not calling .format, and then you've got "gs://{}/{}" which only has two format markers but three parameters. You want something more like:

output = subprocess.call(["cluster", "submit", cluster_name, 
    "gs://{}/{}".format(bucket, file)]).strip()

When you use shell=True, as in your second example, you would pass a string rather than a list. E.g:

output = subprocess.check_output("cluster submit {} gs://{}/{}".format(
    cluster_name, bucket_name, script), shell=True).strip()
larsks
  • 277,717
  • 41
  • 399
  • 399
1
subprocess.call([
  "cluster",
  "submit",
  str(cluster_name),
  "gs://{}/{}".format(bucket_name, script)
])
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441