Issue using subprocess to run a PDAL bash command from Python

Question

Issue:

I cannot run a pdal bash command from Python using subprocess.

Here is the code

based on Running Bash commands in Python:

import os, subprocess

input = '/path/to/file.ply'
output = '/path/to/statfile.json'
if not os.path.isfile(output):
    open(output, 'a').close()

bashcmd = ("pdal info --boundary "
           +input
           +" > "
           +output
           )

print("Bash command is:\n{}\n".format(bashcmd))

process = subprocess.Popen(bashcommand.split(),
                           stdout=subprocess.PIPE,
                           shell=True)
    output, error = process.communicate()
    print("Output:\n{}\n".format(output))
    print("Error:\n{}\n".format(error))

Which gives me this output in the Python console:

Bash command is:
pdal info --boundary /path/to/file.ply > /path/to/statfile.json

Output:
Usage:
  pdal <options>
  pdal <command> <command options>
  --command        The PDAL command
  --debug          Sets the output level to 3 (option deprecated)
  --verbose, -v    Sets the output level (0-8)
  --drivers        List available drivers
  --help, -h       Display help text
  --list-commands  List available commands
  --version        Show program version
  --options        Show options for specified driver (or 'all')
  --log            Log filename (accepts stderr, stdout, stdlog, devnull as
      special cases)
  --logtiming      Turn on timing for log messages

The following commands are available:
  - delta
  - diff
  - fauxplugin
  - ground
  - hausdorff
  - info
  - merge
  - pcl
  - pipeline
  - random
  - smooth
  - sort
  - split
  - tindex
  - translate

See http://pdal.io/apps/ for more detail

Error:
None

It looks as if it had stop reading the arguments of the command after the call to 'pdal' only, which prints this help message.

If I copy the output of the first print and paste it in a bash terminal, it works properly, giving me the output file with the desired metadata. But from Python no output file is created.

Question:

I wonder why (e.g. is there anything wrong with the redirection or the fact that the computation itself may take ~20sec normally?), and how to execute this command from Python?

This doesn't provide a clear enough answer to the present issue.

tripleee · Accepted Answer · 2018-08-21T12:57:00.330

There are multiple errors here.

You are using an undefined variable bashCommand instead of the one you defined above bashcmd.
You are mixing output to a Python file handle with shell redirection.
You are not capturing the stderr of the process. (I will vaguely assume you do not need the standard error anyway.)
You should not split() the command if you run it with shell=True.

More broadly, you should probably avoid the shell=True and let Python take care of the redirection for you by connecting the output to the file you open; and in modern times, you really should not use subprocess.Popen() if you can use subprocess.run() or subprocess.check_call() or friends.

import subprocess

input = '/path/to/file.ply'
output = '/path/to/statfile.json'

with open(output, 'a') as handle:
    bashcmd = ['pdal', 'info', '--boundary', input]

    #print("Bash command is:\n{}\n".format(bashcmd))

    result = subprocess.run(bashcmd, stdout=handle, stderr=subprocess.PIPE)

    # No can do because output goes straight to the file now
    ##print("Output:\n{}\n".format(output))
    #print("Error:\n{}\n".format(result.stdout))

Issue using subprocess to run a PDAL bash command from Python

Issue:

Here is the code

Question:

1 Answers1