How to check if a string is a valid shell command using Python?

Question

I am making a program that adds additional functionality to the standard command shell in Windows. For instance, typing google followed by keywords will open a new tab with Google search for those keywords, etc. Whenever the input doesn't refer to a custom function I've created, it gets processed as a shell command using subprocess.call(rawCommand, shell=True).

Since I'd like to anticipate when my input isn't a valid command and return something like f"Invalid command: {rawCommand}", how should I go about doing that?

So far I've tried subprocess.call(rawCommand) which also return the standard output as well as the exit code. So that looks like this:

>>> from subprocess import call
>>> a, b = call("echo hello!", shell=1), call("xyz arg1 arg2", shell=1)
hello!
'xyz' is not recognized as an internal or external command,
operable program or batch file.
>>> a
0
>>> b
1

I'd like to simply recieve that exit code. Any ideas on how I can do this?

You can use `stdout=open(os.devnull, 'w')` to ignore the standard output, but running a command simply to determine if it is "valid" is not a good idea, as you may not be ready for the side effects. — chepner, Jul 04 '20 at 13:45
@chepner Which side effects are the most common? What should I be aware of when attempting something like this? — peki, Jul 04 '20 at 13:46
@peki, stdout and stderr can be captured. If the command is successful, the user may want to see stdout. If the command fails, then stdout and/or stderr will probably contain information the user needs in order to make changes to be successful. — lit, Jul 04 '20 at 15:48

score 1 · Accepted Answer · answered Jul 07 '20 at 10:10

Should you one day want deal with encoding errors, get back the result of the command you're running, have a timeout or decide which exit codes other than 0 may not trigger errors (i'm looking at you, java runtime !), here's a complete function that does that job:

import os
from logging import getLogger
import subprocess

logger = getLogger()


def command_runner(command, valid_exit_codes=None, timeout=300, shell=False, encoding='utf-8',
                   windows_no_window=False, **kwargs):
    """
    Whenever we can, we need to avoid shell=True in order to preseve better security
    Runs system command, returns exit code and stdout/stderr output, and logs output on error
    valid_exit_codes is a list of codes that don't trigger an error
    windows_no_window will hide the command window (works with Microsoft Windows only)
    
    Accepts subprocess.check_output arguments
        
    """

    # Set default values for kwargs
    errors = kwargs.pop('errors', 'backslashreplace')  # Don't let encoding issues make you mad
    universal_newlines = kwargs.pop('universal_newlines', False)
    creationflags = kwargs.pop('creationflags', 0)
    if windows_no_window:
        creationflags = creationflags | subprocess.CREATE_NO_WINDOW

    try:
        # universal_newlines=True makes netstat command fail under windows
        # timeout does not work under Python 2.7 with subprocess32 < 3.5
        # decoder may be unicode_escape for dos commands or utf-8 for powershell
        output = subprocess.check_output(command, stderr=subprocess.STDOUT, shell=shell,
                                         timeout=timeout, universal_newlines=universal_newlines, encoding=encoding,
                                         errors=errors, creationflags=creationflags, **kwargs)

    except subprocess.CalledProcessError as exc:
        exit_code = exc.returncode
        try:
            output = exc.output
        except Exception:
            output = "command_runner: Could not obtain output from command."
        if exit_code in valid_exit_codes if valid_exit_codes is not None else [0]:
            logger.debug('Command [%s] returned with exit code [%s]. Command output was:' % (command, exit_code))
            if isinstance(output, str):
                logger.debug(output)
            return exc.returncode, output
        else:
            logger.error('Command [%s] failed with exit code [%s]. Command output was:' %
                         (command, exc.returncode))
            logger.error(output)
            return exc.returncode, output
    # OSError if not a valid executable
    except (OSError, IOError) as exc:
        logger.error('Command [%s] failed because of OS [%s].' % (command, exc))
        return None, exc
    except subprocess.TimeoutExpired:
        logger.error('Timeout [%s seconds] expired for command [%s] execution.' % (timeout, command))
        return None, 'Timeout of %s seconds expired.' % timeout
    except Exception as exc:
        logger.error('Command [%s] failed for unknown reasons [%s].' % (command, exc))
        logger.debug('Error:', exc_info=True)
        return None, exc
    else:
        logger.debug('Command [%s] returned with exit code [0]. Command output was:' % command)
        if output:
            logger.debug(output)
        return 0, output

Usage:

exit_code, output = command_runner('whoami', shell=True)

Thank you so much!! Using `getLogger` is really clever, but I'm really inexperienced with it, so this means a bunch to me, heh... — peki, Jul 07 '20 at 15:31
Using `getLogger` in this script supposes you have configured logging somewhere beforehand (see basicConfig to begin with). In any case this subprocess implementation is way more reliable because it can handle alot of corner cases.If you're working with windows, give the parameter `encoding='cp437'` a try if you have special characters (accents etc). Good luck. — Orsiris de Jong, Jul 07 '20 at 17:49

score 0 · Answer 2 · answered Jul 04 '20 at 16:18

Some shells have a syntax-checking mode (e.g., bash -n), but that’s the only form of error that’s separable from “try to execute the commands and see what happens”. Defining a larger class of “immediate” errors is a fraught proposition: if echo hello; ./foo is invalid because foo can’t be found as a command, what about false && ./foo, which will never try to run it, or cp /bin/ls foo; ./foo, which may succeed (or might fail to copy)? What about eval $(configure_shell); foo which might or might not manipulate PATH so as to find foo? What about foo || install_foo, where the failure might be anticipated?

As such, anticipating failure is not possible in any meaningful sense: your only real option is to capture the command’s output/error (as mentioned in the comments) and report them in some useful way.

Your basic analysis is spot on. But, the user, by using the `cmd` tag has specified that a Windows system is being used. Validation problems for commands like `echo hello & .\foo` are, as you say, a fraught proposition. — lit, Jul 04 '20 at 23:07
@lit: I’m not good enough with `cmd` to concoct meaningful examples for it (or identify a syntax-check option for it, if any), but as you said that doesn’t affect the principle at all. — Davis Herring, Jul 04 '20 at 23:26

How to check if a string is a valid shell command using Python?

2 Answers2