5

I have been teaching myself python over the past few months and am finally starting to do some useful things.

What I am trying to ultimately do is have a python script that acts as a queue. That is, I would like to have a folder with a bunch of input files that another program uses to run calculations (I am a theoretical physicist and do many computational jobs a day).

The way I must do this now is put all of the input files on the box that has the computational software. Then I have to convert the dos input files to unix (dos2unix), following this I must copy the new input file to a file called 'INPUT'. Finally I run a command that starts the job.

All of these tasks are handled in a command prompt. My question is how to I interface my program with the command prompt? Then, how can I monitor the process (which I normally do via cpu usage and the TOP command), and have python start the next job as soon as the last job finishes.

Sorry for rambling, I just do not know how to control a command prompt from a script, and then have it automatically 'watch' the job.

Thanks

Michael R
  • 259
  • 5
  • 16
  • 4
    I think the [subprocess](http://docs.python.org/2/library/subprocess.html) module is what you want. It's a good question, I'll give you that, but there are many [duplicates](http://stackoverflow.com/q/89228/645270) of it here on SO. – keyser Jul 10 '13 at 19:22
  • What you're actually asking for—controlling a command prompt from a script—is hard. But when the command prompt you're trying to control (in this case, the shell) isn't doing anything you can't do more easily in Python (in this case, with `subprocess`), you don't need to control the prompt. – abarnert Jul 10 '13 at 19:56

2 Answers2

6

The subprocess module has many tools for executing system commands in python.

from subprocess import call
call(["ls", "-l"])

source

call will wait for the command to finish and return its returncode, so you can call another one afterwards knowing that the previous one has finished.

os.system is an older way to do it, but has fewer tools and isn't recommended:

import os
os.system('"C:/Temp/a b c/Notepad.exe"')

edit FvD left a comment explaning how to "watch" the process below

Community
  • 1
  • 1
Stephan
  • 16,509
  • 7
  • 35
  • 61
  • And within subprocess you can use Popen.communicate(input=None) to "watch" the job, as the OP put it. – FvD Jul 10 '13 at 19:25
  • How is `os.system` any easier than `subprocess.call`? It's the same number of concepts (and about the same number of keystrokes, too) in trivial cases, and more complicated as soon as anything else comes up (like arguments with spaces in them). – abarnert Jul 10 '13 at 19:40
  • Major deja vu from the duplicate I linked to :p (with the `os.system` discussion) – keyser Jul 10 '13 at 19:42
  • ty @Keyser :) forgot to cite like i usually do – Stephan Jul 10 '13 at 19:42
1

If you actually need to drive an interactive command-line interface, there is no way to do that with the stdlib.

There are a number of third-party options for this; I think pexpect is probably the most popular.

However, if you don't really need to drive it interactively—if the program only needs you to give it arguments on the command line, or a "batch mode" dump to its standard input, then subprocess makes it easy. For example, to drive the sort program, you can just do this:

with Popen(['sort', '-n'], stdin=PIPE, stdout=PIPE) as p:
    sorted_data = p.communicate(input_data)

This is of course a silly example, because you can do anything sort can do with Python's sorted with a key argument, and probably a lot more readably.

More generally: often when you think you need to interactively script some program, you really don't, and sometimes you don't even need to run it at all.

And your particular case is exactly such a case. You're asking about interactively scripting the shell. But you don't need actually need to do so, because Python can do all the things you need from the shell.

abarnert
  • 354,177
  • 51
  • 601
  • 671