3

I am executing few shell scripts from a python script. A particular shell script(last line of the code) call is used to run a Hive query to get some table information and I would like to redirect this output to a file. Here is how my python script looks like.

$ cat test.py
    import argparse
    from subprocess import call

    parser = argparse.ArgumentParser()
    parser.add_argument('-c','--cutoff',required=True)
    args = parser.parse_args()

    Table="elements_status_values_"+ args.cutoff
    call(["clear"])
    print ">> Running the hive query"
    call(["hive", "-S", "-e" "select * from %s order by rand() limit 2;" % cutoffTable])

When I execute this, I get the result on the terminal but I need to redirect the output to a file from python script to do few more operations using the output. So I basically want my Hive query output to be redirected to a file so that I can use the file for other operations within the same script. In shell script we can redirect the output using '>' to a file but is there a way where we can get this done using python? I have searched posts related to this but all of them are redirecting the python scripts output to a file which I don't want to.

Alex Raj Kaliamoorthy
  • 2,035
  • 3
  • 29
  • 46

1 Answers1

4

Check out documentation for subprocess.call: you can supply params like stdout and stdin.

with open('output.txt', 'w') as output_file:
    call([your_stuff], stdout=output_file)

This is a generic way - similar approach would work in C and in many other languages too.

Old python-specific way would be to use subprocess.check_output instead of subprocess.call.

Current python-specific way would be to use subprocess.run and literally check its stdout: output = run(..., check=True, stdout=PIPE).stdout

Daerdemandt
  • 2,281
  • 18
  • 19