0

I am trying to get the header info of a hive table to dynamically make a partitioned Hive table later.

Running the below code freezes the system: Python 2...

What is going wrong, I am new to python.

def get_final_table_columns_file():



    logger.info("========================================================================================")
    logger.info("Executing Hive DDL to get Final Hive Table Columns")

    column_grab_cmd = []
    column_grab_cmd.append(os.environ['HIVE_HOME'] + '/bin/hive')
    column_grab_cmd.append('-S')
    column_grab_cmd.append('-e')
    column_grab_cmd.append('\"SET hive.cli.print.header=true;')
    column_grab_cmd.append('USE '+hive_db_name+ '; SELECT * FROM '+ staging_table_name +' LIMIT 0;\"')

    printArray(column_grab_cmd)

    column_grab_cmd_process = subprocess.call(column_grab_cmd,stdout=subprocess.PIPE, shell=True)
    (output, err) = column_grab_cmd_process.communicate()


    success_message = "Execution of Hive DDL to grab headers succeed: \\n" + output
    logger.info(success_message)
Dalton L. Heiland
  • 7
  • 1
  • 5
  • Have you tried running the command in `column_grab_cmd` manually? – FamousJameous Jan 30 '17 at 23:29
  • 2
    You may want to consider using subprocess.Popen() instead of subprocess.call() if you are sending a lot of data to stdout: http://stackoverflow.com/questions/9785179/calling-subprocess-call-hangs-when-i-set-the-stdout – Apollo2020 Jan 30 '17 at 23:34
  • I have run the command itself and it runs as expected. And with Popen, can I poll the process to get the data? – Dalton L. Heiland Jan 31 '17 at 01:32
  • Just pasting the command of `printArray(column_grab_cmd)` and pressing Return is not the same as running the command with `shell=True`. [See more](http://stackoverflow.com/a/36008455/7484570). – Marcos Modenesi Jan 31 '17 at 01:58

0 Answers0