0

I want to take the output of a Python script and convert it into list of dictionaries to perform some action. I have written a below code:

import subprocess
import json
chief_complaint = """ "Eernise, Randell is an 64-year-old male" """
command= """python predict_icd.py --cc """+str(chief_complaint)
command_proc=subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)
print("response - ",command_proc.stdout.read())

The output that I am getting of above code is:

command_proc.stdout.read() -  b"[{'text': 'CHA', 'start': 51, 'end': 54, 'code': ''}, {'text': 'CHA', 'start': 221, 'end': 224, 'code': ''}, {'text': 'dog', 'start': 286, 'end': 289, 'code': ''}, {'text': 'Schizophrenia', 'start': 351, 'end': 364, 'code': 'F20'}, {'text': 'chronic pain', 'start': 366, 'end': 378, 'code': 'G89.29'}, {'text': 'chest pain', 'start': 599, 'end': 609, 'code': 'R07.89'}, {'text': 'palpitations', 'start': 613, 'end': 625, 'code': 'R00.2'}, {'text': 'cough', 'start': 829, 'end': 834, 'code': 'R05'}, {'text': 'wheezing', 'start': 836, 'end': 844, 'code': 'R06.2'}, {'text': 'pain', 'start': 966, 'end': 970, 'code': ''}, {'text': 'schizophrenia', 'start': 1078, 'end': 1091, 'code': 'F20'}]\n"
bad_coder
  • 11,289
  • 20
  • 44
  • 72
Kritik Alawat
  • 9
  • 9

1 Answers1

0

The absolutely best solution is to not run Python as a subprocess of itself. Instead, refactor predict_icd.py so you can import it and call its functions directly. See What is the best way to call a script from another script?

Your second best option is to change predict_icd.py so that it prints its results in a standard format like JSON. Then you can call it from any tool, not just Python.

The distant third option is to go with what you have and use ast.literal_eval() to parse out the results. I have also refactored to avoid the pesky and unnecessary shell=True and fix the orphaned Popen.

import subprocess
# import json  # unused import
from ast import literal_eval

chief_complaint = """ "Eernise, Randell is an 64-year-old male" """
command_proc = subprocess.run(
    ['python', 'predict_icd.py', '--cc', chief_complaint],
    capture_output=True, text=True, check=True)
response = literal_eval(command_proc.stdout)
print("response = ", response)

In some more detail, Popen will only start a process; you have to take care of it during its lifetime if you do that - but subprocess.run() already takes care of all this plumbing for you. Like its documentation says, avoid Popen when the higher-level functions from the standard library already do what you want.

Notice also the text=True which tells Python to send and receive text, not bytes to and from the subprocess.

tripleee
  • 175,061
  • 34
  • 275
  • 318
  • Thanks for answering. I am using python3.6 and using your code, I got 2 errors 1st - TypeError: __init__() got an unexpected keyword argument 'capture_output'....2nd - TypeError: __init__() got an unexpected keyword argument 'text' – Kritik Alawat Oct 25 '21 at 16:34
  • The keyword arguments `capture_output` and `text` are a bit newer than that. If you are really stuck on 3.6, you'll have to revert to using explicit `stdout=subprocess.PIPE` etc and switch to the legacy synonym `universal_newlines=True` instead of `text=True` ... still `run` should be a significant improvement over an incomplete ad-hoc reimplementation over `Popen`. You'll notice that 3.6 is slated to go EOL at the end of 2021. – tripleee Oct 25 '21 at 16:36
  • And of course, my primary recommendation remains to not run Python as a subprocess of itself. – tripleee Oct 25 '21 at 16:39