3

So I have created a logger which logs to both file and console. However, I need to call subprocess to run other python files but the problem is that console and file is only populated with logging messages once the subprocess has complete even though I have implemented logging within the python files. I would like it to log to console during a subprocess. That is, I am able to see output during the running on the other files.

Any ideas? or is there another way to run a python file within python?

My subprocess function:

def _run_cmd(args_list):
    """
    run linux commands
    """
    # import subprocess
    print('Running system command: {0}'.format(' '.join(args_list)))
    proc = subprocess.Popen(args_list, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    s_output, s_err = proc.communicate()
    s_return = proc.returncode
    return s_return, s_output, s_err

Where I call the subprocess:

try:
    s_return, s_output, s_err = _run_cmd(["python", abs_path])
    if s_err:
        finish_fail(config, logger)
except Exception as e:
    logger.error("error", str(e))

abs_path is the location to the file which I would like to run.

Bryce Ramgovind
  • 3,127
  • 10
  • 41
  • 72

1 Answers1

4
  1. There has been a question similar to this line here. You should use log_subprocess_output to log the outputs of your subprocess. As it is a subprocess it won't automatically print the logs to your console or file while you are running it. That's why it first completes the process and stores the logs in file at the end of the process.
  2. The second way to do that is to import the python files that you wanna call them into the main python file and keep the logging process as it is.
  3. Another way to handle that will be by using spur to run the python files from the main python file. (Less Recommended way) Spur

Thanks.

capt2101akash
  • 184
  • 4