Popen in Flask starts Python interpreter

Question

I'm trying to stream stdout of a script, myscript.py, through a Flask endpoint.

I'm using this SO answer as my starting point. That script worked fine for me. The code here is virtually unchanged from that answer, except for calling a Python script instead of "dmesg" directly.

@app.route('/yield')
def index():
     def inner():
         proc = subprocess.Popen(
            ['python', 'myscript.py'],             
            shell=True,
            stdout=subprocess.PIPE
        )

        for line in iter(proc.stdout.readline,''):
            time.sleep(1)                          
            yield line.rstrip() + '<br/>\n'

    return Flask.Response(inner(), mimetype='text/html')

myscript.py

import subprocess
def main():
    subprocess.Popen(['dmesg'], shell=True, stdout=subprocess.PIPE)

if __name__ == '__main__':
    main()

When I navigate to "/yield" in my browser I get a "This page can't be reached" error, and the flask test server drops into a Python interpreter (see below) and nothing else happens.

 * Running on http://0.0.0.0:7000/ (Press CTRL+C to quit)
 * Restarting with stat
 * Debugger is active!
 * Debugger pin code: 262-399-135
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more  information.
>>>

What can I fix in the above to call a Python script for streaming through Flask?

score 1 · Accepted Answer · answered Nov 02 '16 at 20:22

You are calling the python executable, which will always start an interpreter. You likely need to call the executable that you're working with directly. I'm not super familiar with Flask, but I assume it's actually running a WSGI application, and if you're using a virtual environment or something, the shell you're opening won't know where that virtual Python is, since the path is set dynamically. Something like this will get the correct executable path:

import sys

try:
    import uwsgi
    ex = os.path.join(uwsgi.opt['home'], 'bin/python') # or some default location
except (ImportError, KeyError):
    ex = sys.executable

Then call:

 proc = subprocess.Popen(
    [ex,  'myscript.py'],             
    shell=True,
    stdout=subprocess.PIPE
 )

Now, this might not really fix your issue. It also seems like the script is not being passed properly as a parameter to the shell. You might want to mess around with it in the default interpreter or something to double check that the command that it's calling is correct. Alternatively, you should be able to get away with not separating the parameters:

 proc = subprocess.Popen(
    ['python myscript.py'],             
    shell=True,
    stdout=subprocess.PIPE
 )

Thanks! The second change you recommended was all that I needed to get things running. — ABM, Nov 02 '16 at 20:36

Popen in Flask starts Python interpreter

1 Answers1

Linked