1

I have a flask app with SocketIO. Due to this, I choose to run the application using the eventlet library. Under the hood, eventlet uses green threads to achieve concurrency if I'm not mistaken.

In my app, I want to spawn a process and stream the output over web sockets. Below is a toy example

# eventlet==0.25.1
# flask==1.1.1
# flask-socketio==4.2.1

import eventlet
import subprocess
from flask import Flask, jsonify
from flask_socketio import SocketIO

app = Flask(__name__)
socketio = SocketIO(app)
socketio.init_app(app, cors_allowed_origins='*')

@app.route('/ping')
def start_ping():
    eventlet.spawn_n(ping)
    return jsonify(success=True)


@app.route('/hello')
def hello():
    return jsonify(data='Hello World')


def ping():
    proc = subprocess.Popen(
        ('ping', '-t', 'google.com'),
        stdout=subprocess.PIPE,
    )
    for line in proc.stdout:
        eventlet.sleep(0.5)
        socketio.emit('log_receive', str(line))


if __name__ == '__main__':
    socketio.run(app)
  1. User hits the /ping endpoint.
  2. The ping() function is executed in a green thread which runs the ping command in a child process
  3. lines are read from the subprocess's stdout and emited via web sockets
  4. eventlet.sleep(0.5) is used to give other parts of the application a chance to run.

Question:

for line in proc.stdout: is blocking. Until something comes through stdout, eventlet.sleep(0.5) will not execute and therefore the rest of the application isn't given a chance to run. Thus rendering the app, unresponsive.

I came across this question on how to do non-blocking reads from subprocess.PIPE and the suggestion is to essentially use a separate thread to do the reading.

Unfortunately, I can not use a separate thread because of the concurrent/coroutine programming model I'm restricted to because of eventlet/greenlet.

I could use fcntl() to do non-blocking reads but I'm on windows and so it isn't an option

What's an alternative to avoid having the application be at mercy of the subprocess's stdout

Community
  • 1
  • 1
Krimson
  • 7,386
  • 11
  • 60
  • 97

0 Answers0