Python Flask REST API freezes during a call to subprocess command

Question

We have a working REST API that handles many diff endpoints for FrontEnd. During one specific endpoint where subprocess command should process operation all other endpoints halts and wait for the subprocess to finish. Can anyone help me to understand why it's happening? My thoughts that Python Flask runs asynchronously?

[...]

class withdrawCrypto(Resource):
    def get(self):
        auth = json.loads('{"ERROR" : "User authentication failed!"}')
        wrongAmount = json.loads('{"ERROR" : "Wrong amount"}')
        wrongWlt = json.loads('{"ERROR" : "Invalid wallet provided. Please check the wallet addr!"}')
        notEnough = json.loads('{"ERROR" : "You don\'t have enough crypto to withdraw this amount"}')
        account = str(request.args.get('account'))
        token = str(request.args.get('token'))
        wallet = str(request.args.get('wlt'))
        count = int(request.args.get('count'))

        if len(wallet) != 34:
            return jsonify(data=wrongWlt)

        if wallet[0] != 'B':
            return jsonify(data=wrongWlt)

        cursorLG.execute("select balance from btc WHERE login=%s;", account)
        checkBalance = cursorLG.fetchall()

        if checkBalance[0]['balance'] < int(count):
            return jsonify(data=notEnough)

        cursorLG.execute("select cred from accounts WHERE login=%s;", account)
        userCheck = cursorLG.fetchall()

        if userCheck[0]['secret'] == token:
            if count and int(count) > 0:
                host = credentials['rpc']
                user = credentials['rpcuser']
                passwd= credentials['rpcpassword']
                timeout = credentials['rpcclienttimeout']
                command = 'bitcoin-cli -rpcconnect=' + host + ' -rpcuser=' + user + ' -rpcpassword=' + passwd  + ' -rpcclienttimeout=' + timeout + ' sendtoaddress ' + wallet + ' ' + str(count)
                result = subprocess.check_output(command,shell=True).strip()

                cursorLG.execute("select balance from btc WHERE login=%s", account)
                current = cursorLG.fetchall()

                setNew = int(int(current[0]['balance']) - int(count))

                cursorLG.execute("replace into btc (login, balance, lastwithdrawalwlt) values (%s, %s, %s) ", (account, setNew, wallet))
                return jsonify(data=result.decode("utf-8"))
            else:
                return jsonify(data=wrongAmount)
        else:
            print('Failed Crypto withdrawal! Actual passw / user sent: ', userCheck[0]['secret'], token)
            return jsonify(data=auth)

[...]

# Serve the high performance http server
if __name__ == '__main__':
    http_server = WSGIServer(('', 9000), app)
    http_server.serve_forever()

All other endpoints work fast without any delays. Any help appreciated.

Flask runs **not** asynchronous. Thus, you have a single thread. Maybe [this](https://stackoverflow.com/questions/10938360/how-many-concurrent-requests-does-a-single-flask-process-receive) helps. — above_c_level, Mar 24 '20 at 10:32

score 0 · Accepted Answer · answered Mar 24 '20 at 12:09

The problem was:

result = subprocess.check_output(command,shell=True).strip()

More specific:

shell=True

which waits until the process stops and reads the

STDOUT

As a quick workaround installed gunicorn app and served with a flag --timeout 120 --workers 20

So now 1 worker busy, 19 still returning other requests.

Python Flask REST API freezes during a call to subprocess command

1 Answers1