Why is python-requests to localhost slow?

Question

I have a simple flask server defined like so:

import sys

import flask
from flask import request

app = flask.Flask(__name__)
port = 4057

@app.route('/search', methods=['POST'])
def search():
  request.json['query']
  results = ['fake', 'data']
  return flask.jsonify(results)

if __name__ == '__main__':
 app.config['TEMPLATES_AUTO_RELOAD'] = True
 app.run(host='0.0.0.0', port=port, debug=(port != 80))

I have a simple client defined like this:

import json

import requests

headers = {'content-type': 'application/json'}
resp = requests.post('http://localhost:4057/search', json.dumps({'query': 'foo'}), headers=headers)
print resp.content

The client works, but it takes like 3 seconds to complete the request.

curl completes in like half a second:

curl 'http://localhost:4057/search' -H 'Content-Type: application/json' -d '{"query": "foo"}'

Does the time taken change when you change the flask server to be `threaded=True` in the `app.run()`? — Sudheesh Singanamalla, Dec 12 '17 at 05:59
How are you measuring the time it takes, and what Python implementation are you using? I bet most of the time is being spent on `import requests` and Python's start up. — Francisco, Dec 12 '17 at 05:59
@FranciscoCouzo Good guess, but no, if I just `import requests` and `print "foo"` it's < .5 seconds. — Jesse Aldridge, Dec 12 '17 at 06:03
It seems like you are timing the entire script instead fo the core logic. Keep in mind that the `requests` import takes considerable amount of time. — sayan, Dec 12 '17 at 06:03
@SudheeshSinganamalla If flask was the problem then the `curl` would also be slow, no? — Jesse Aldridge, Dec 12 '17 at 06:04
what if you change it to `127.0.0.1`? maybe some odd resolution. — salparadise, Dec 12 '17 at 06:04
I just tested you example on my machine(`localhost`), for me it took on average ~ `110 MS` — Sohaib Farooqi, Dec 12 '17 at 06:07
BTW: `'0.0.0.0'` connects server to all network cards in computer, not only `127.0.0.1` but also `Wi-Fi` and wired connection. — furas, Dec 12 '17 at 08:20

salparadise · Accepted Answer · 2017-12-12T06:43:35.537

16

Try 127.0.0.1 There maybe some odd name resolution rules fumbling requests.

Ah ok, this is my work machine. I took a look at /etc/hosts and saw ~200 routes defined that I didn't realize were there

As mention in the comments, this does not explain the odd behavior replicated using curl.

edited Dec 12 '17 at 06:43

answered Dec 12 '17 at 06:07

salparadise

5,699
1
26
32

1

Ah ok, this is my work machine. I took a look at `/etc/hosts` and saw ~200 routes defined that I didn't realize were there. – Jesse Aldridge Dec 12 '17 at 06:10
4

I don't think this answer solve the question essentially. This question is interesting so I have done a wide search. There is a possible reason. As modern OS usually support both ipv6 and ipv4, `localhost` is first resolved to ipv6 address but fail in default timeout window (usually one second), then fallback to ipv4 address. What strange is that curl works well, maybe it perfers ipv4? Or this reason is also not the key point. – Sraw Dec 12 '17 at 06:28
@Sraw thanks. I used [this solution](https://superuser.com/a/436944/273248) for configuring IPv4 to take precedence over IPv6 when accessing `localhost`. – yair Dec 29 '19 at 15:39

forcedude0 · Answer 2 · 2023-02-12T05:47:00.097

I've recently encountered a similar issue with slow connections from requests to 'localhost', and also found that using '127.0.0.1' was much faster.

After some investigation, I found that in my case, the slow connections on Windows were related to urllib3 (the transport library used by requests) attempting an IPv6 connection first, which is refused since the server is listening on IPv4 only. For Windows only, however, there is an unavoidable 1 sec timeout on socket.connect for refused TCP connections, while the OS 'helpfully' retries the connection up to 3 times. (See: Why do failed attempts of Socket.connect take 1 sec on Windows?).

On Linux, socket.connect fails immediately if the connection is refused, and it can attempt the IPv4 connection without any delay. In addition, some software, like curl, support limiting name resolution to only IPv4 or IPv6. Unfortunately, requests and urllib3 do not support this, and do not seem to plan to support this anytime soon (See: https://github.com/psf/requests/issues/1691)

score 0 · Answer 3 · answered Mar 27 '18 at 13:14

For those trying to figure this out as no one gave a clear solution here:

I encountered this issue a while ago, and noticed that the Flask test server is not concurrent. The Python Requests takes FOREVER to retrieve the data from your flask app unless you make it concurrent. Enabling "Threaded" in your app.run option will do the trick!

app.run(port=5000, debug=True, threaded=True)

Why is python-requests to localhost slow?

3 Answers3