6

I'm a new python programmer and was tasked to do a simple app to make an API call. It's working fine and all but however I am not satisfied with the speed. I have tried it on both slow speed internet connection and faster speed internet connection. On average, the time taken to make the API call ranging from 1.3 to 2.4 sec. I have tested this on 5mbps to 80mbps internet connection and both shown similar result (plus minus 0.5 sec difference).
 
The time taken doesn't look too bad on paper but however, when I checked in the server where the call was made to, the processing time is only around 0.2 - 0.5 seconds, meaning there's 1 - 2 seconds of time lost during the request process.
 
Here's the code that I'm using: 

import requests
import json
import time
import uuid
import requests.packages.urllib3

requests.packages.urllib3.disable_warnings() 

#get api call start time
startTime = time.time()

#make api call
r=requests.post(APIUrl, data=payload, headers=headers, verify=False)

#get api call finish time
endTime = time.time()

#calculate time taken
totalTimeTaken = str(float(round((endTime - startTime ),3)))

json_obj = r.json(strict=False)
print "Response: "+str(json_obj['response'])
print "Elapsed: "+str(r.elapsed)
print "Time Taken: "+totalTimeTaken
akmalmzamri
  • 1,035
  • 1
  • 15
  • 31
  • 3
    How did you measure the server time? There is nothing invalid about the way you use `requests.post()` here, this is all server and data transit time you are waiting for. – Martijn Pieters Mar 29 '17 at 14:23
  • 2
    `requests` is fine. There may be lots of other reasons: your computer is slow or the server is slow, or the payload is huge... – ForceBru Mar 29 '17 at 14:24
  • Check with other address – RaminNietzsche Mar 29 '17 at 15:06
  • 2
    @MartijnPieters, ForceBru : I disagree. I have full control of the server as well as the client. Using Chrome to do a GET request of the resource, I can see the server respond instantly. But with requests, there is a 4 second delay before the server sees the request, and then the response is near immediate. Additionally, use of the timeout parameter, set to 1 second, illustrates further that the overall transaction takes 4.4 seconds, but the request did not time out. Thus. somewhere else there is a slow-down other than the data retrieval. FWIW server response time is <40ms. – xobes May 17 '17 at 14:58
  • see also: http://stackoverflow.com/questions/32505410/python-requests-module-is-very-slow-on-specific-machine – xobes May 17 '17 at 15:00
  • 1
    @xobes: sorry, what is your relationship to this question? I don't ever see such delays in `requests`, such a delay is unique to your setup it seems. Is Chrome set up to use a proxy? How fast are nameserver lookups? Is there anything installed on the server that might tarpit specific user agents? – Martijn Pieters May 17 '17 at 15:25
  • for me the answer was found in http://stackoverflow.com/questions/28521535/requests-how-to-disable-bypass-proxy – xobes May 17 '17 at 16:20
  • I encountered the same issue, the requests.post very slow, it took 4ms when I tested the WEB API with postmane, but it took 2000+ms ( like 2040ms ) when using the requests.post method, interesting, why requests took so much? – Liping Huang Jun 10 '19 at 09:14
  • @LipingHuang I solved this by using `requests.Session()`. You can refer here: https://stackoverflow.com/questions/28521535/requests-how-to-disable-bypass-proxy – akmalmzamri Jun 10 '19 at 09:57
  • 4
    @akmalhakimi1991 thanks, I finally catched the problem, maybe this is the requests library bugs(?) on windows os(win10), if I config the url like 'http://localhost:port', the `create_connection` takes almost 2000ms to get the connection, but I config the url like 'http://127.0.0.1:port', it takes only 4ms to get the connection. – Liping Huang Jun 11 '19 at 09:01
  • 1
    @LipingHuang Dude, you are my hero! Thank you so much! You should actually post this as answer as this is exactly what I was experiencing. From 2 seconds down to 20ms! – timothy3001 May 26 '20 at 19:32

1 Answers1

5

Adding @Liping Huang's comment as an answer so it's more visible as most people don't read comments.

Changing the URL from localhost to 127.0.0.1 reduces the overhead by almost 2 seconds to almost nothing.

This means that DNS lookups have been slow for the OP, it's not related to requests.post

hek2mgl
  • 152,036
  • 28
  • 249
  • 266
Motti
  • 110,860
  • 49
  • 189
  • 262