1

I am trying to test the request module to trigger a service in the cloud (e.g. Google Cloud Functions/Cloud run) that takes a long time to run.

I have created a test service that provides a Json response object after 45 minutes. I can see in the cloud run logs that this is working correctly and finishes with a 200 status.

@app.route("/")
def test_high_latency_request():

    i=0
    while i < 9:
        time.sleep(300)
        i+=1
        print(str(i*5) + " mins have passed")

    response_json = {
        "asgfd":"dagdfs",
        "fgasdg":"gsadgs"
    }

    return response_json, 200

if __name__=="__main__":
    app.run(debug=True, host="0.0.0.0", port=int(os.environ.get("PORT", 8080)))

However, the response is never picked up by either of the following requests, with the function continuing to run either indefinitely or until it hits the timeout specified, even though the response is available, evident by the Cloud Run logs.

requests.get("https://high-latency-conn-test-hyk3kw2bxq-nw.a.run.app",stream=True)

requests.get("https://high-latency-conn-test-hyk3kw2bxq-nw.a.run.app",timeout=(3600,3600))

I have also tried initiating a requests.Session() instance but that didn't help either.

How can I handle high latency requests with this module? Or should I be using a different library.

steak_ale_piethon
  • 67
  • 1
  • 4
  • Did you tried to call this endpoint asynchronous? If I understand you correctly, you just want to trigger a service inside the cloud, so you need not really a response... See Link: https://stackoverflow.com/questions/9110593/asynchronous-requests-with-python-requests – droebi Sep 01 '21 at 15:59
  • Thanks, I'll look into sending these requests in an asynchronous fashion. I do want to capture the response though, as I will be making these requests as part of an airflow dag and I want to make sure the task has completed successfully before moving on to the next task. – steak_ale_piethon Sep 02 '21 at 08:34
  • @steak_ale_piethon I have this exact issue. Did you find a way to capture the response from the asynchronous request? – femeloper Aug 23 '23 at 10:43

0 Answers0