Stack Overflow
Stack Overflow

Questions tagged [requests-futures]

A Python library built for asynchronous HTTP requests.

is a Python library built for asynchronous HTTP requests.

GitHub: https://github.com/ross/requests-futures

11 questions
15
votes
4 answers

Efficiently download files asynchronously with requests

I want to download files as fast as possible with python.Here is my code import pandas as pd import requests from requests_futures.sessions import FuturesSession import os import pathlib from timeit import default_timer as timer class…
user2650277
  • 6,289
  • 17
  • 63
  • 132
3
votes
1 answer

how to pass headers to async with session.get()

I would like to know how to pass the header in the following get call headers = { 'User-Agent': 'Mozilla' } async def fetch(url, session): async with session.get(url) as response: resp = await response.read() return resp I tried the…
Akhil Pakkath
  • 283
  • 1
  • 5
  • 14
1
vote
0 answers

Concurrent Futures: Best Settings for Multiple Async Requests?

I'm wanting to optimize a script to make as many network requests as possible. I see the max_workers is maybe limited to the number of cores on the machine. Does this mean if this script is run on an EC2 machine, for example t2.2xlarge with 8 vCPUs…
Vinnie James
  • 5,763
  • 6
  • 43
  • 52
0
votes
0 answers

How can make the requests_futures module not create an error with Heroku?

When I try deploying my Flask app to Heroku, I get a ModuleNotFound Error for requests_futures. I have included it in requirements.txt and I am still getting the issue. Heroku deploys fine without the requests_futures module. Is there anything I can…
Varun Sahni
  • 1
  • 2
0
votes
0 answers

For what reason would a Flask application run fine, but fail to load a module in the unittest?

So, I created a small flask API. Basically a wrapper for another API. I used requests-futures to send multiple calls asynchronously. I set up my virtual environment. Everything works exactly as I intend. However, when I try to run some tests I've…
CVerica
  • 327
  • 1
  • 10
0
votes
1 answer

How to use 'continue' with requests-futures?

I have a code where I want to fetch results from the web. And I want my code to wait if the internet connection goes down. But when I tried to perform a continue statement within the for loop it's giving me some tricky results. And I don't know how…
Mr. lindroid
  • 162
  • 1
  • 12
0
votes
0 answers

How to run retry fro the pending header items using futures = session.get(url, headers=header for header in headers1)

I am creating my header list using this function from CSV file.
Chandan
  • 1
0
votes
1 answer

How to measure time of a request with requests_futures even an exception occurs?

I would to measure the time of an HTTP request with python requests_futures even an exception occurs. Although the elapsed inherited from the original requests module can be used to measure a successful request. But it can not be used to measure a…
user1424739
  • 11,937
  • 17
  • 63
  • 152
0
votes
1 answer

python flask: asynchronous requests

In a flask view I receive data via an API call and this data has to be passed to an external API that is sometimes very slow. So I want the view to return a positive status code while the request is being handled async. I have tried with…
caliph
  • 1,389
  • 3
  • 27
  • 52
0
votes
2 answers

Faster way to make asynchronous requests

I'm trying to use the requests-futures library to send a batch of asynchronous HTTP requests and identify the presence or absence of a specific bytestring in the content of each page. Here is the synchronous version. Please note the actual site I'm…
Brad Solomon
  • 38,521
  • 31
  • 149
  • 235
-2
votes
1 answer

Kucoin How to Get Order Book for futures from api?

I'm trying to find how to get futures order book for all tickers, i successfully found how to get active contracts using : https://api-futures.kucoin.com/api/v1/contracts/active I read the api documentation, it says Get Order Book HTTP Request GET…
One boy
  • 216
  • 1
  • 7