-1

I have multiple URL's like 'https://static.nseindia.com/s3fs-public/2022-09/ind_prs01092022.pdf' and I want to loop through an array of these and download them to a local folder. I saw that I may need to use s3fs, but I am unsure what the bucket name should be. (download file using s3fs)

wanderingtrader
  • 3
  • 1
  • Do you know the URLs anyhow? If so, you can just open a [`requests.Session`](https://requests.readthedocs.io/en/latest/user/advanced/#session-objects) and `get` the files one by one in a loop or in parallel using [`aiohttp`](https://docs.aiohttp.org/en/stable/client_quickstart.html). – StSav012 Sep 13 '22 at 09:13
  • @StSav012 I tried that. Just times out – wanderingtrader Sep 14 '22 at 06:37

1 Answers1

0

It appears the web server doesn't respond unless a user agent is among the request headers. The behavior is fairly common.

import requests

with requests.Session() as s:
    s.get('https://static.nseindia.com/s3fs-public/2022-09/ind_prs01092022.pdf',
           headers={'User-Agent': 'Python'}  # or any non-empty string
    )
StSav012
  • 776
  • 5
  • 15