1

I would like to create a programme that can download a specific CDF file from this website:

http://research.ssl.berkeley.edu/data/psp/data/sci/fields/l2/mag_RTN_4_Sa_per_Cyc/2018/10/

For example I would like the user to get asked which specific date he would like to download and the programme to download the file and store it as data.

In this site all the file names end with the date. For example:

psp_fld_l2_mag_RTN_4_Sa_per_Cyc_20181003_v01.cdf

Where, 20181003 means 2018/10/03 (the date)

Is this possible?

1 Answers1

0

as this is a static website and doesnt involve javascript in loading the files, you can go ahead with requests to get the html from the url using r = requests.get(url)

you can then go ahead and get all the links using beautifulsoup's web scraping, and finally save the files using

r = requests.get(fetched_url,allow_redirects=True)

open(filename,'wb').write(r.content)

Yash
  • 1,271
  • 1
  • 7
  • 9
  • Thank you very much! However, would 'nt this download all of the filles? What if I wanted to download one file at a time according to to the date the user prefers? –  Sep 17 '20 at 13:49
  • this would download one file at a time. If you want all the files, add all the links from beautifulsoup to a list and download each of them using a loop. – Yash Sep 17 '20 at 15:08
  • However I would have to specify the files name inside open(filename,'wb').write(r.content) right? Is there a way to ask the user which date he wants and then the code to find the specific file? Thanks a lot! –  Sep 17 '20 at 15:27
  • @Jokerp Sure. use ```input()``` to ask for user input, and manipulate the data however you want. – Yash Sep 17 '20 at 15:35
  • Thanks @Yash could you provide a small code of how I coulkd do this? am really not able to do it ! thanks again –  Sep 17 '20 at 15:42
  • Sorry @Jokerp cant help you with whole code. But [input](https://stackoverflow.com/questions/19615427/how-to-enter-input-in-python/19615442), [requests](https://stackoverflow.com/questions/18810777/how-do-i-read-a-response-from-python-requests), [downloading File](https://www.tutorialspoint.com/downloading-files-from-web-using-python) can be starters for help – Yash Sep 17 '20 at 15:50