I am attempting to download some files from a website where I have a login. I am able to generate the list of file URLs but simply logging in from my browser and using download.file in R does not scrape the file.
I tried the solution presented here (Reading information from a password protected site), but it also did not work in R or directly from the browser as it sent me back to the login page.
I did notice that if I am logged in and click on the generated links to each of the files, I get sent to a new URL related to that file. If I copy this new link into the URL input of download.file, I am able to download the file from within R.
My question is if there is a method to copy this redirected URL into R so that I am able to automate the downloading? I can directly access the URL with browseURL, but I cannot bring back this new URL into R.
If there is another more efficient way to access data that requires a login from R, I am also happy to be linked to this.
Thanks!
