0

I have a URL list in R and want to download it.

However, I do not like to use download.file() as this function gives me error when my list is big (more than 500 links):

InternetOpenUrl failed: 'The operation timed out'

Here is the download list:

list_1 <- c("https://e4ftl01.cr.usgs.gov//MODV6_Cmp_A/MOLT/MOD11B3.006/2004.12.01/MOD11B3.A2004336.h18v08.006.2015240192212.hdf",
        "https://e4ftl01.cr.usgs.gov//MODV6_Cmp_A/MOLT/MOD11B3.006/2004.12.01/MOD11B3.A2004336.h19v07.006.2015240192220.hdf",
        "https://e4ftl01.cr.usgs.gov//MODV6_Cmp_A/MOLT/MOD11B3.006/2004.12.01/MOD11B3.A2004336.h19v08.006.2015240192212.hdf",
        "https://e4ftl01.cr.usgs.gov//MODV6_Cmp_A/MOLT/MOD11B3.006/2004.12.01/MOD11B3.A2004336.h18v07.006.2015240192213.hdf")

list_1

Any idea to download this list using a better function in R?

Thanks so much.

Canada2015
  • 187
  • 1
  • 12
  • Thanks for your comment. Before posting, I went through the existed questions suggested by the website. However, I did not find the similar one. Could you please share the link for the one you mentioned? – Canada2015 May 15 '19 at 19:13
  • If you are getting an operation time out, it's likely that you are probably just overloading the server. Asking for a large number of files at one time is not a nice thing to do to a server (especially if those files are large). Use a loop and add some wait time between requests. – MrFlick May 15 '19 at 19:13
  • Thanks. Despite using **options(timeout=600)**, this function still give me the error sometime. – Canada2015 May 15 '19 at 19:15
  • MrFlick, please remove this from the page: This question already has an answer here. Because as I said that is not the case – Canada2015 May 15 '19 at 19:16
  • If you want to automatically retry on failure, then try an option like: https://stackoverflow.com/questions/48067906/http-error-400-in-r-error-handling-how-to-retry-instead-of-forcing-to-stop – MrFlick May 15 '19 at 19:16
  • 1
    You've tried all 6 answers on all 4 questions and none of them worked? If that's the case, you should add whatever information would set your question apart from the previous ones. – camille May 15 '19 at 19:24
  • Good. Let me check all those methods. If they worked, I will delete my question here. – Canada2015 May 15 '19 at 19:28
  • @lilster , I tried to employ your code in order to solve my question here. https://stackoverflow.com/questions/48067906/http-error-400-in-r-error-handling-how-to-retry-instead-of-forcing-to-stop/56157389#56157389 But I could not. Could you please help me with that? Appreciate. – Canada2015 May 15 '19 at 20:50

0 Answers0