how to pause an R script until a file has been downloaded?

Question

I have the following piece of code which runs a python selenium script that downloads a report

library(reticulate)
source_python("C:/Users/Gunathilakel/Desktop/Dashboard Project/Dashboard/PBK-Report-Automation-for-Dashboard-master/pbk automation.py")

I want to make sure that R waits until the earlier piece of code has downloaded a file into my downloads folder before the script executes this next piece of code

my.file.copy <- function(from, to) {
   todir <- dirname(to)
   if (!isTRUE(file.info(todir)$isdir)) dir.create(todir, recursive=TRUE)
   file.copy(from = from,  to = to,overwrite = TRUE)
 }

 my.file.copy(from = "C:/Users/Gunathilakel/Downloads/Issued and Referral Charge.csv",
                to = "C:/Users/Gunathilakel/Desktop/Dashboard Project/Dashboard/RawData/Issued and Referral Charge2020.csv")

I found this question How to make execution pause, sleep, wait for X seconds in R?

But is it possible to wait for execution until a file has been downloaded?

Change your Python script to do a synchronous download – Hong Ooi Jun 10 '20 at 21:30 — Hong Ooi, Jun 10 '20 at 21:30
Seconding @HongOoi's comment as the best answer. – dpritch Jun 11 '20 at 03:33 — dpritch, Jun 11 '20 at 03:33

score 2 · Accepted Answer · answered Jun 10 '20 at 22:02

2

actually found a solution from this question

Wait for file to exist

library(reticulate)
source_python("C:/Users/Gunathilakel/Desktop/Dashboard Project/Dashboard/PBK-Report-Automation-for-Dashboard-master/pbk automation.py") 

while (!file.exists("C:/Users/Gunathilakel/Downloads/Issued and Referral Charge.csv")) {
  Sys.sleep(1)
}


 # Moves Downloaded CSV file of Issued_and_refused from Downloads --------
 my.file.copy <- function(from, to) {
   todir <- dirname(to)
   if (!isTRUE(file.info(todir)$isdir)) dir.create(todir, recursive=TRUE)
   file.copy(from = from,  to = to,overwrite = TRUE)
 }

 my.file.copy(from = "C:/Users/Gunathilakel/Downloads/Issued and Referral Charge.csv",
                to = "C:/Users/Gunathilakel/Desktop/Dashboard Project/Dashboard/RawData/Issued and Referral Charge2020.csv")

answered Jun 10 '20 at 22:02

Nathan123

763
5
18

Just beware that if that file doesn't end up where you expect, then you will have an infinite loop. It might be worth adding an escape hatch, or better yet change the Python script to do a synchronous download as suggested by @HongOoi. – dpritch Jun 11 '20 at 03:40
@dpritch I am using selenium to download the file using a command like the following `driver.find_element(By.XPATH,'//*[@id="ctl00_ContentPlaceHolder1_rvReports_ctl09_ctl04_ctl00_Menu"]/div[7]/a').click() # This clicks on the csv file to be downloaded` can you share any resources on how to do a synchronous download inside a python script. – Nathan123 Jun 12 '20 at 17:17
According to [this](https://stackoverflow.com/a/34338926/5518304) it appears that in the general case there is no way to wait until the download is completed and what you have proposed is probably the best recourse. In some cases though, you might be able to extract the target URL using Selenium, in which case you could perform the download using e.g. the requests package which would give you much greater control over the download attempt. – dpritch Jun 14 '20 at 16:04
can also look for file size with for loop and sleep. If same then break loop – KKW Nov 16 '21 at 16:05

how to pause an R script until a file has been downloaded?

1 Answers1