0

I need to upload multiple csv files (dummy multiple files) and apply some operations to each of the files, and then merge the resultant dataframes into one dataframe and download the output as csv file. My raw R code works but I need to put it on shiny in order to automate the process, but somehow when I uploaded two files, I got this error: "subscript out of bounds". Help will be appreciated. This is my original raw R Code that worked:

 files = list.files("G:/JSON to CSV/merge")
 numfiles = nrow(inFile)
 kata_csv1 = list()
 for (i in 1:numfiles)
 {
   kata_csv = function(y){
   JSON_csv = read.csv(y, header = TRUE)
   lastrow = nrow(JSON_csv)
   shift = function(x, n){
    c(x[-(seq(n))], rep(NA, n))
    }
  JSON_csv$companyID1 = shift(JSON_csv$companyID1, 1)
  JSON_csv = JSON_csv[-lastrow, ]
  JSON_csv 
  }
 kata_csv1[[i]] = kata_csv(files[i])
 }

 myMergedData = do.call(rbind, kata_csv1)
 write.csv(myMergedData, "myMergedData.csv", row.names=FALSE)

Here is my Shiny code that did not work:

UI.R:

  ui <- fluidPage(
   fluidPage(
     titlePanel("MY CSV FILES MERGER"),
     sidebarLayout(
       sidebarPanel(
         fileInput("file1",
              "Choose CSV files from directory",
              multiple = TRUE,
              accept=c('text/csv', 
                       'text/comma-separated-values,text/plain', 
                       '.csv')),
         downloadButton('downloadData', 'Download')
       ),
       mainPanel(
         tableOutput('contents')
       )
     )
   )
 )

Server.R:

 library(shiny)
 library(dplyr)
 function(input, output) {
   getData <- reactive({
     inFile <- input$file1
     if (is.null(inFile)){
     return(NULL)
     } else {
  numfiles = length(inFile) 
  kata_csv1 = list()
  for (i in 1:numfiles)
  {

      JSON_csv = read.csv(input$file1[[i, 'datapath']], header = TRUE)
      lastrow = nrow(JSON_csv)
      shift = function(x, n){
        c(x[-(seq(n))], rep(NA, n))
      }
      JSON_csv$companyID1 = shift(JSON_csv$companyID1, 1)
      kata_csv1[[i]] = JSON_csv[-lastrow, ]

    }
    do.call(rbind, kata_csv1)
   }
   })
   output$contents <- renderTable( 
   getData() 
   )
   output$downloadData <- downloadHandler(
    filename = function() { 
    paste("data-", Sys.Date(), ".csv", sep="")
   },
   content = function(file) { 
   write.csv(getData(), file, row.names=FALSE)   
  })
  }
William
  • 340
  • 7
  • 17

1 Answers1

3

The error is because the numfiles that you are calculating as numfiles = length(inFile) gives you the number of column in your inFile dataframe. You'll instead have to use numfiles = nrow(inFile) as the number of rows in the dataframe gives you the number of files that you have uploaded.

[EDIT]: The code that is working for me is:

 library(shiny)
  library(dplyr)
 server <-  function(input, output) {
    getData <- reactive({
      inFile <- input$file1
      if (is.null(inFile)){
        return(NULL)
      }else {
        # browser()
        numfiles = nrow(inFile) 
        kata_csv1 = list()


        for (i in 1:numfiles)
        {

          JSON_csv = read.csv(input$file1[[i, 'datapath']], header = TRUE)
          lastrow = nrow(JSON_csv)
          shift = function(x, n){
            c(x[-(seq(n))], rep(NA, n))
          }
          JSON_csv$companyID1 = shift(JSON_csv$companyID1, 1)
          kata_csv1[[i]] = JSON_csv[-lastrow, ]

        }
        # browser()
        do.call(rbind, kata_csv1)
      }
    })
    output$contents <- renderTable( 
      getData() 
    )
    output$downloadData <- downloadHandler(
      filename = function() { 
        paste("data-", Sys.Date(), ".csv", sep="")
      },
      content = function(file) { 
        write.csv(getData(), file, row.names=FALSE)   
      })
  }

 shinyApp(ui = ui, server = server)
SBista
  • 7,479
  • 1
  • 27
  • 58
  • Hi there, thanks for your contribution. I have changed `numfiles = length(inFile)` to `numfiles = nrow(inFile)`, and there is error, but then, the downloaded output file which should be a dataframe shows only quotes " ". Any clue? Thanks. – William Oct 20 '17 at 11:40
  • Could you attach a sample data file? Without the data file it's a little hard to guess what might be the problem. – SBista Oct 20 '17 at 11:42
  • Hi there. Thanks. I have uploaded two dummy multiple files. What is to be done is to upload these two files, correct the anomalies in the two files using the shift() in the for loop, rbind the two files and down the output as csv file. Actually, the anomaly in the two files to be corrected are (1) first value in the first column need to be deleted and the cells in the 1st column move one step up ,(2) therefater the last row of the dataframe is to be deleted . The shift() is dealing with this problem. Because I am dealing with a lot of files, I want shiny to automate the process. Thanks. – William Oct 20 '17 at 12:23
  • Hi @William , using the sample data you provided I was to download the output file. – SBista Oct 20 '17 at 15:30
  • Hi @SBista, do you mean you were able to download the output csv file using the sample data l provided? – William Oct 20 '17 at 19:13
  • Hi @SBista, this should be the [downloaded output file in csv](https://drive.google.com/file/d/0B0MjygvBH6bsdS0tNzg1X05va1k/view?usp=sharing). I am able to do this with my raw R code. With, my shiny code after I changed `numfiles = length(inFile)` to `numfiles = nrow(inFile)`, I only downloaded a blank file with a quote `" "`. I have combed stackoverflow for solution but have not found. Your contribution will be appreciated – William Oct 20 '17 at 22:21
  • Yes I was able to download a similar csv file with the sample data you provided. By the way, are you opening your app in rstudio browser or any other web browser? – SBista Oct 21 '17 at 03:36
  • I have been running the app in rstudio viewer. When I changed to rstudio pane, it is the same result of downloading a blank file with no format. However, running the app with an external browser allowed me to download a csv file, but still, the csv files contains no data. Is the problem associated with the kind of browser used to run the app? The browser I used was Microsoft Edge. – William Oct 21 '17 at 04:25
  • It shouldn't be related to the browser, just so you know I used chrome. I'll also post the code that's working for me. – SBista Oct 21 '17 at 04:51
  • Hi @SBista, this up vote for you. I am still not sure what is wrong with my code because it is exactly the same as yours, but the difference is that your code is a one-file(app.R) app, while mine is a two-file app(ui.R and server.R). In principle, this should not make any difference. In any case, thanks. Best wishes – William Oct 21 '17 at 05:24
  • Hi @SBista, thanks for your contribution. However,running the app is very very slow when I uploaded multiple files (50 - 100mb), perhaps due to the `for` loop. I know that [lapply is faster](https://stackoverflow.com/questions/23995384/read-and-rbind-multiple-csv-files) than `for` loop in reading multiple files but when [I introduced `for` loop in to my code](https://stackoverflow.com/questions/46872965/uploading-multiple-files-in-shiny-process-the-files-using-lapply-rbind-the-res/46873180#46873180) and ran the app, I got this error message: `ERROR: Invalid 'description' argument`. – William Oct 22 '17 at 11:02