8

I have a CSV DTOutput("table1") file with several columns and their values in it or how it should be done using dput() in R shiny, I would like to add those to the bottom column as headers and values.

How I should bring it in R shiny? could someone assist me?

CSV Data

ID  Type   Range
21  A1     100
22  C1     200
23  E1     300
ID Range  Type    Period
24 500    A2      2005
26 100    G2      2008
28 300    C3      2010

Expected Output

ID  Type   Range ID Range Type Period
21  A1     100   24  500  A2   2005
22  C1     200   26  100  G2   2008
23  E1     300   28  150  C3   2010

app.R

library(shiny)
library(reshape2)
library(DT)
library(tibble)


###function for deleting the rows
splitColumn <- function(data, column_name) {
  newColNames <- c("Unmerged_type1", "Unmerged_type2")
  newCols <- colsplit(data[[column_name]], " ", newColNames)
  after_merge <- cbind(data, newCols)
  after_merge[[column_name]] <- NULL
  after_merge
}
###_______________________________________________
### function for inserting a new column

fillvalues <- function(data, values, columName){
  df_fill <- data
  vec <- strsplit(values, ",")[[1]]
  df_fill <- tibble::add_column(df_fill, newcolumn = vec, .after = columName)
  df_fill
}

##function for removing the colum

removecolumn <- function(df, nameofthecolumn){
  df[ , -which(names(df) %in% nameofthecolumn)]
}

### use a_splitme.csv for testing this program

ui <- fluidPage(
  sidebarLayout(
    sidebarPanel(
      fileInput("file1", "Choose CSV File", accept = ".csv"),
      checkboxInput("header", "Header", TRUE),
      actionButton("Splitcolumn", "SplitColumn", class = "btn-warning" ),
      uiOutput("selectUI"),
      
      
      actionButton("replacevalues", label = 'Replace values', class= "btn-Secondary"),
      actionButton("removecolumn", "Remove Column"),
      actionButton("Undo", 'Undo', style="color: #fff; background-color: #337ab7; border-color: #2e6da4"),
      actionButton("deleteRows", "Delete Rows"),
      textInput("textbox", label="Input the value to replace:"),
      actionButton('downloadbtn', label= 'Download'),
    ),
    mainPanel(
      DTOutput("table1")
    )
  )
)

server <- function(session, input, output) {
  rv <- reactiveValues(data = NULL, orig=NULL)
  
  observeEvent(input$file1, {
    file <- input$file1
    ext <- tools::file_ext(file$datapath)
    
    req(file)
    
    validate(need(ext == "csv", "Please upload a csv file"))
    
    rv$orig <- read.csv(file$datapath, header = input$header)
    rv$data <- rv$orig
  })
  
  output$selectUI<-renderUI({
    req(rv$data)
    selectInput(inputId='selectcolumn', label='select column', choices = names(rv$data))
  })
  
  
  observeEvent(input$Splitcolumn, {
    rv$data <- splitColumn(rv$data, input$selectcolumn)
  })
  
  observeEvent(input$deleteRows,{
    if (!is.null(input$table1_rows_selected)) {
      rv$data <- rv$data[-as.numeric(input$table1_rows_selected),]
    }
  })
  
  output$table1 <- renderDT(
    rv$data, selection = 'none', server = F, editable = T
  )
  #includes extra column after the 'select column' and replaces the values specified 'Input the value to replace:'
  observeEvent(input$replacevalues, {
    rv$data <- fillvalues(rv$data, input$textbox, input$selectcolumn)
  })
  #Removing the specifield column through select column
  observeEvent(input$removecolumn, {
    rv$data <- removecolumn(rv$data,input$selectcolumn)
  })
  observeEvent(input$Undo, {
    rv$data <- rv$orig
  })
  #Storing the csv file through download button
  observeEvent(input$downloadbtn,{
    write.csv(rv$data,'test.csv')
    print ('file has been downloaded')
  })
  observeEvent(input$downloadbtn, {
    showModal(modalDialog(
      title = "Download Status.",
      paste0("csv file has been downloaded",input$downloadbtn,'.'),
      easyClose = TRUE,
      footer = NULL
    ))
  })
}

shinyApp(ui, server)
Kevin Tracey
  • 154
  • 1
  • 16
  • 1
    Did you try adding a func with data preprocessing wrapped inside? – rkabuk Nov 22 '21 at 08:49
  • @rkabuk, I haven't heard that Datawrapper can be used because I'm new to R shiny. Could you perhaps assist me in providing further information on how to use? – Kevin Tracey Nov 22 '21 at 09:18
  • as i understand the code of yours you have a button which loads dataset into shinyapp. You can convert a little bit this button to be helding a function which will load the data set , then preprocess it. as far as i remember there is a parameter in read.csv() func which allows you to load data from certain row. – rkabuk Nov 22 '21 at 10:09
  • Is there a way to manually choose a column row in a Datatable and then automatically create it a side header with its values? – Kevin Tracey Nov 22 '21 at 15:42
  • I think it can be done using Cbind, But I do not know how to explore it in Datable(Rshiny) – Kevin Tracey Nov 23 '21 at 08:13
  • I don't understand what you are trying to achive. Please provide your input data via copying the output of `dput(your_input_data)`. – ismirsehregal Nov 25 '21 at 12:06
  • @ismirsehregal, Thank you.. Could you provide me with more details to explore this.? – Kevin Tracey Nov 26 '21 at 06:10
  • Please see [this](https://stackoverflow.com/a/49995752/9841389). – ismirsehregal Nov 26 '21 at 06:13
  • 1
    @KevinTracey Please can you provide more information about the csv? If i'm no mistaken, You want everything after the 4th row to represent new columns and bind them together? In this case is possible because both resulting df's will have the same number of rows. – jpdugo17 Nov 28 '21 at 17:38
  • @jpdugo17, Thank you for requesting the clarification. The column row does not have to be identified every fourth row, it is dependent on the column headers to be found (regardless of the row). I hope I've made myself clear. – Kevin Tracey Nov 29 '21 at 04:04
  • @KevinTracey Do you have a way to read your csv file in R and use, for example ,`head(dput(object), 30)` to get the first 30 rows so we can see the output that you are getting in order to split the data accordingly? Thanks! – jpdugo17 Nov 29 '21 at 18:26
  • @jpdugo17, Could you please explain how to utilize 'head(dput(object), 30)'? As far as I can tell, the head of 30 rows is visible, but my original datasets will be larger, and we don't know where the column headings in rows will appear. – Kevin Tracey Nov 30 '21 at 03:59
  • First try to import the csv with using this code as template `df <- read_csv("path to csv file")`, if it works (doesn't matter is the data is dirty) you can do `dput(head(df))` and copy the resulting output from the console. Maybe there's a chance to clean the data from there. Sorry about the last comment, i reversed the order of the functions. – jpdugo17 Nov 30 '21 at 04:14
  • @jpdugo17, As I checked your code can be used in the R console, but how to use the same in R shiny.? – Kevin Tracey Nov 30 '21 at 05:56
  • @KevinTracey That is to see how the table will look like when uploaded into the shiny app. Can you edit your question if possible providing the result from dput? – jpdugo17 Nov 30 '21 at 06:29
  • @jpdugo17, I edited the question. if you spare some time for me, Can we connect through 1:1. if you're fine with it. Let me know how I may get in touch with you to explore more about the solution. – Kevin Tracey Nov 30 '21 at 07:34
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/239684/discussion-between-kevin-tracey-and-jpdugo17). – Kevin Tracey Nov 30 '21 at 07:36
  • 2
    @KevinTracey, what guarantees can you make about the structure of the csv? For example, will the "bottom table" always have the same number of rows as the "top table"? Will it always have at least one column with the same name as one of the columns in the "top table"? Could there be more than two sub-tables, or will there always be exactly two? – A. S. K. Dec 22 '21 at 21:22
  • @A.S.K., please find my inline answers. Question 1: will the "bottom table" always have the same number of rows as the "top table"? Answer: No it will not. Question 2: Will it always have at least one column with the same name as one of the columns in the "top table"? Yes. Question 3: Could there be more than two sub-tables, or will there always be exactly two? Answer: There will be more than 2 sub-tables for sure. – Kevin Tracey Dec 23 '21 at 07:15
  • @KevinTracey, do the rows that end up next to each other describe the same observation? Or are we displaying unrelated columns next to each other because the table is more human-readable that way? – A. S. K. Dec 24 '21 at 03:56
  • @A.S.K., These are unrelated columns only – Kevin Tracey Dec 24 '21 at 10:05
  • In that case, is it important to combine the columns into a single data frame? Or would it be okay to have multiple data frames and display them side by side? – A. S. K. Dec 24 '21 at 16:07

2 Answers2

2

Not sure if this helps but I was able to get your desired output by filtering each column for rows containing one of the column names and cbinding them together.

observeEvent(input$Splitcolumn, {


    df <-rv$data %>% 
      select(-1)

    # get existing column names from dataframe
    temp <- names(df)

    # find rows in first column that contain a column name
    inds <- which(df[1] == temp[1] | df[1] == temp[2] | df[1] ==  temp[3])

    # gather rows in first column that are after the row with column name
    df2 <- df[sort(unique(inds+1:nrow(df))), ] %>% select(1)

    # change df2 column name to row name
    new1 = df %>%  slice(inds:inds) %>%  select(1)
    names(df2)[1] <- paste0(as.character(new1[[1]]))
    
    #- repeat for rest of columns 
    inds2 <- which(df$Type == temp[1] | df$Type == temp[2] | df$Type ==  temp[3])
    new1 = df %>%  slice(inds2:inds2) %>%  select(2)
    df3 <- df[sort(unique(inds2+1:nrow(df))), ] %>% select(2)
    names(df3)[1] <- paste0(as.character(new1[[1]]))
    #
    inds3 <- which(df[3] == temp[1] | df[3] == temp[2] | df[3] ==  temp[3])
    new1 = df %>%  slice(inds3:inds3) %>%  select(3)
    df4 <-  df[sort(unique(inds3+1:nrow(df))), ] %>%  select(3)
    names(df4)[1] <- paste0(as.character(new1[[1]]))
    #
    inds4 <- which(df[4] == 'Period')
    new1 = df %>%  slice(inds4:inds4) %>%  select(4)
    df5 <-  df[sort(unique(inds4+1:nrow(df))), ] %>%   select(4)
    names(df5)[1] <- paste0(as.character(new1[[1]]))
    
    #- cbind new dfs and remove na
    newdf <- cbind(df2,df3,df4,df5) %>% 
      filter(., !is.na(.[1]))

    #- filter original df to remove rows present in new df using ID column.
    df <- df %>% filter(., !ID%in%newdf$ID) %>% 
      filter(., !ID%in%temp[1]) %>% 
      select(., 1,2,3)
    newdf <- cbind(df, newdf)
    rv$data <- newdf
    #rv$data <- splitColumn(rv$data, input$selectcolumn)
  })


    
  • Yes it works, but the code `inds4 <- which(df[4] == 'Period')` where the column header explicitly called `Period`. Could you please show me how to create this code without using the column name as a hardcoded variable? Apologize if I am wrong. – Kevin Tracey Dec 02 '21 at 03:16
  • Updated comment : It works, but the code `inds4 <- which(df[4] == 'Period') the column header is expressly called 'Period'. Could you kindly show me how to do this code without having to use the column name as a hardcoded variable? Please accept my apologies if I am mistaken. Also. My column headers (additional columns at the bottom) in rows should be brought side by side as per the above-mentioned intended results. Is this also a possibility? – Kevin Tracey Dec 02 '21 at 03:27
  • Could you help me on this? – Kevin Tracey Dec 13 '21 at 13:28
1

Here's an approach that displays separate DTs, one for each sub-table in the input csv. This works with the example csv, although it may need some fiddling to work with the full csv.

(I've removed the other functions in order to focus on rendering the tables.)

Here's the UI. The mainPanel now contains a single uiOutput, which will be populated with as many DTs as we eventually need. (Inspired by this answer.)

ui <- fluidPage(
  sidebarLayout(
    sidebarPanel(
      fileInput("file1", "Choose CSV File", accept = ".csv"),
      checkboxInput("header", "Header", TRUE),
      uiOutput("selectUI"),
    ),
    mainPanel(
      uiOutput("tables")
    )
  )
)

And here's the server. It walks through the input csv; every time it encounters a line that appears to contain headers, it starts a new dataframe. At the end, we have a list of all the sub-dataframes contained in the csv, and we display them all.

server <- function(session, input, output) {
  
  rv <- reactiveValues(data = NULL, orig=NULL)
  
  observeEvent(input$file1, {
    
    # Validate the input file.
    file = input$file1
    ext = tools::file_ext(file$datapath)
    req(file)
    validate(need(ext == "csv", "Please upload a csv file"))
    
    # Read in the raw csv.
    raw.df = read.csv(file$datapath, header = input$header)
    rv$orig = raw.df
    
    # Initialize a list that will hold all the dataframes.
    dfs = list()
    
    # A vector of all the column names we've observed so far.
    all.colnames = colnames(raw.df)
    
    # Iterate over rows in the raw csv.  If we find a row where at least one
    # value matches one of the column names we've observed, assume that this row
    # is actually a header.  In that case, add all previous rows (since the last
    # header we saw) to a new dataframe.  The re-read the csv starting from the
    # line with the new header.
    current.row = 1
    total.headers = 1
    while(current.row <= nrow(raw.df)) {
      possible.colnames = unname(unlist(raw.df[current.row,]))
      if(length(intersect(all.colnames, possible.colnames)) > 0) {
        all.colnames = union(all.colnames, possible.colnames)
        dfs[[length(dfs) + 1]] = raw.df[1:(current.row-1),]
        raw.df = read.csv(file$datapath, skip = current.row + total.headers - 1,
                          header = input$header)
        current.row = 0
        total.headers = total.headers + 1
      }
      current.row = current.row + 1
    }
    dfs[[length(dfs) + 1]] = raw.df
    
    # Add the split dataframes to the reactive values.
    rv$data = dfs
    
    # Display however many tables we found.
    output$tables = renderUI({
      table.list = lapply(
        1:length(dfs),
        function(i) {
          table.name = paste("table", i, sep = "")
          column(width = 6, renderDT(dfs[[i]]))
        }
      )
      tagList(table.list)
    })
    
  })
  
}

enter image description here

A. S. K.
  • 2,504
  • 13
  • 22
  • 1
    Thank you, but how I should bring the `Expected Output` as I have stated in my first post above?, It would really help me if the solution meets my expected output – Kevin Tracey Dec 27 '21 at 15:39
  • Do the tables need to be displayed next to each other (horizontally), or do they really need to be a single table? And if so, can you help me understand why? Since the rows that happen to line up with each other aren't actually related, R isn't really designed to do this. – A. S. K. Dec 27 '21 at 16:32
  • Question: Do the tables need to be displayed next to each other (horizontally) Answer: Yes. All of the columns (if located in the bottoms) should be brought up horizontally. – Kevin Tracey Dec 27 '21 at 16:58
  • Here's the crucial difference: does the _display_ have to be horizontal, or do all the aligned rows have to be part of the _same table_? (Imagine taking the picture in this answer, cutting it in half, and showing the two tables horizontally instead of vertically. Would that work?) – A. S. K. Dec 27 '21 at 18:29
  • The table should be positioned horizontally, with the top and bottom columns aligned vertically if they contain duplicate headers (example: ID, Type, Range), and the rest of the columns (column: Period) aligned horizontally. – Kevin Tracey Dec 28 '21 at 03:11
  • I'm not asking about the positions of things in the display; that's clear. I'm asking about the underlying data structure. I've edited the answer to display two separate tables side by side. – A. S. K. Dec 28 '21 at 06:53
  • What is the best way to combine the same findings into a single dataframe? Also, when I have many bottom headers, how do I get the same results? – Kevin Tracey Dec 28 '21 at 11:49
  • Help me understand _why_ it needs to be single dataframe. If the aligned rows are unrelated, we're not creating [tidy data](https://cran.r-project.org/web/packages/tidyr/vignettes/tidy-data.html); the resulting dataframe will be hard to work with even after it's created. Also, it would help if you could edit your answer to include a bigger example of the csv. Just paste the raw csv text at the bottom of your answer (_including commas_). – A. S. K. Dec 28 '21 at 16:28