1

I have data for 40 buildings. Each building contains folders like CO2, Temperature, heat. then inside each folder there is data for different regions of house. For instance, for CO2, I have data (time series collection) for hallway, bedroom, living room etc. I need heatmaps for all these data file. So far, what i could do is, I wrote a code to go inside each directory, then each folder and then access files and draw heatmap inside Rstudi. Code is:

setwd("C:/Users/...")
folders <- list.dirs(full.names = TRUE)

result <- sapply(folders[-1], function(x){
res<-lapply(files, function (x) {
[some heatmap function]
P1<- ggplot(df, aes(Time, Date, fill =reading)) + geom_tile(colour = "grey") + scale_fill_gradientn(colours=c("darkblue", "red", "yellow"),   values=rescale(c(0, 1000, 2000)),  guide="colorbar")+scale_x_discrete(breaks = lab1)
plot(P1)
  })})

what i actually want is, to store the heatmap results of each file in the same folder with the name of the folder+name of the file (like CO2.hallway.jpeg etc). I am trying various codes since last 2 days but none is working for me. can anyone please help me with this. I'll very grateful to you.

USAL
  • 53
  • 1
  • 10

2 Answers2

2

What you want to do is change the working directory or the path to the file inside the lapply-loop.

Something like this

setwd("...")
folders <- list.dirs(full.names = T)

res <- sapply(folders[-1], function(dir){

  # INSERT OTHER CODE PARTS HERE

  P1 <- ggplot(...) + geom_x()

  # Option 1
  setwd(dir)
  ggsave(P1, filename = paste0("CO2.", dir, ".jpg")) 

  # Or Option2
  ggsave(P1, filename = paste(dir, paste0("CO2.", dir, ".jpg"), sep = "/"))
  # I encourage you to use pdf (best quality, can be included in LaTeX and Markdown), otherwise png (better quality)
})

Added: Minimum-Working-Example

A MWE to save plots in all sub-directions looks like this:

folders <- list.dirs(full.names = T)

lapply(folders[-1], function(dir){
  dat <- data.frame(x = 1:10, y = cumsum(rnorm(10)))

  P1 <- ggplot(dat, aes(x = x, y  = y)) + geom_line()
  ggsave(P1, filename = paste(dir, paste0("plot.png"), sep = "/"))
})

Added v2: MWE incl reading the data

lapply(folders[-1], function(dir2){

  # read the data
  files <- list.files(dir2, pattern = "*.csv", recursive = F)

  # finds the last "./" and takes everything afterwards
  # aka, it returns the up-most folder
  folder <- substr(dir2, 
                   start = regexpr("\\./[^\\./]*$", dir2) + 2, 
                   stop = nchar(dir2)) 

  lapply(files, function(file, folder){
    # find the filename, aka. exclude .csv
    f.name <- substr(file, start = 1,
                     stop = regexpr(".csv", file) - 1)

    # load each file to the loadeddat-data.frame
    loadeddat <- read.table(paste(folder, file, sep = "/"))

    # plot the data as you wish
    P1 <- ggplot(loadeddat, aes(x = x, y  = y)) + geom_line()

    # create the name for the plot
    nam <- paste(folder, # i.e., folder1
                 paste0(folder, "-", f.name, ".png"), # i.e., folder1-file1.png
                 sep = "/") # whole name/path looks like this: 
                 # "folder1/folder1-file1.png"

    # save it
    ggsave(P1, filename = nam)
  }, folder = folder)
})

Does that help?

David
  • 9,216
  • 4
  • 45
  • 78
  • Thank you David. your code gives me this error: Error in file(file, "rt") : cannot open the connection, Also, CO2 was just an example, i want the name of each separate file to be pasted there with the name of directory. – USAL Nov 10 '15 at 13:15
  • Its probably due to some points and slashes in the dir name... Try the added minimum working example and expand your code from it (I`ve added it to my answer) – David Nov 10 '15 at 13:20
  • with your code I can save png plots in each folder in the working directory with the name plot.png. But when i use my function with this code, it does not work. I think the problem is with reading the file. This is how i read file `folders <- list.dirs(full.names = TRUE)` result <- sapply(folders[-1], function(dir){ `files <- list.files(dir, pattern="*.csv", full.names=TRUE, recursive=FALSE)` `res<-lapply(files, function (x) { t <- read.csv(x, header=T) data<-t` `P1<- ggplot(setwd(dir) ggsave(P1, filename = paste(dir, paste0("plot.jpeg"), sep = "/")))` Any clue? – USAL Nov 10 '15 at 13:49
  • No, sorry, it was an editing mistake. My apologies. The real problem i face is reading the data files. I think I am doing some mistake there. Do you have any clue about that? – USAL Nov 10 '15 at 14:14
  • 1
    I added a second MWE that includes loading the data.... Does that help you? If so, please considering upvoting and marking it as an answer! Thank you! – David Nov 10 '15 at 14:31
  • This answer is very useful dear David. Thank you very much for taking time, communicating with me and then solving my problem. All this is very much appreciated. Thank you! – USAL Nov 10 '15 at 14:41
  • Pleasure to help someone using R! :) If the comments have solved your question, please also mark it as an answer (below the voting arrows), also see: http://stackoverflow.com/tour – David Nov 10 '15 at 14:54
  • David, Although, your code has solved major problem, but a minor thing is, when it process all datafiles of one folder, it creates only one plot with the name plot.png. can we create plot for all files of one folder. more interestingly, is it possible if we give name to the plot as the name of the folder plus name of the file together, like foldername.filename.png? – USAL Nov 10 '15 at 16:18
  • Sure, thats of course possible, I've edited the code. Does that give you the right answer? – David Nov 10 '15 at 16:35
  • Hi David, I tried your code this morning, but it gave me this error: `Error in file(file, "rt") : cannot open the connection`. Any idea about this? Thanks. – USAL Nov 11 '15 at 10:21
  • Apparently, it seems like problem could be here `"\\./[^\\./]*$"` , when we match the folder path pattern. Or, it could be here `regexpr(".csv"`. – USAL Nov 11 '15 at 11:33
  • It works for me, the commands should be fine. Have a look at this question: http://stackoverflow.com/questions/27466317/error-in-filefile-rt-cannot-open-the-connection – David Nov 11 '15 at 14:16
  • Thank you, I actually checked all these discussions before pasting problem here. I think if you can elaborate your code (the last one for naming plots) that how does this work then after that i might be able to track my error. Can you please do that. just 1 or 2 line of how this code work. I'll be grateful to you! – USAL Nov 11 '15 at 14:55
  • Make sure that you have the rights to read and write the files, I'll add descriptions. – David Nov 11 '15 at 16:16
  • Yes, I can read and write the files. Error inspector shows that the error is `folder <- substr(dir, start = regexpr("\\./[^\\./]*$", dir) + 2, stop = nchar(dir))`. I don't know how this reg expression is working here. I am trying to read the documents to understand this. – USAL Nov 11 '15 at 16:21
  • `regexpr("\\./[^\\./]*$", dir)` gives you the stringposition of the last occurence of "./". Can you print something like `folders[2]` to see what it looks like, or the `dir` when triggering an error? – David Nov 11 '15 at 16:25
  • `folders[2]` returns this `[1] "./b1_26_windermere_road"` and if i print all folders then this: `[1] "." "./b1_26_windermere_road" "./b1_26_windermere_road/CO2" [4] "./b1_26_windermere_road/Electricity" "./b1_26_windermere_road/Humidity" "./b2_28_windermere_road" [7] "./b2_28_windermere_road/CO2" "./b2_28_windermere_road/Electricity"` currently i am processing test data for two buildings `b1....` and `b2....` – USAL Nov 11 '15 at 16:35
  • for `dir` It gives `function (path = ".", pattern = NULL, all.files = FALSE, full.names = FALSE, recursive = FALSE, ignore.case = FALSE, include.dirs = FALSE, no.. = FALSE) .Internal(list.files(path, pattern, all.files, full.names, recursive, ignore.case, include.dirs, no..)) ` seems like dir has nothing to return. – USAL Nov 11 '15 at 16:37
  • Seems you have some scoping issues. Clearly `dir` is not meant to be a function in the base-environment, but a string containing the direction. Try replacing all dir with dir2 or something else (as in the updated version). – David Nov 11 '15 at 16:41
  • It is still giving me the problem. when I looked at traceback function, it points error in `read.csv(paste(folder, file, header=TRUE, sep="/")`. Further more in `Values` it says, row.names and col.names and text as missing arguments. I wrote this code as: `read.csv(paste(folder, file, header=TRUE, sep="/", row.names=NULL, col.names=NULL))` but failed. I feel like you must have got fed-up now :( i feel embarrassed asking you again and again. If we couldn't solve now, I'll try to find any other solution, But you have have helped me a lot with this. – USAL Nov 11 '15 at 17:44
  • No problems, why don't you shoot me an E-Mail, so we can discuss things somewhere else and don't clogg SO. You find my email here https://datashenanigan.wordpress.com/ – David Nov 11 '15 at 19:42
0

Loop through the names and use the dir.create() function to create the folders you need and then access them. Super simple: http://rfunction.com/archives/2432

seeiespi
  • 3,628
  • 2
  • 35
  • 37