How to to match values with file names?

Question

I have one file in dir1 named like this:File_01_02_2013.img.
I have other files in another folder dir2 named like this:

File_01_02_2013_00.img 
File_01_02_2013_01.img

So I have one file a matrix in dir1 but 24 files in dir2.
What I need is:

 00 …>>>> File_01_02_2013_00.img >>>> 4 >>>> put in the new matrix `out_01_02_2013_.img`

To read the file in dir1:

con <- file("C:\\dir1\\ File_01_02_2013.img", "rb")
pva<- readBin(con, numeric(), size=4,  n=1000*500)
dat <- matrix((data=pva), ncol=500, nrow=1000)

to read the files in dir2:

These 24 files have the same dim as File_01_02_2013.img and can be read by the same lines above.

There is no need to match file names. `dir2` is an ordered character vector containing the filenames from `...00.img` to `...23.img`. So if you read `03` you could convert it to numeric and read the fourth file: `readBin(dir2[[as.numeric("03")+1]], ...)`. — sgibb, Sep 11 '15 at 10:24

MSJ · Accepted Answer · 2015-09-11T22:15:39.913

sgibb is right, if the file names in dir2 contain all values from 00 to 23, then to access say file ...04.img you would want to open the fifth file in your dir2 list.

So assume you are getting the first value in your dat matrix

val <- dat[1,1]

and val is equal to 4, then you can access the fifth file using readBin(dir2[[val + 1]]... (if the values in dat are numeric. If they are not, you have to convert it to numeric as in sgibb's comment).

Now when you mean "extract the corresponding value", do you mean use the index that you used to get val above? So do you want load the dir2 file, and then get value [1,1] from it? And then take that value and put it in [1,1] in your final matrix?

Edit

After clarification, here's one (not very elegant) thing you could do:

nrows <- 1000
ncols <- 500
outmat <- matrix(,nrows,ncols)
for (nr in 1:nrows){
 for(nc in 1:ncols{
   val <- dat[nr,nc]
   dir2file <- readBin(dir2[[val + 1]], numeric(), size=4, n=1000*500)
   dir2val <- dir2file[nr,nc]
   outmat[nr,nc] <- dir2val
 }
}

Edit #2

Here's trying to loop through your entire dataset. Since you didn't supply any sample data I wasn't able to test this code, so I cannot guarantee that it works without you having to debug it. But maybe you're lucky ;-)

dir1 <- list.files("C:\\dir1", "*.img", full.names = TRUE)
dir2 <- list.files("C:\\dir2", "*.img", full.names = TRUE)

# get a list of strings of just the file names
dir1str <- list.files("C:\\dir1", "*.img", full.names = FALSE)


nrows <- 1000
ncols <- 500

for (fileInd in length(dir1)){
  # read in file 1 of dir1 (copied from your code)
  pva<- readBin(dir1[[fileInd]], numeric(), size=4,  n=1000*500)
  dat <- matrix((data=pva), ncol=500, nrow=1000)

  # retrieve the file name of dir1
  dir1strfile <- dir1str[[fileInd]] 
  # split the string at the underscores (_)
  file.attr <- unlist(strsplit(dir1strfile,"_"))
  # Paste the strings back together to get File_date
  fdate <- paste0('File_',file.attr[2],'_',file.attr[3])

  # Get indeces of files in dir2 that match the date 
  dir2date <- grep(fdate,dir2)
  dir2lst <- dir2[dir2date]

  # pre-allocate output matrix
  outmat <- matrix(NA,nrows,ncols)

  # now loop through the dir1 file, and save the outmat as a csv
  for (nr in 1:nrows){
    for(nc in 1:ncols){
      val <- dat[nr,nc]
      dir2file <- readBin(dir2lst[[val + 1]], numeric(), size=4, n=1000*500)
      outmat[nr,nc] <-  dir2file[nr,nc]
    }
  }
  # you probably have to turn the '/' into a '\' below
  outname <- paste0(dir2, '/' ,fdate,'_output.csv') 
  write.csv(outmat,outname)
  }

I'd also appreciate if someone posted a more elegant solution!

Sure. Are the `dir2` files also formatted in the same way, so: `File_02_02_2013_00.img -> File_02_02_2013_23.img`? — MSJ, Sep 11 '15 at 21:15
We can try to cut down on the for loops and use the apply functions, maybe that will help. I think part of the issue is that we are indexing a lot, as well as loading a lot of files. Maybe we should ask that as a separate question. I'm sorry, I'm not super experienced with R. I am starting to look at posts like this one: http://stackoverflow.com/questions/2908822/speed-up-the-loop-operation-in-r. Maybe I'll find something applicable. — MSJ, Sep 14 '15 at 19:35

How to to match values with file names?

1 Answers1