For-loop in R to create a new file (but gives incorrect/unexpected output)

Question

I'm currently busy with some data and I need to check their validity. Therefore, I would like to use a for-loop to go through all my data files. In this for-loop, I would like to calculate some things (like mean, min,max...).

My code below works but produced an incorrectly written csv file. The problem occurs after the calculations (and their values) are done during csv file creation. CSV:

"c.1..1..1004.89081855716..630.174466667434..461.738905906677.." "c.1..1..950.990843858612..479.98560814955..517.955102920532.."
1 1
1 1
1004.89081855716 950.990843858612
630.174466667434 479.98560814955
461.738905906677 517.955102920532
1535.86795806885 1452.30199813843
-13.3948961645365 3.72026950120926
1259.26423788071 1159.17089223862

Approach/What I'm expecting: So I start from some data files with eye tracking data in it. As you can see at the beginning of the code, I try to get some values out of this eye tracking data (validity, new file with only validity == 1 data...). Once I created the filtered_data dataframe, I want to calculate some extra values out of it (mean, sd, min/max). My plan is to create a new csv file (validity_loop.csv) in which I can find all my calculations (validity_left, validity_right,mean_eye_x, mean_eye_y, min_eye_x,max_eye_x,min_eye_y,max_eye_y). All in a row. One row for each data set (file_list[i]).

Can someone help me in how to tackle and solve this issue?

Here is my code:

set <- setwd("/Users/Sarah/Documents")

file_list <- list.files(set, pattern = ".csv", all.files = TRUE)
validity_list <- data_list <- vector("list", "length" = length(file_list))

for(i in seq_along(file_list)){

  filename = file_list[i]

  #read files

  data_frame = read.csv(filename, sep = ",", dec = ".", 
                        header = TRUE, 
                        stringsAsFactors = FALSE)

  #what has to be done
  #validity

  validity_left <- mean(is.numeric(data_frame$left_gaze_point_validity))
  validity_right <-mean(is.numeric(data_frame$right_gaze_point_validity))

  #Zuiver dataframe (validity ==1)

  to_keep = which(data_frame$left_gaze_point_validity == 1 &
                  data_frame$right_gaze_point_validity==1)

  filtered_data = data_frame[to_keep,]
  filtered_data$left_eye_x = as.numeric(filtered_data$left_eye_x)
  filtered_data$left_eye_y = as.numeric(filtered_data$left_eye_y)
  filtered_data$right_eye_x = as.numeric(filtered_data$right_eye_x)
  filtered_data$right_eye_y = as.numeric(filtered_data$right_eye_y)

  #1 eye-data 

  filtered_data$eye_x <- (filtered_data$left_eye_x+filtered_data$right_eye_x)/2
  filtered_data$eye_y <- (filtered_data$left_eye_y+filtered_data$right_eye_y)/2

  #Pixels 

  filtered_data$eye_x <- (filtered_data$eye_x)*1920
  filtered_data$eye_y <- (filtered_data$eye_y)*1080

  #SD and Mean + min-max

  mean_eye_x<- mean(filtered_data$eye_x)
  mean_eye_y <- mean(filtered_data$eye_y)

  sd_eye_x <- sd(filtered_data$eye_x)
  sd_eye_y <- sd(filtered_data$eye_y)

  min_eye_x <- min(filtered_data$eye_x)
  min_eye_y <- min(filtered_data$eye_y)
  max_eye_x <- max(filtered_data$eye_x)
  max_eye_y <- max(filtered_data$eye_y)

  #add everything to new file

  validity_list[[i]] <- c(validity_left, validity_right, 
                          mean_eye_x, mean_eye_y,
                          min_eye_x, min_eye_y,
                          max_eye_x, max_eye_y) 

}

#new document
write.table(validity_list, 
            file = "Master T&O/Thesis /Loop/Validity/validity_loop.csv",
            col.names = TRUE, row.names = FALSE)

Hi Sarah.. please attach your csv file because that shows what goes wrong and helps to solve your problem more easily. And please complete the tour... to get the benefits and hints on posting q's ;-) — ZF007, Dec 04 '17 at 12:38
`validity_list` is a list and `write.table` prefers matrix or data.frame — pogibas, Dec 04 '17 at 12:39
How is the file incorrect? Could you edit your post with what you were expecting and what you currently get? — John, Dec 04 '17 at 12:39
And I'm currently writing down what a expect and what is not working! — Sarah.d, Dec 04 '17 at 12:42
As R is not my native language I take the approach of printing stepwise results to standardout when using an editor like komodo edit for example. That way I see my input and thereafter result printed on screen. I assume you got your own editor in place? This way you iter through code and find the typo or other type of mistake. Reading up PoGibas answer might be right on this one? Otherwise: [http://rprogramming.net/write-csv-in-r/]. — ZF007, Dec 04 '17 at 12:50
I also thiought about the answer of PoGibas, but honestly I'm new at using R, so I'm not really sure how I can make a matriw out of my validity_list? — Sarah.d, Dec 04 '17 at 12:54
Did you see this [https://stackoverflow.com/questions/13224553/how-to-convert-a-list-to-a-matrix-more-efficiently-in-r] example? That may help :-) — ZF007, Dec 04 '17 at 13:05
ZF007, thank you for the example! I'll look at it and see if it can help me out ;-) — Sarah.d, Dec 04 '17 at 13:08
once you got it fixed post your solution as a new answer to the question. This helps others to solve faster similar problems. Enjoy ;-) — ZF007, Dec 04 '17 at 13:10
ZF007, No problem, I will do it! I already see some change in my csv, the only problem is that it make one column instead of one column for each value, So i'm still looking for it ;-) — Sarah.d, Dec 04 '17 at 13:13

score 1 · Answer 1 · answered Dec 04 '17 at 13:36

I managed to get a new data frame in R, which contains the value of my validity_list as a matrix form.

#FOR LOOP poging 2 
set <- setwd("/Users/Sarah/Documents/Master T&O/Thesis /Loop")

file_list <- list.files(set, pattern = ".csv", all.files = TRUE)
validity_list <- vector("list", "length" = length(file_list))

for(i in seq_along(file_list)){
  filename = file_list[i]
  #read files
  data_frame = read.csv(filename, sep = ",", dec = ".", header = TRUE, stringsAsFactors = FALSE)
  #what has to be done
  #validity
  validity_left <- mean(is.numeric(data_frame$left_gaze_point_validity))
  validity_right <-mean(is.numeric(data_frame$right_gaze_point_validity))
  #Zuiver dataframe (validity ==1)
  to_keep = which(data_frame$left_gaze_point_validity == 1 & data_frame$right_gaze_point_validity==1)

  filtered_data = data_frame[to_keep,]
  filtered_data$left_eye_x = as.numeric(filtered_data$left_eye_x)
  filtered_data$left_eye_y = as.numeric(filtered_data$left_eye_y)
  filtered_data$right_eye_x = as.numeric(filtered_data$right_eye_x)
  filtered_data$right_eye_y = as.numeric(filtered_data$right_eye_y)
  #1 eye-data 
  filtered_data$eye_x <- (filtered_data$left_eye_x+filtered_data$right_eye_x)/2
  filtered_data$eye_y <- (filtered_data$left_eye_y+filtered_data$right_eye_y)/2
  #Pixels 
  filtered_data$eye_x <- (filtered_data$eye_x)*1920
  filtered_data$eye_y <- (filtered_data$eye_y)*1080
  #SD and Mean + min-max
  mean_eye_x<- mean(filtered_data$eye_x)
  mean_eye_y <- mean(filtered_data$eye_y)

  sd_eye_x <- sd(filtered_data$eye_x)
  sd_eye_y <- sd(filtered_data$eye_y)

  min_eye_x <- min(filtered_data$eye_x)
  min_eye_y <- min(filtered_data$eye_y)
  max_eye_x <- max(filtered_data$eye_x)
  max_eye_y <- max(filtered_data$eye_y)

  #add everything to new file
  validity_list[[i]] <- c(validity_left, validity_right,mean_eye_x, mean_eye_y, min_eye_x,max_eye_x,min_eye_y,max_eye_y)
  validity_matrix <- matrix(unlist(validity_list), ncol = 8, byrow = TRUE)
}

#new document
write.table(validity_matrix, file = "/Users/Sarah/Documents/Master T&O/Thesis /Loop/Validity/validity_loop.csv", dec = ".")

The only problem I have now, is the fact that my values for the validity_list items are wrong, but that's another problem and I'm trying to fix it!

score 0 · Answer 2 · edited Dec 16 '17 at 10:16

0

If I get it then the following line grabs all your data together:

validity_list[[i]] <- c (validity_left, validity_right,mean_eye_x,
                         mean_eye_y, min_eye_x,max_eye_x,min_eye_y,max_eye_y).

if it's like in python then I would have:

validity_list = (validity_left, validity_right,mean_eye_x,
                      mean_eye_y, min_eye_x,max_eye_x,min_eye_y,max_eye_y)

... whereas the '=' tell the interpreter that everything behind it is a tuple '(', data, ')' ...which makes it one single dataset and if I then write it... it would be end up in one column. If you do a pick using a for-loop I would get "validity_left" writing in a separate column. In your case adding this to your below code an option?

for item in validity_list:
         function to process item..etc.

edited Dec 16 '17 at 10:16

marc_s

732,580
175
1,330
1,459

answered Dec 04 '17 at 13:26

ZF007

3,708
8
29
48

Thank you for the help! With the previous comments, I managed to get a data frame in R (validity_matrix) which looks like what I want. My only problem now, is that I got wrong values in my matrix, but that's a problem in the calculation code of the validity_list elements. I'll put my code as answer below! – Sarah.d Dec 04 '17 at 13:33
1

Well done and Grats! Don't forget to post your answer separately :-) – ZF007 Dec 04 '17 at 13:35

For-loop in R to create a new file (but gives incorrect/unexpected output)

2 Answers2