Dealing with NA value in data importing loop in R

Question

I'm importing several hundred files into a single file in order to analyze is after using:

files.pet <- sort(list.files(pattern = '1998[0-9][0-9][0-9][0-9].pet'), decreasing = FALSE)
    all_data.pet <- NA;
    for (pet.atual in files.pet) {
      data.atual <-
        read.table(file = pet.atual,
                   header = FALSE,
                   sep = ",",
                   quote = "\"",            
                   comment.char = ";");
      data.atual <- cbind(data.atual, Desig = pet.atual)
      all_data.pet <- rbind(all_data.pet, data.atual)
    }

Which runs good until it finds one file giving this error:

Error in scan(file = file, what = what, sep = sep, quote = quote, dec = dec,  : 
  line 399 did not have 9 elements

Which has a NA value in one of the columns, is there a way to tell the loop to ignore this and keep importing? Or should i just erase/replace NA in the row?

Also while I'm asking can anyone give me a insight on the meaning of:

In scan(file = file, what = what, sep = sep, quote = quote, dec = dec,  :
  embedded nul(s) found in input

I read around but had little luck understanding what it actually means.

Thanks alot! (sorry if the questions are pretty obvious but I'm new to R)

import data upto 398 and then in the second run, start from 400. You could use `skip` and `nrows` arguments in `read.table` — Sathish, Mar 17 '17 at 02:44
I tried using it before, but it didn't work. Probably put it in the wrong place in the code but i'll see what i can find about it! — stasisOo, Mar 17 '17 at 02:49
The fill = NA worked, gonna have a look on what it means! Any ideia about the other warning? — stasisOo, Mar 17 '17 at 02:52
also change `fileEncoding` to a different encoding format and see if it passes through. I would first try `skipNul = TRUE` — Sathish, Mar 17 '17 at 02:56
This answer may be helpful http://stackoverflow.com/questions/24734911/warning-message-line-appears-to-contain-embedded-nulls — Sathish, Mar 17 '17 at 02:57
the skipNul does not solve the problem. As the for enconding tried both fileEncoding = "UCS-2LE" and "UTF-16LE" and the warnings just went up xD — stasisOo, Mar 17 '17 at 03:12
Use `fread` from `data.table` package and select only columns of interest. — discipulus, Mar 17 '17 at 03:28
@Sathish it's wierd, when i try and type of fileEncoding it only processes 183 observations (out of 2M) — stasisOo, Mar 17 '17 at 03:55
@discipulus problem is that all columns are of interest, but thanks for the reply — stasisOo, Mar 17 '17 at 04:00
see if you could import all data by reading them in chunks and also changing the fileconding. Later combine all together — Sathish, Mar 17 '17 at 04:18

Dealing with NA value in data importing loop in R

0 Answers0