Error in as(x, class(k)) : no method or default for coercing “NULL” to “data.frame”

Question

I am currently facing an error mentioned below which is related to NULL values being coerced to a data frame. The data set does contain nulls, however I have tried both is.na() and is.null() functions to replace the null values with something else. The data is stored on hdfs and is stored in a pig.hive format. I have also attached the code below. The code works fine if I remove v[,25] from the key.

Code:

AM = c("AN");
UK = c("PP");
sample.map <- function(k,v){
 key <- data.frame(acc = v[!which(is.na(v[,1],1], 
                   year = substr(v[!which(is.na(v[,1]),2],1,4), 
                   month = substr(v[!which(is.na(v[,1]),2],5,6))
 value <- data.frame(v[,3],count=1)
 keyval(key,value)
}

sample.reduce <- function(key,v){
  AT <- sum(v[which(v[,1] %in% AM=="TRUE"),2])
  UnknownT <- sum(v[which(v[,1] %in% UK=="TRUE"),2])
  Total <- AT + UnknownT
  d <- data.frame(AT,UnknownT,Total)
  keyval(key,d)
 }
 out <- mapreduce(input ="/user/hduser/input",
             output = "/user/hduser/output",
             input.format = make.input.format("pig.hive", sep = "\u0001")                            
             output.format = make.output.format("csv", sep = ","),
             map= sample.map)
             reduce = sample.reduce)

Error:

Warning in asMethod(object) : NAs introduced by coercion 
Warning in split.default(1:rmr.length(y), unique(ind), drop = TRUE) :   data length is not a multiple of split variable 
Warning in rmr.split(x, x, FALSE, keep.rownames = FALSE) :   number of items to replace is not a multiple of replacement length Warning in        split.default(1:rmr.length(y), unique(ind), drop = TRUE) :   
data length is not a multiple of split variable 
Warning in rmr.split(v, ind, lossy = lossy, keep.rownames = TRUE) :   number of items to replace is not a multiple of replacement length 
Error in as(x, class(k)) :    
no method or default for coercing “NULL” to “data.frame” 
Calls: <Anonymous> ... apply.reduce -> c.keyval -> reduce.keyval -> lapply -> FUN -> as No traceback available

UPDATE I have added the sample data and edited the code above. Hope this helps!

Sample Data:

NULL,"2014-03-14","PP"
345689202,"2014-03-14","AN"
234539390,"2014-03-14","PP"
123125444,"2014-03-14","AN"
NULL,"2014-03-14","AN"
901828393,"2014-03-14","AN"

Hi Roman, Does this help? Also I wanted to mention that the data is stored on hdfs and this snapshot is anonymized. But it looks something like this. — Satej Wagle, Dec 09 '15 at 21:42

score 1 · Accepted Answer · edited May 23 '17 at 12:31

1

There are some issues with as which have been identified recently. I don't see why as can't handle this by default, but you can modify coerce which handles the conversion with an S4 method to call as.data.frame.

setMethod("coerce",c("NULL","data.frame"), function(from, to, strict=TRUE) as.data.frame(from))
[1] "coerce"
as(NULL,"data.frame")
data frame with 0 columns and 0 rows

edited May 23 '17 at 12:31

Community

1
1

answered Dec 09 '15 at 20:24

James

65,548
14
155
193

Where should I run this code? As of now my hadoop environment contains 3 worker nodes which have R and Rmr2 packages installed. Should I run this on all those nodes? Also should I run this method everytime I run the script? Sorry for asking too many questions. – Satej Wagle Dec 09 '15 at 21:32
Yes, it needs to be run by each worker that will require to use the method. It is probably best to put it into a .profile file to run at startup. – James Dec 09 '15 at 23:36
This worked! I added this to the .profile file and restarted my R session. Thank you James for the prompt response :) – Satej Wagle Dec 10 '15 at 01:28

Error in as(x, class(k)) : no method or default for coercing “NULL” to “data.frame”

1 Answers1

Linked