Error in table(data, reference, dnn = dnn, ...) : all arguments must have the same length when run confusionMatrix with caret, in R

Question

I have an issue running a confusionMatrix.

here is what I do:

rf <- caret::train(tested ~., 
                               data = training_data, 
                               method = "rf",
                               trControl = ctrlInside,
                               metric = "ROC", 
                               na.action = na.exclude)

rf

After I get my model this is the next step I take:

evalResult.rf <- predict(rf, testing_data, type = "prob")
predict_rf <- as.factor(ifelse(evalResult.rf <0.5, "positive", "negative"))

And then I am running my confusion matrix.

cm_rf_forest <- confusionMatrix(predict_rf, testing_data$tested, "positive")

And the error comes after I apply the confusionMatrix:

Error in table(data, reference, dnn = dnn, ...) : 
  all arguments must have the same length

Nevertheless, I give you bits of my data.

train data:

structure(list(tested = structure(c(1L, 1L, 1L, 1L, 1L, 
1L), .Label = c("negative", "positive"), class = "factor"), Gender = structure(c(2L, 
2L, 1L, 1L, 2L, 2L), .Label = c("Female", "Male", "Other"), class = "factor"), 
    Age = c(63, 23, 28, 40, 31, 60), number_days_symptoms = c(1, 
    1, 16, 1, 14, 1), care_home_worker = structure(c(1L, 2L, 
    1L, 1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    health_care_worker = structure(c(1L, 1L, 1L, 1L, 2L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), how_unwell = c(1, 1, 6, 4, 2, 
    1), self_diagnosis = structure(c(1L, 1L, 2L, 1L, 2L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), chills = structure(c(1L, 1L, 2L, 
    1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    cough = structure(c(1L, 1L, 2L, 2L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), diarrhoea = structure(c(1L, 1L, 
    1L, 1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    fatigue = structure(c(1L, 2L, 2L, 2L, 2L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), headache = structure(c(2L, 2L, 
    3L, 2L, 2L, 2L), .Label = c("Headcahe", "No", "Yes"), class = "factor"), 
    loss_smell_taste = structure(c(1L, 1L, 1L, 1L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), muscle_ache = structure(c(1L, 
    1L, 2L, 2L, 2L, 2L), .Label = c("No", "Yes"), class = "factor"), 
    nasal_congestion = structure(c(1L, 1L, 1L, 2L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), nausea_vomiting = structure(c(1L, 
    1L, 1L, 1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    shortness_breath = structure(c(1L, 1L, 1L, 1L, 2L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), sore_throat = structure(c(1L, 
    1L, 1L, 2L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    sputum = structure(c(1L, 1L, 2L, 2L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), temperature = structure(c(4L, 
    4L, 4L, 4L, 1L, 4L), .Label = c("37.5-38", "38.1-39", "39.1-41", 
    "No"), class = "factor"), asthma = structure(c(2L, 1L, 1L, 
    1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    diabetes_type_one = structure(c(1L, 1L, 1L, 1L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), diabetes_type_two = structure(c(2L, 
    1L, 1L, 1L, 1L, 2L), .Label = c("No", "Yes"), class = "factor"), 
    obesity = structure(c(1L, 2L, 2L, 1L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), hypertension = structure(c(1L, 
    1L, 2L, 1L, 1L, 2L), .Label = c("No", "Yes"), class = "factor"), 
    heart_disease = structure(c(1L, 1L, 1L, 1L, 1L, 2L), .Label = c("No", 
    "Yes"), class = "factor"), lung_condition = structure(c(1L, 
    1L, 1L, 1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    liver_disease = structure(c(1L, 1L, 1L, 1L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), kidney_disease = structure(c(1L, 
    1L, 1L, 1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor")), row.names = c(1L, 
3L, 4L, 5L, 6L, 7L), class = "data.frame")

and here is my test_data:

structure(list(tested = structure(c(1L, 1L, 1L, 1L, 1L, 
1L), .Label = c("negative", "positive"), class = "factor"), Gender = structure(c(1L, 
2L, 1L, 1L, 1L, 2L), .Label = c("Female", "Male", "Other"), class = "factor"), 
    Age = c(19, 26, 30, 45, 40, 43), number_days_symptoms = c(20, 
    1, 1, 20, 14, 1), care_home_worker = structure(c(1L, 1L, 
    1L, 1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    health_care_worker = structure(c(1L, 1L, 1L, 1L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), how_unwell = c(7, 6, 6, 6, 6, 
    2), self_diagnosis = structure(c(2L, 1L, 1L, 2L, 2L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), chills = structure(c(2L, 1L, 1L, 
    1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    cough = structure(c(2L, 1L, 1L, 2L, 2L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), diarrhoea = structure(c(2L, 1L, 
    1L, 1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    fatigue = structure(c(2L, 1L, 1L, 2L, 2L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), headache = structure(c(2L, 2L, 
    2L, 3L, 2L, 3L), .Label = c("Headcahe", "No", "Yes"), class = "factor"), 
    loss_smell_taste = structure(c(1L, 1L, 1L, 1L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), muscle_ache = structure(c(2L, 
    1L, 1L, 1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    nasal_congestion = structure(c(1L, 1L, 1L, 1L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), nausea_vomiting = structure(c(1L, 
    1L, 1L, 1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    shortness_breath = structure(c(2L, 1L, 1L, 1L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), sore_throat = structure(c(1L, 
    1L, 1L, 2L, 1L, 2L), .Label = c("No", "Yes"), class = "factor"), 
    sputum = structure(c(2L, 1L, 1L, 2L, 1L, 2L), .Label = c("No", 
    "Yes"), class = "factor"), temperature = structure(c(4L, 
    4L, 4L, 1L, 1L, 4L), .Label = c("37.5-38", "38.1-39", "39.1-41", 
    "No"), class = "factor"), asthma = structure(c(1L, 1L, 1L, 
    1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    diabetes_type_one = structure(c(1L, 1L, 1L, 1L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), diabetes_type_two = structure(c(1L, 
    1L, 1L, 1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    obesity = structure(c(1L, 1L, 1L, 1L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), hypertension = structure(c(1L, 
    1L, 1L, 1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    heart_disease = structure(c(1L, 1L, 1L, 1L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), lung_condition = structure(c(1L, 
    1L, 1L, 1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor"), 
    liver_disease = structure(c(1L, 1L, 1L, 1L, 1L, 1L), .Label = c("No", 
    "Yes"), class = "factor"), kidney_disease = structure(c(1L, 
    1L, 1L, 1L, 1L, 1L), .Label = c("No", "Yes"), class = "factor")), row.names = c(2L, 
8L, 11L, 14L, 20L, 27L), class = "data.frame")

Additionally, I perform a smote balancing class, on a subsample in ctrInside.

This is my smote function:

smotest <- list(name = "SMOTE with more neighbors!",
                func = function (x, y) {
                  115
                  library(DMwR)
                  dat <- if (is.data.frame(x)) x else as.data.frame(x)
                  dat$.y <- y
                  dat <- SMOTE(.y ~ ., data = dat, k = 3, perc.over = 100, perc.under =
                                 200)
                  list(x = dat[, !grepl(".y", colnames(dat), fixed = TRUE)],
                       y = dat$.y) },
                first = TRUE)

And ctrlInside is this:

ctrlInside <- trainControl(method = "repeatedcv", 
                           number = 10,
                           repeats = 5,
                           summaryFunction = twoClassSummary,
                           classProbs = TRUE,
                           savePredictions = TRUE, 
                           search = "grid",
                           sampling = smotest)

Those function are given just so that you have an idea of what I am doing per whole. Is there a reason why this is happening?

Hey @GaB, can you check what you have posted, firstly, it is ```evalResult.rf <- predict(rf_tuned, testing_data, type = "prob")``` then ```predict_rf <- as.factor(ifelse(rf_pred<0.5, "positive", "negative"))```, but ```rf_pred``` was not assigned — StupidWolf, May 29 '20 at 22:24
then you have ```confusionMatrix(predict_rf, testing_data$Covid_tested, "positive")``` but testing_data has no Covid_tested column — StupidWolf, May 29 '20 at 22:25
@StupidWolf - okay - done all the corrections in this post. I have nevertheless corrected in my scripts and still got the same result — GaB, May 29 '20 at 22:34
ok then you need to check when there is any NAs in your testing_data, for example, table(complete.cases(testing_data)) and see if any FALSE — StupidWolf, May 31 '20 at 08:33
sorry @GaB, if i try running what you have with made up data with the same columns it works. so there must be something in your data, and you needa check that — StupidWolf, May 31 '20 at 08:34
@StupidWolf:So, this is probably the reason. I am running the table(complete.cases and the output is: FALSE TRUE 1 12160 What shall I do? — GaB, Jun 01 '20 at 09:48
i looked through again, and posted a more detailed answer on how to solve your problem. Give it a shot and it should work out well — StupidWolf, Jun 01 '20 at 17:11

score 1 · Accepted Answer · answered Jun 01 '20 at 17:10

You can use complete.cases to predict only those that have no nas, also you must operate on the matrix, I will show below. Using an example dataset, I make 10 of the variable in a column NAs, and train:

idx = sample(nrow(iris),100)
data = iris
data$Petal.Length[sample(nrow(data),10)] = NA
data$tested = factor(ifelse(data$Species=="versicolor","positive","negative"))
data = data[,-5]
training_data = data[idx,]
testing_data= data[-idx,]

rf <- caret::train(tested ~., data = training_data, 
                              method = "rf",
                              trControl = ctrlInside,
                              metric = "ROC", 
                              na.action = na.exclude)

Do the evaluation result and you can see i get the same error:

evalResult.rf <- predict(rf, testing_data, type = "prob")
predict_rf <- as.factor(ifelse(evalResult.rf <0.5, "positive", "negative"))
cm_rf_forest <- confusionMatrix(predict_rf, testing_data$tested, "positive") 

Error in table(data, reference, dnn = dnn, ...) : 
  all arguments must have the same length

So there's two sources of error, 1.. you have NAs and they cannot predict that, and second, evalResult.rf returns a matrix of probabilities, first column is probability being negative class, 2nd being postive:

head(evalResult.rf)
   negative positive
3     1.000    0.000
6     1.000    0.000
9     0.948    0.052
12    1.000    0.000
13    0.976    0.024
19    0.998    0.002

To get the classes, you do, get the column with max value for each row, and return the corresponding column name, which is the class:

colnames(evalResult.rf)[max.col(evalResult.rf)]

We do now:

testing_data = testing_data[complete.cases(testing_data),]
evalResult.rf <- predict(rf, testing_data, type = "prob")
predict_rf <- factor(colnames(evalResult.rf)[max.col(evalResult.rf)])
cm_rf_forest <- confusionMatrix(predict_rf, testing_data$tested, "positive")

Confusion Matrix and Statistics

          Reference
Prediction negative positive
  negative       33        1
  positive        0       11

               Accuracy : 0.9778          
                 95% CI : (0.8823, 0.9994)
    No Information Rate : 0.7333          
    P-Value [Acc > NIR] : 1.507e-05       

                  Kappa : 0.9416

it worked! I did this: predict_rf <- factor(colnames(evalResult.rf)[max.col(evalResult.rf)]), after I run a complete.cases on testing data! Thanks a lot. — GaB, Jun 03 '20 at 10:36

Error in table(data, reference, dnn = dnn, ...) : all arguments must have the same length when run confusionMatrix with caret, in R

1 Answers1

Linked