Error in ConfusionMatrix the data and reference factors must have the same number of levels

Question

I've trained a tree model with R caret. I'm now trying to generate a confusion matrix and keep getting the following error:

Error in confusionMatrix.default(predictionsTree, testdata$catgeory) : the data and reference factors must have the same number of levels

prob <- 0.5 #Specify class split
singleSplit <- createDataPartition(modellingData2$category, p=prob,
                                   times=1, list=FALSE)
cvControl <- trainControl(method="repeatedcv", number=10, repeats=5)
traindata <- modellingData2[singleSplit,]
testdata <- modellingData2[-singleSplit,]
treeFit <- train(traindata$category~., data=traindata,
                 trControl=cvControl, method="rpart", tuneLength=10)
predictionsTree <- predict(treeFit, testdata)
confusionMatrix(predictionsTree, testdata$catgeory)

The error occurs when generating the confusion matrix. The levels are the same on both objects. I cant figure out what the problem is. Their structure and levels are given below. They should be the same. Any help would be greatly appreciated as its making me cracked!!

> str(predictionsTree)
 Factor w/ 30 levels "16-Merchant Service Charge",..: 28 22 22 22 22 6 6 6 6 6 ...
> str(testdata$category)
 Factor w/ 30 levels "16-Merchant Service Charge",..: 30 30 7 7 7 7 7 30 7 7 ...

> levels(predictionsTree)
 [1] "16-Merchant Service Charge"   "17-Unpaid Cheque Fee"         "18-Gov. Stamp Duty"           "Misc"                         "26-Standard Transfer Charge" 
 [6] "29-Bank Giro Credit"          "3-Cheques Debit"              "32-Standing Order - Debit"    "33-Inter Branch Payment"      "34-International"            
[11] "35-Point of Sale"             "39-Direct Debits Received"    "4-Notified Bank Fees"         "40-Cash Lodged"               "42-International Receipts"   
[16] "46-Direct Debits Paid"        "56-Credit Card Receipts"      "57-Inter Branch"              "58-Unpaid Items"              "59-Inter Company Transfers"  
[21] "6-Notified Interest Credited" "61-Domestic"                  "64-Charge Refund"             "66-Inter Company Transfers"   "67-Suppliers"                
[26] "68-Payroll"                   "69-Domestic"                  "73-Credit Card Payments"      "82-CHAPS Fee"                 "Uncategorised"   

> levels(testdata$category)
 [1] "16-Merchant Service Charge"   "17-Unpaid Cheque Fee"         "18-Gov. Stamp Duty"           "Misc"                         "26-Standard Transfer Charge" 
 [6] "29-Bank Giro Credit"          "3-Cheques Debit"              "32-Standing Order - Debit"    "33-Inter Branch Payment"      "34-International"            
[11] "35-Point of Sale"             "39-Direct Debits Received"    "4-Notified Bank Fees"         "40-Cash Lodged"               "42-International Receipts"   
[16] "46-Direct Debits Paid"        "56-Credit Card Receipts"      "57-Inter Branch"              "58-Unpaid Items"              "59-Inter Company Transfers"  
[21] "6-Notified Interest Credited" "61-Domestic"                  "64-Charge Refund"             "66-Inter Company Transfers"   "67-Suppliers"                
[26] "68-Payroll"                   "69-Domestic"                  "73-Credit Card Payments"      "82-CHAPS Fee"                 "Uncategorised"

In your error, `category` is spelled `catgeory`. If the problem is not related, what's the output of `identical(levels(predictionsTree),levels(testdata$category))` ? — fxi, Jul 17 '14 at 14:41
Hi thanks for that i ammended the silly spelling mistake....doh!!! I ran the identical function and it outputted [1] TRUE.........now I'm getting the following error when I run the confusionMatrix function.....Error in table(data, reference, dnn = dnn, ...) : all arguments must have the same length — user2987739, Jul 17 '14 at 15:36
Check for another misspelled `catgeory`, check `length(testdata$category)` and `length(predictionsTree`, check also your summary of both vectors. If you want just have a simple confusion matrix : `table(predictionsTree,testdata$category)` — fxi, Jul 17 '14 at 15:52

score 24 · Answer 1 · edited Jun 03 '18 at 21:59

24

Try use:

confusionMatrix(table(Argument 1, Argument 2))

Thats worked for me.

edited Jun 03 '18 at 21:59

Karolis Koncevičius

9,417
9
56
89

answered Jun 03 '18 at 11:14

Mayk Tulio

241
2
2

score 5 · Answer 2 · edited Feb 12 '19 at 11:02

5

Maybe your model is not predicting a certain factor. Use the table() function instead of confusionMatrix() to see if that is the problem.

edited Feb 12 '19 at 11:02

mag_zbc

6,801
14
40
62

answered Oct 31 '14 at 05:36

Red

61
1
2

4

You can add this as comment. – Top Cat Oct 31 '14 at 05:55
I found this very helpful, yet now I am wondering, it doesn't seem like there is much of a difference between the two. Is it only graphical? – Rich_Rich Jun 24 '18 at 23:13
If this is the case, then, how can we fix or work it around gracefully? – Nuclear03020704 Dec 21 '19 at 07:48

score 4 · Answer 3 · edited Feb 01 '18 at 16:09

4

Try specifying na.pass for the na.action option:

predictionsTree <- predict(treeFit, testdata,na.action = na.pass)

edited Feb 01 '18 at 16:09

duckmayr

16,303
3
35
53

answered Nov 12 '15 at 03:02

aristotll

8,694
6
33
53

score 2 · Answer 4 · answered Aug 09 '18 at 05:46

Change them into a data frame and then use them in confusionMatrix function:

pridicted <- factor(predict(treeFit, testdata))
real <- factor(testdata$catgeory)

my_data1 <- data.frame(data = pridicted, type = "prediction")
my_data2 <- data.frame(data = real, type = "real")
my_data3 <- rbind(my_data1,my_data2)

# Check if the levels are identical
identical(levels(my_data3[my_data3$type == "prediction",1]) , levels(my_data3[my_data3$type == "real",1]))

confusionMatrix(my_data3[my_data3$type == "prediction",1], my_data3[my_data3$type == "real",1],  dnn = c("Prediction", "Reference"))

score 1 · Answer 5 · answered Nov 21 '15 at 18:54

1

I had same issue but went ahead and changed it after reading data file like so..

data = na.omit(data)

Thanks all for pointer!

answered Nov 21 '15 at 18:54

Alicia

11
1

score 0 · Answer 6 · edited Jan 11 '15 at 08:47

0

Might be there are missing values in the testdata, Add the following line before "predictionsTree <- predict(treeFit, testdata)" to remove NAs. I had the same error and now it works for me.

testdata <- testdata[complete.cases(testdata),]

edited Jan 11 '15 at 08:47

Jason Aller

3,541
28
38
38

answered Jan 11 '15 at 07:12

EaswerC

11
5

score 0 · Answer 7 · answered May 21 '15 at 21:06

0

The length problem you're running into is probably due to the presence of NAs in the training set -- either drop the cases that are not complete, or impute so that you do not have missing values.

answered May 21 '15 at 21:06

orange1

2,871
3
32
58

score 0 · Answer 8 · answered Jun 24 '19 at 19:58

0

make sure you installed the package with all its dependencies:

install.packages('caret', dependencies = TRUE)

confusionMatrix( table(prediction, true_value) )

answered Jun 24 '19 at 19:58

desval

2,345
2
16
23

score 0 · Answer 9 · answered Jul 17 '19 at 13:03

0

If your data contains NAs then sometimes it will be considered as a factor level,So omit these NAs initially

DF = na.omit(DF)

Then,if your model fit is predicting some incorrect level,then it is better to use tables

confusionMatrix(table(Arg1, Arg2))

answered Jul 17 '19 at 13:03

Sanjay Nandakumar

383
2
10

score 0 · Answer 10 · answered Dec 08 '20 at 19:06

0

I just ran into the same problem, I solved it by using R ordered factor data type.

levels <- levels(predictionsTree)
levels <- levels[order(levels)]    
table(ordered(predictionsTree,levels), ordered(testdata$catgeory, levels))

answered Dec 08 '20 at 19:06

Guile

233
4
7

score 0 · Answer 11 · answered Jun 13 '23 at 09:52

0

Look at the data type! My issue was that data had type int and reference had num. They need the same type.

answered Jun 13 '23 at 09:52

Hannes

1

Your answer could be improved by sharing the an example with code of how to solve it. Generally, try to be as specific as possible. – Rico Jun 14 '23 at 20:44

Error in ConfusionMatrix the data and reference factors must have the same number of levels

11 Answers11

Linked