0

I am running the following code. I want to calculate accuracy of my ANN for test data. I am using windows platfrom, python 3.5

import numpy 
import pandas as pd
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.metrics import accuracy_score
dataset=pd.read_csv('main.csv')
dataset=dataset.fillna(0)
X=dataset.iloc[:, 0:6].values
#X = X[numpy.logical_not(numpy.isnan(X))]
y=dataset.iloc[:, 6:8].values
#y = y[numpy.logical_not(numpy.isnan(y))]
#regr = LinearRegression()
#regr.fit(numpy.transpose(numpy.matrix(X)), numpy.transpose(numpy.matrix(y)))

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test=train_test_split(X,y, test_size=0.24,random_state=0)

create model

model = Sequential()
model.add(Dense(4, input_dim=6, kernel_initializer='normal', activation='relu'))
model.add(Dense(4, kernel_initializer='normal', activation='relu'))
model.add(Dense(2, kernel_initializer='normal'))

Compile model

model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X_train, y_train, batch_size=5, epochs=5)

y_pred=model.predict(X_test)

Now, i want to calculate the accuracy of y_pred. Any help will be appreciated. The above code is self explanatory. I am currently using only 5 epochs just for experimenting.

Mohamed Ali JAMAOUI
  • 14,275
  • 14
  • 73
  • 117
Kashish Goyal
  • 37
  • 1
  • 9

4 Answers4

0

Keras already implements metrics such as accuracy, so you just need to change the model.compile line to:

model.compile(loss='mean_squared_error', optimizer='adam',
              metrics = ["accuracy"])

Then training and validation accuracy (in the [0, 1] range) will be presented at the progress bar during training, and you can compute accuracy with model.evaluate as well, which will return a tuple of loss and metrics (accuracy in this case).

Dr. Snoopy
  • 55,122
  • 7
  • 121
  • 140
  • Maybe a reminder that, normally, MSE loss & accuracy do not go together, as we have discussed [here](https://stackoverflow.com/questions/48775305/what-function-defines-accuracy-in-keras-when-the-loss-is-mean-squared-error-mse), would be useful here – desertnaut Mar 21 '18 at 12:58
0

Besides the suggestion of using keras. You can compute the accuracy using scikit-learn as follows: 

from sklearn.metrics import accuracy_score 
accuracy_score(y_test, y_pred)

For more information, check the documentation : sklearn.metrics.accuracy_score

Mohamed Ali JAMAOUI
  • 14,275
  • 14
  • 73
  • 117
0

Although in a narrow technical sense both answers already provided are correct, there is a more general issue with your question which affects the essence of it: are you in a regression or a classification context?

  • If you are in a regression context (as implied by your loss='mean_squared_error' and the linear activation in your output layer), then the simple augmentation of model compilation

    model.compile(loss='mean_squared_error', optimizer='adam',
      metrics = ["accuracy"])

    will, as Matias says, provide the accuracy. Nevertheless, accuracy is meaningless in a regression setting; see the answer & discussion here for more details.

  • If you are in a classification context (as implied by your wish to calculate the accuracy, which is meaningful only in classification), then your loss function should not be the MSE, but the cross-entropy instead, plus that the activation of your last layer should not be linear.

desertnaut
  • 57,590
  • 26
  • 140
  • 166
-1

to compute accuracy we can use model.evaluate function

RJFF
  • 37
  • 1
  • 5