I came across eli5.permutation_importance get_score_importances from the following article: https://towardsdatascience.com/how-to-find-feature-importances-for-blackbox-models-c418b694659d; here is the documentation.
When I try to use get_score_importances on my neural network my RAM increases and my session crashes in Google Colab. The shape of x_test is (34463, 2355).
Here is my NN Model
NN_model = Sequential()
NN_model.add(Dense(128, kernel_initializer='normal',input_dim =
x_train.shape[1], activation='relu'))
NN_model.add(Dense(256, kernel_initializer='normal',activation='relu'))
NN_model.add(Dense(1, kernel_initializer='normal',activation='linear'))
# Compile the network :
NN_model.compile(loss='mean_absolute_error', optimizer='adam', metrics=
['mean_absolute_error'])
NN_model.summary()
#Define a checkpoint callback
checkpoint_name = 'Weights-{epoch:03d}--{val_loss:.5f}.hdf5'
checkpoint = ModelCheckpoint(checkpoint_name, monitor='val_loss', verbose =
1, save_best_only = True,
mode ='auto')
callbacks_list = [checkpoint]
#train a neural network model
NN_model.fit(x_train, y_train, epochs=500, batch_size=32, validation_split =
0.2,
callbacks=callbacks_list)
Here's the code I use to run get_score_importances
import numpy as np
from eli5.permutation_importance import get_score_importances
import numpy as np
y_array = np.array(y_test)
x_array = np.array(x_test)
#score function
from sklearn.metrics import mean_absolute_error
def score(X, y):
predictions = NN_model.predict(X)
return mean_absolute_error(y , predictions)
# This function takes only numpy arrays as inputs
#The following line crashes the session
base_score, score_decreases = get_score_importances(score,x_array, y_array)
#The following line never gets run
feature_importances = np.mean(score_decreases, axis=0)
Is there something I'm doing wrong that's making Google Colab crash? I'm running 8GB of RAM with Intel Core i7 5500U CPU at 2.4 GHZ, Windows 64 bit. Any help would be much appreciated.
