12

I am currently using SHAP Package to determine the feature contributions. I have used the approach for XGBoost and RandomForest and it worked really well. Since the data I am working on is a sequential data I tried using LSTM and CNN to train the model and then get the feature importance using the SHAP's DeepExplainer; but it is continuously throwing error. The error I am getting is:

AssertionError: <class 'keras.callbacks.History'> is not currently a supported model type!.

I am attaching the sample code as well (LSTM). It would be helpful if someone could help me with it.

shap.initjs()
model = Sequential()
model.add(LSTM(n_neurons, input_shape=(X.shape[1],X.shape[2]), return_sequences=True))
model.add(LSTM(n_neurons, return_sequences=False))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
h=model.fit(X, y, epochs=nb_epochs, batch_size=n_batch, verbose=1, shuffle=True)
background = X[np.random.choice(X.shape[0],100, replace=False)]
explainer = shap.DeepExplainer(h,background)
today
  • 32,602
  • 8
  • 95
  • 115
Yash Sharma
  • 142
  • 1
  • 8

1 Answers1

10

The returned value of model.fit is not the model instance; rather, it's the history of training (i.e. stats like loss and metric values) as an instance of keras.callbacks.History class. That's why you get the mentioned error when you pass the returned History object to shap.DeepExplainer. Instead, you should pass the model instance itself:

explainer = shap.DeepExplainer(model, background)
today
  • 32,602
  • 8
  • 95
  • 115
  • 1
    Hi, Thanks for the help it does work, and also for making me understand the base problem as well. – Yash Sharma Apr 30 '20 at 09:15
  • Does it work whatever the way the Keras model is created? I am experiencing issues when I create the neural network with model subclassing. – AleB Oct 27 '20 at 23:16