I am new to machine learning and tried to follow the tutorial https://www.tensorflow.org/tutorials/audio/music_generation. I noticed that sometimes the results become very bad, depending on the data input. After talking about it, I now suspect that this might be a case of gradient vanishing/exploding.
In order to check this hypothesis, I would like to plot the gradient. However, everything I found online didn't seem to work or I was unable to make it work with my limited knowledge on that matter.
What I have tried
- How to visualize RNN/LSTM gradients in Keras/TensorFlow?. I did try rolling TensorFlow back to 2.9, but the input wasn't compatible with my input and I didn't see an easy way to fix it. Just splitting the label and data again, didn't seem to work.
- I tried taking a look at the weights directly instead of the gradients, but just got more confused, because the heatmap is a bit hard to get - at least for me: How to get weight in each layer and epoch then save in file
