Recurrent Neural Networks (RNN) With Attention Mechanism is generally used for Machine Translation and Natural Language Processing. In Python, implementation of RNN With Attention Mechanism is abundant in Machine Translation (For Eg. https://talbaumel.github.io/blog/attention/, however what I would like to do is to use RNN With Attention Mechanism on a temporal data file (not any textual/sentence based data).
I have a CSV file with of dimensions 21392 x 1972, which I have converted to a Dataframe using Pandas. The first column is of Datetime Format and last column consists of target classes like "Class1", "Class2", "Class3" etc. which I would like to identify. So in total, there are 21392 rows (instances of data in 10 minutes time-steps) and 1971 features. The last (1972th column) is the label column, with 14 different classes in total.
I have looked into available implementation documentation on Keras (https://medium.com/datalogue/attention-in-keras-1892773a4f22) as well as on Tensorflow (Visualizing attention activation in Tensorflow), but none of them seem to be doing what I want to accomplish. I understand that this is an unusual approach, but would want to try this and use the attention mechanism because many of my features are presumably redundant in the data.
import pandas as pd
mydataset = pd.read_csv('final_merged_data.csv')
It is predominant from existing literature that an Attention Mechanism works quite well when coupled into the RNN. I am unable to locate any such implementation of RNN with Attention Mechanism, which can also provide a visualisation as well. I am also unable to understand how I can convert my data into a sequence (or a list of lists) so that I can use it with One Hot Encoding afterwards for using RNN with Attention. I am new to using Python as well as Keras/Tensorflow, and am quite confused on the procedure to convert my data/typecast it to a form which will be able to mimic the sequence classification problem. My problem is basically of multi-class classification, like one would normally do using Machine Learning Classifiers to predict the labels, but using RNN with Attention. Any help in this regard would be highly appreciated. Cheers!
