0

I am trying to solve the problem of sequence completion. Let's suppose we have ground truth sequence (1,2,4,7,6,8,10,12,18,20)

The input to our model is an incomplete sequence. i.e (1,2,4, _ , _ ,_,10,12,18,20). From this incomplete sequence, we want to predict the original sequence (Ground Truth sequence). Which deep learning models can be used to solve this problem?

Is this the problem of encoder-decoder LSTM architecture?

Note: we have thousands of complete sequences to train and test the model.

Any help is appreciated.

Asif Khan
  • 1,228
  • 1
  • 11
  • 22

2 Answers2

1

This not exactly sequence-to-sequence problem, this is a sequence labeling problem. I would suggest either stacking bidirectional LSTM layers followed by a classifier or Transformer layers followed by a classifier.

Encoder-decoder architecture requires plenty of data to train properly and is particularly useful if the target sequence can be of arbitrary length, only vaguely depending on the source sequence length. It would eventually learn to do the job with enough, but sequence labeling is a more straightforward problem.

With sequence labeling, you can set a custom mask over the output, so the model will only predict the missing numbers. An encoder-decoder model would need to learn to copy most of the input first.

Jindřich
  • 10,270
  • 2
  • 23
  • 44
1

In your sequence completion task, are you trying to predict next items in a sequence or learn only the missing values? Training a neural network with missing data is an issue on its own terms. If you're using Keras and LSTM-type NN for solving your problem, you should consider masking, you can refer to this stackoverflow thread for more details: Multivariate LSTM with missing values Regarding predicting the missing values, why not try auto-encoders?

sol4ris
  • 41
  • 5
  • actually I am only trying to learn missing values. or in other words, if I input partial sequence, it should return full sequence. Is this possible using any of deep learning approaches? – Asif Khan Oct 17 '19 at 07:35
  • 1
    So this is a classical data imputation problem. There are traditional approaches based on probabilistic models or kNN or Decision Trees -- for more details see http://jmlr.org/papers/v18/17-073.html. But yes, you can use deep learning, if you want. Denoising autoencoders (DAE) would be one of them. Some examples: [1](https://blog.keras.io/building-autoencoders-in-keras.html), [2](https://www.curiousily.com/posts/data-imputation-using-autoencoders) – sol4ris Oct 17 '19 at 08:33
  • actually I want to take this as sequence prediction problem. I want to hide range of values from input sequence and predict those numbers. e,g. input sequence is 1, 2,3,4,5,6,7,8, 9, 10, then I hide 6,7,8 from it. input rest of the values and predict the whole sequence from values 1 to 10. – Asif Khan Nov 14 '19 at 07:08