Stack Overflow
Stack Overflow

Questions tagged [sequence-to-sequence]

This tag is used for Google's deprecated seq2seq framework, an encoder-decoder framework for Tensorflow (revamped version is called Neural Machine Translation)

94 questions
26
votes
5 answers

How to convert predicted sequence back to text in keras?

I have a sequence to sequence learning model which works fine and able to predict some outputs. The problem is I have no idea how to convert the output back to text sequence. This is my code. from keras.preprocessing.text import…
Eka
  • 14,170
  • 38
  • 128
  • 212
16
votes
2 answers

TypeError: can't pickle _thread.lock objects in Seq2Seq

I'm having trouble using buckets in my Tensorflow model. When I run it with buckets = [(100, 100)], it works fine. When I run it with buckets = [(100, 100), (200, 200)] it doesn't work at all (stacktrace at bottom). Interestingly, running…
Evan Weissburg
  • 1,564
  • 2
  • 17
  • 38
12
votes
3 answers

How should we pad text sequence in keras using pad_sequences?

I have coded a sequence to sequence learning LSTM in keras myself using the knowledge gained from the web tutorials and my own intuitions. I converted my sample text to sequences and then padded using pad_sequence function in keras. from…
Eka
  • 14,170
  • 38
  • 128
  • 212
12
votes
1 answer

Visualizing attention activation in Tensorflow

Is there a way to visualize the attention weights on some input like the figure in the link above(from Bahdanau et al., 2014), in TensorFlow's seq2seq models? I have found TensorFlow's github issue regarding this, but I couldn't find out how to…
reiste
  • 123
  • 1
  • 5
9
votes
1 answer

How to use the PyTorch Transformer with multi-dimensional sequence-to-seqence?

I'm trying to go seq2seq with a Transformer model. My input and output are the same shape (torch.Size([499, 128]) where 499 is the sequence length and 128 is the number of features. My input looks like: My output looks like: My training loop is: …
Shamoon
  • 41,293
  • 91
  • 306
  • 570
9
votes
1 answer

Trouble understanding tf.contrib.seq2seq.TrainingHelper

I managed to build a sequence to sequence model in tensorflow using the tf.contrib.seq2seq classes in 1.1 version. For know I use the TrainingHelper for training my model. But does this helper feed previously decoded values in the decoder for…
JimZer
  • 918
  • 2
  • 9
  • 19
9
votes
3 answers

scheduled sampling in Tensorflow

The newest Tensorflow api about seq2seq model has included scheduled…
Kevin Zeng
  • 121
  • 1
  • 5
8
votes
0 answers

Multivariate binary sequence prediction with CRF

this question is an extension of this one which focuses on LSTM as opposed to CRF. Unfortunately, I do not have any experience with CRFs, which is why I'm asking these questions. Problem: I would like to predict a sequence of binary signal for…
IVR
  • 1,718
  • 2
  • 23
  • 41
7
votes
2 answers

Sequence to Sequence - for time series prediction

I've tried to build a sequence to sequence model to predict a sensor signal over time based on its first few inputs (see figure below) The model works OK, but I want to 'spice things up' and try to add an attention layer between the two LSTM…
Roni Gadot
  • 437
  • 2
  • 19
  • 30
6
votes
1 answer

How exactly does Keras take dimension argumentsfor LSTM / time series problems?

I can't seem to find a concrete answer to the question of how to feed data into Keras. Most examples seem to work off image / text data and have clearly defined data points. I'm trying to feed music into an LSTM neural network. I want the network…
Tyler Carey
  • 71
  • 4
6
votes
1 answer

How to code a sequence to sequence RNN in keras?

I am trying to write a sequence to sequence RNN in keras. I coded this program using what I understood from the web. I first tokenized the text then converted the text into sequence and padded to form feature variable X. The target variable Y was…
Eka
  • 14,170
  • 38
  • 128
  • 212
5
votes
1 answer

Multivariate binary sequence prediction with LSTM

I'm working on a sequence forecasting problem and I don't have much experience in this area, so some of the below questions might be naive. FYI: I've created a follow-up question with a focus on CRFs here I have the following problem: I would like…
IVR
  • 1,718
  • 2
  • 23
  • 41
4
votes
1 answer

How can i use BERT fo machine Translation?

I got a big problem. For my bachelor thesis I have to make a machine tranlation model with BERT. But I am not getting anywhere right now. Do you know a documentation or something that can help me here? I have read some papers in that direction…
Darin
  • 83
  • 1
  • 5
4
votes
1 answer

Concept of Bucketing in Seq2Seq model

To handle sequences of different lengths we use bucketing and padding. In bucketing we make different bucket for some max_len and we do this to reduce the amount of padding, after making different buckets we train different model on different…
Shubham Tyagi
  • 181
  • 1
  • 3
  • 14
4
votes
1 answer

AttentionDecoderRNN without MAX_LENGTH

From the PyTorch Seq2Seq tutorial, http://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html#attention-decoder We see that the attention mechanism is heavily reliant on the MAX_LENGTH parameter to determine the output dimensions of…
alvas
  • 115,346
  • 109
  • 446
  • 738
1
2 3 4 5 6 7