5

In the tensorflow conv1D layer documentation, it says that;

'When using this layer as the first layer in a model, provide an input_shape argument (tuple of integers or None, e.g. (10, 128) for sequences of 10 vectors of 128-dimensional vectors, or (None, 128) for variable-length sequences of 128-dimensional vectors.'

So I understand that we can input variable length sequences but when I use a ragged tensor input for conv1D layer, it gives me an error:

ValueError: Layer conv1d does not support RaggedTensors as input.

What is really meant with variable length sequences if not RaggedTensors?

Thank you,

Arwen
  • 168
  • 1
  • 13
  • 1
    Initially I thought maybe you could have a [`Masking`](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Masking) layer before and it would take that into account, but as far as I can see convolutional layers do not support masking (it is meant for RNN mainly). I think what that means in the documentation is that you can have batches with different sizes in the sequence length dimension. For example, in a dataset, [`padded_batch`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#padded_batch) can return batches with different shapes each time. – jdehesa May 13 '20 at 10:38
  • About ragged tensors, they do not seem to be supported, so your best bet is probably convert it to a regular tensor with [`to_tensor`](https://www.tensorflow.org/api_docs/python/tf/RaggedTensor#to_tensor). You can later come back to ragged if you save the [`row_lengths`](https://www.tensorflow.org/api_docs/python/tf/RaggedTensor#row_lengths) and pass the result to [`from_tensor`](https://www.tensorflow.org/api_docs/python/tf/RaggedTensor#from_tensor). – jdehesa May 13 '20 at 10:42
  • Thanks for the suggestion, I have padded the ragged tensor and it worked.. But performance has significantly went down... – Arwen May 14 '20 at 05:56
  • How do you mean? I thought you could not get it to run before. Performance went down with respect to what? – jdehesa May 14 '20 at 09:07
  • 1
    conv1D layer does not accept ragged tensors as input so I have padded it using tf.keras.preprocessing.sequence.pad_sequences() first and now it is working. And sorry for the confusion, I meant it takes a lot longer now. – Arwen May 14 '20 at 10:01

1 Answers1

5

Providing an answer here for the community, even if the answer is already present in the comment section.

tf.keras.layers.Conv1D does not support Ragged Tensors instead you can pad the sequences using tf.keras.preprocessing.sequence.pad_sequences and use it as an input to the Conv1D layer.

Here is the example to pad_sequenes.

sequence = [[1], [2, 3], [4, 5, 6]]
tf.keras.preprocessing.sequence.pad_sequences(sequence)

array([[0, 0, 1],[0, 2, 3],[4, 5, 6]], dtype=int32)

You can also do a fixed-length padding, change the padding values, and post padding like below: 

sequence = [[1], [2, 3], [4, 5, 6]]
tf.keras.preprocessing.sequence.pad_sequences(sequence,maxlen=2,value=-1,padding="post")

array([[ 1, -1],[ 2, 3],[ 5, 6]], dtype=int32)

  • can you please have a look here https://stackoverflow.com/questions/65869272/raggedtensor-request-to-tensorflow-serving-fails ? – Shlomi Schwartz Jan 26 '21 at 08:44