3

The goal is to transform each index to a embedded vector and then average all vectors to a single one. I must to ignore padded zeros!

The averaged vector should be transferred to the next layers.

This is my code :

from keras.layers import Embedding,Input,AveragePooling1D
from keras.models import Model
from keras.preprocessing.text import Tokenizer as Keras_Tokenizer
from keras.preprocessing.sequence import pad_sequences
import numpy as np


embedding_size = 4
vocab_size = 9 + 1
max_sequence_length = 5 


my_input = Input(shape=(max_sequence_length,), name='input')
embedding_layer = Embedding(output_dim=embedding_size, input_dim=vocab_size,input_length=1, mask_zero=True,name='my_embedding')
embedding = embedding_layer(my_input)



avg=AveragePooling1D(pool_size=max_sequence_length)(embedding)#Calc average off all embedding vectors
model = Model(inputs= [my_input], outputs=avg)
model.get_weights() 



aa = np.array([[0,0,2,4]])#sanity checks
model.predict(aa)[0][0]

And getting this error :

TypeError: Layer average_pooling1d_1 does not support masking, but was passed an input_mask: Tensor("my_embedding_9/NotEqual:0", shape=(?, 5), dtype=bool)

Any one can assist?

syltruong
  • 2,563
  • 20
  • 33
Boris
  • 2,005
  • 2
  • 11
  • 10
  • Switch to just `Average` as your pool size is the same as the sequence length. `Average` should support masking. – nuric Aug 12 '18 at 15:02
  • 1
    `Average` takes a list of tensors as input, which not suited for the current case (see [here](https://keras.io/layers/merge/#average)). It is a `merge` layer. What we are looking for instead is a `GlobalAveragePooling1D` which supports masking. – syltruong Aug 20 '18 at 03:37

1 Answers1

0

Your use-case is very closely related to this one.

It will get you to write a custom layer to support masking in GlobalAveragePooling1D or AveragePooling1D.

syltruong
  • 2,563
  • 20
  • 33