2

Let's say my data has 25 features. In Keras, I could easily implement a Embedding layer for each input feature and merge them together to feed to later layers.

I see that tf.nn.embedding_lookup accepts a id parameter which could be just a plain integer or a array of integers( [1,2,3,..] ). However, the feature input is often of the shape 

x = tf.placeholder(tf.float32, [None, in_feature_num])

I could split the feature to their own by using 

X = tf.split(1,in_feature_num,x)

and each feature input is of the shape [?, 1]. But embedding_lookup does not accpet a shape of [?,1], and since we don't have a specified row length, I can't reshape or unpack it to the shape like [?].

So, how could I transform a input like 

[[1],
 [2],
 [3],
  ...
]

into a embedding representation like this :

[
  [....], #a vector
  [....], #a vector
  [....], #a vector 
   ...
]

Related SO post are: What does tf.nn.embedding_lookup function do? and TensorFlow Embedding Lookup but those posts do not solved my problem.

Community
  • 1
  • 1
Allan Ruin
  • 5,229
  • 7
  • 37
  • 42
  • I am not a 100% sure what you mean by "your data has features". My understanding is that embeddings are the representatives of your input data. So they are features. You can choose to add an embedding layer in front of your NN and learn the weights to actually learn an embedding layer or can add a generalized pre-trained one. Could you be more specific about your data? – roopalgarg Jan 27 '17 at 02:45
  • Are you saying you have a batch of 25 features, and you want each feature to have its own embedding? – mazecreator Feb 10 '17 at 21:00

1 Answers1

-2

I think I know what you mean. The question is what are the features? If they are numerical, then you don't really need an embedding layer - you can use a fully connected layer instead. If they are categorical however, I think you have two options: 1. separate embedding for every features (so you'd end up with 25 embedding layers, which would need to be concatenated) 2. merge the features into a single one first

bosmart
  • 325
  • 3
  • 11
  • yes,"end up with 25 embedding layers" that's what I was tring to implement. I know how to do this in keras, however, I don't know how to do it in tensorflow... – Allan Ruin Feb 12 '17 at 15:01
  • This is not an answer... it is clear from the question that he wants option 1, but you don't tell him how to do this – Jan van der Vegt Feb 15 '17 at 15:28
  • 1
    Question is very clear. Also, it is important to have embedding laters especially when merging two vectors as an input to the network. For an example let's say there are two vectors where you need to use as the input. Think one vector has dimensions of 2048 and other one is one hot vector with 4 dimensions. So you need embedding layers. This can be same for trying to get an embedding for each feature. – Shamane Siriwardhana Jun 05 '18 at 07:48