0

My training data contains ~1500 labels(string,one label per record) and I want to do batch training (just load one batch into memory to update weights in a neural network). I was wondering if there is a class in tensorflow to do one hot encoding for the labels in each batch? Something like in sklearn we can do

onehot_encoder = OneHotEncoder(sparse=False)
onehot_encoder.fit(entire training labels)

And then in each batch in tensorflow session, I can transform my batch label and feed into tensorflow for traning

batch_label = onehot_encoder.transform(batch training labels)
sess.run(feed_dict={x:...,y:batch_label)

An example will be appreciated. Thanks.

Undecided
  • 611
  • 8
  • 13
  • You should be able to do a classifier without doing the one-hot encoding. You can use the sparse_cross_entropy_with_logits. – Aaron Sep 30 '17 at 16:47
  • @Aaron I guess both the label and logits in the function must have shape of [None,Number of Classes]?. However, my data now is one label per record – Undecided Sep 30 '17 at 18:20
  • That's not true. Look at the sparse cross entropy function. – Aaron Sep 30 '17 at 20:20

1 Answers1

0

I think this post is similar to that one Tensorflow One Hot Encoder?

A sort answer from this link http://www.tensorflow.org/api_docs/python/tf/one_hot

indices = [0, 1, 2]
depth = 3
tf.one_hot(indices, depth)  
# output: [3 x 3]
# [[1., 0., 0.],
#  [0., 1., 0.],
#  [0., 0., 1.]]

Just posting it to save your time ;)

MrInCrysis
  • 61
  • 8