How do I compute one hot encoding using tf.one_hot?

Question

I'm trying to build a one hot encoding of y_train of mnist data-set using tensorflow. I couldn't understand how to do it?

# unique values 0 - 9
y_train = array([5, 0, 4, ..., 5, 6, 8], dtype=uint8)

In keras we'll do something like

# this converts it into one hot encoding
one hot_encoding = tf.keras.utils.to_categorical(y_train)

Where as in tf.one_hot what should be my input to indices & depth parameters? After doing one hot encoding how can I convert it back to numpy array from 2d-tensor?

score 6 · Accepted Answer · answered Jun 20 '19 at 23:52

I'm not familiar with Tensorflow but after some tests, this is what I've found:

tf.one_hot() takes an indices and a depth. The indices are the values to actually convert to a one-hot encoding. depth refers to the maximum value to utilize.

For example, take the following code:

y = [1, 2, 3, 2, 1]
tf.keras.utils.to_categorical(y)
sess = tf.Session();
with sess.as_default():
    print(tf.one_hot(y, 2).eval())
    print(tf.one_hot(y, 4).eval())
    print(tf.one_hot(y, 6).eval())

tf.keras.utils.to_categorical(y) Returns the following:

array([[0., 1., 0., 0.],
       [0., 0., 1., 0.],
       [0., 0., 0., 1.],
       [0., 0., 1., 0.],
       [0., 1., 0., 0.]], dtype=float32)

In contrast, the tf.one_hot() options (2, 4, and 6) do the following:

[[0. 1.]
 [0. 0.]
 [0. 0.]
 [0. 0.]
 [0. 1.]]
[[0. 1. 0. 0.]
 [0. 0. 1. 0.]
 [0. 0. 0. 1.]
 [0. 0. 1. 0.]
 [0. 1. 0. 0.]]
[[0. 1. 0. 0. 0. 0.]
 [0. 0. 1. 0. 0. 0.]
 [0. 0. 0. 1. 0. 0.]
 [0. 0. 1. 0. 0. 0.]
 [0. 1. 0. 0. 0. 0.]]

As can be seen here, to mimic tf.keras.utils.to_categorical() using tf.one_hot(), the depth parameter should be equivalent to the maximum value present in the array, +1 for 0. In this case, the maximum value is 3, so there are four possible values in the encoding - 0, 1, 2, and 3. As such, a depth of 4 is required to represent all of these values in the one-hot encoding.

As for conversion to numpy, as shown above, using a Tensorflow session, running eval() on a tensor converts it to a numpy array. For methods on doing this, refer to How can I convert a tensor into a numpy array in TensorFlow?.

I'm not familiar with Tensorflow but I hope this helps.

Note: for the purposes of MNIST, a depth of 10 should be sufficient.

score 2 · Answer 2 · answered Dec 12 '19 at 12:03

I'd like to counter what @Andrew Fan has said. First, the above y label list does not start at index 0, which is what is required. Just look at the first column (i.e. index 0) in all of those examples: they're all empty. This will create a redundant class in the learning and may cause problems. One hot creates a simple list with 1 for that index position and zeros elsewhere. Therefore, your depth has to be the same as the number of classes, but you also have to start at index 0.

How do I compute one hot encoding using tf.one_hot?

2 Answers2