Softmax function in Tensorflow not displaying correct answer

Question

I was testing out the softmax function from Tensorflow but the answers I got don't appear to be correct.

So in the code below kh is a [5,4] matrix. softmaxkh should be the softmax matrix of kh. However even without doing the calculations, you can tell that the maximum numbers in a particular column or row of kh do not necessarily correspond to the the maximum numbers in softmaxkh.

For example '65' in the middle row of the last column is the highest number in both its column and row however in both its row and column in softmaxkh it does not represent the highest number.

import tensorflow as tf 

kh = tf.random_uniform(
    shape= [5,4],
    maxval=67,
    dtype=tf.int32,
    seed=None,
    name=None
)

sess = tf.InteractiveSession()

kh = tf.cast(kh, tf.float32)

softmaxkh = tf.nn.softmax(kh)


print(sess.run(kh))

Which returns

    [[ 55.  49.  48.  30.]
     [ 21.  39.  20.  11.]
     [ 40.  33.  58.  65.]
     [ 55.  19.  12.  24.]
     [ 17.   8.  14.   0.]]

and

print(sess.run(softmaxkh))

returns

    [[  1.42468502e-21   9.99663830e-01   8.31249167e-07   3.35349847e-04]
     [  3.53262839e-24   1.56288218e-18   1.00000000e+00   3.13913289e-17]
     [  6.10305051e-06   6.69280719e-03   9.93300676e-01   3.03852971e-07]
     [  2.86251861e-20   2.31952296e-16   8.75651089e-27   1.00000000e+00]
     [  5.74948687e-19   2.61026280e-23   9.99993801e-01   6.14417422e-06]]

P-Gn · Accepted Answer · 2017-07-26T15:07:37.830

That is because a random generator such as random_uniform draws different numbers every time you call it.

You need to store the result in a Variable to reuse random generated values across different graph runs:

import tensorflow as tf 

kh = tf.random_uniform(
    shape= [5,4],
    maxval=67,
    dtype=tf.int32,
    seed=None,
    name=None
)

kh = tf.cast(kh, tf.float32)
kh = tf.Variable(kh)

sess = tf.InteractiveSession()
tf.global_variables_initializer().run()

softmaxkh = tf.nn.softmax(kh)

# run graph
print(sess.run(kh))
# run graph again
print(sess.run(softmaxkh))

Atlernatively, if those values are used only once but at multiple locations, you could run the graph calling all the desired output at once.

import tensorflow as tf 

kh = tf.random_uniform(
    shape= [5,4],
    maxval=67,
    dtype=tf.int32,
    seed=None,
    name=None
)

kh = tf.cast(kh, tf.float32)

sess = tf.InteractiveSession()

softmaxkh = tf.nn.softmax(kh)

# produces consistent output values
print(sess.run([kh, softmaxkh))
# also produces consistent values, but different from above
print(sess.run([kh, softmaxkh))

Softmax function in Tensorflow not displaying correct answer

1 Answers1