Implement softmax function with numpy array

Question

My current function is as follows:

def soft_max(z):
    t = np.exp(z)
    a = np.exp(z) / np.sum(t, axis=1)
    return a

However I get the error: ValueError: operands could not be broadcast together with shapes (20,10) (20,) since np.sum(t, axis=1) isn't a scalar.

I want to have t / the sum of each row but I don't know how to do this.

I tried that and i get a lot of `RuntimeWarning: overflow encountered in exp` — Wizard, May 19 '18 at 08:29
https://stackoverflow.com/questions/40726490/overflow-error-in-pythons-numpy-exp-function, https://nolanbconaway.github.io/blog/2017/softmax-numpy, https://stackoverflow.com/questions/9559346/deal-with-overflow-in-exp-using-numpy? — Divakar, May 19 '18 at 08:31

score 3 · Answer 1 · answered May 19 '18 at 13:13

3

You want to do something like (see this post)

def softmax(x, axis=None):
    x = x - x.max(axis=axis, keepdims=True)
    y = np.exp(x)
    return y / y.sum(axis=axis, keepdims=True)

answered May 19 '18 at 13:13

Paul Panzer

score 3 · Answer 2 · answered Jan 03 '19 at 17:34

3

As of version 1.2.0, scipy includes softmax as a special function:

Use the axis argument do run it over rows.

from scipy.special import softmax
softmax(arr, axis=0)

answered Jan 03 '19 at 17:34

Nolan Conaway

score 2 · Answer 3 · answered May 19 '18 at 09:19

2

suppose your z is a 2 dimensional array, try

def soft_max(z):
    t = np.exp(z)
    a = np.exp(z) / np.sum(t, axis=1).reshape(-1,1)
    return a

answered May 19 '18 at 09:19

lelouchkako

sorry, `np.sum(t, axis=1, keepdims=True)` is better than using `.reshape`. they both are doing the same thing. `Overflow` may be caused by your input data I think. – lelouchkako May 19 '18 at 09:23

3 Answers3