1

I'm trying to write down the gini index calculation as a tensorflow cost function. Gini index is: https://en.wikipedia.org/wiki/Gini_coefficient

a numpy solution would be

def ginic(actual, pred):
    n = len(actual)
    a_s = actual[np.argsort(pred)]
    a_c = a_s.cumsum()
    giniSum = a_c.sum() / a_s.sum() - (n + 1) / 2.0
    return giniSum / n

Can someone help me figure out how to do this in tf (for example, in tf there is no argsort that can be part of a function that is differentiated, AFAIK)

Ilya
  • 561
  • 2
  • 17
  • There is an implementation of a differentiable sort operator, a pretty recent research result also available in TF https://github.com/google-research/fast-soft-sort – linello Jun 08 '21 at 15:45

1 Answers1

2

You can perform the argsorting by using tf.nn.top_k(). This function returns a tuple, the second element being the indices. Its order must be reversed since the order is descending.

def ginicTF(actual:tf.Tensor,pred:tf.Tensor):
    n = int(actual.get_shape()[-1])
    inds =  tf.reverse(tf.nn.top_k(pred,n)[1],axis=[0]) # this is the equivalent of np.argsort
    a_s = tf.gather(actual,inds) # this is the equivalent of numpy indexing
    a_c = tf.cumsum(a_s)
    giniSum = tf.reduce_sum(a_c)/tf.reduce_sum(a_s) - (n+1)/2.0
    return giniSum / n

Here is a code you can use for verification that this function returns the same numerical value as your numpy function ginic:

sess = tf.InteractiveSession()
ac = tf.placeholder(shape=(50,),dtype=tf.float32)
pr = tf.placeholder(shape=(50,),dtype=tf.float32)
actual  = np.random.normal(size=(50,))
pred  = np.random.normal(size=(50,))
print('numpy version: {:.4f}'.format(ginic(actual,pred)))
print('tensorflow version: {:.4f}'.format(ginicTF(ac,pr).eval(feed_dict={ac:actual,pr:pred})))
Lior
  • 2,019
  • 1
  • 15
  • 22
  • Ok, this looks good but when passed to a NN as a loss function it returns an error for line: ---> 14 n = int(actual.get_shape()[-1]) error: TypeError: \__int\__ returned non-int (type NoneType) It does work as expected if I just run a session – Ilya Oct 10 '17 at 07:39
  • I think this happens because the shape of your placeholder\tensor for `actual` is `(None,)`, which means that it doesn't have a pre-defined length, thus `n` cannot be computed at the time of graph construction. In that case, what you can do is just pass `n` (the length of the arrays) as an additional argument to the function, instead of computing it. – Lior Oct 10 '17 at 16:47
  • Ok, I wasn't able to resolve this (tried giving default value of n, but this does not solve it). I made a new question for this particular problem https://stackoverflow.com/questions/46674293/custom-loss-function-in-keras-how-to-deal-with-placeholders Again, thank you for writing down the function in TF! – Ilya Oct 10 '17 at 19:17