Weighted Gini coefficient in Python

Question

Here's a simple implementation of the Gini coefficient in Python, from https://stackoverflow.com/a/39513799/1840471:

def gini(x):
    # Mean absolute difference.
    mad = np.abs(np.subtract.outer(x, x)).mean()
    # Relative mean absolute difference
    rmad = mad / np.mean(x)
    # Gini coefficient is half the relative mean absolute difference.
    return 0.5 * rmad

How can this be adjusted to take an array of weights as a second vector? This should take noninteger weights, so not just blow up the array by the weights.

Example:

gini([1, 2, 3])  # No weight: 0.22.
gini([1, 1, 1, 2, 2, 3])  # Manually weighted: 0.23.
gini([1, 2, 3], weight=[3, 2, 1])  # Should also give 0.23.

Unfortunately the `mad` line calculates the mean of a matrix, so can't apply weights there. I suspect some matrix math using weights and the `np.subtract.outer` result, plus a normal weighted average to calculate `rmd` , will do the trick. — Max Ghenis, Feb 26 '18 at 05:04
That didn't quite work, but the accepted answer did the trick. — Max Ghenis, Feb 26 '18 at 05:25
Can we avoid an all pair comparison of the input? `.outer` applies `.subtract` to all pairs but that might be expensive for very large inputs that do not fit in memory. Is there a streaming algorithm that does not require the input to be sorted? — Marsellus Wallace, Dec 18 '20 at 22:34
@MarsellusWallace this solution is more efficient: https://stackoverflow.com/questions/48999542/more-efficient-weighted-gini-coefficient-in-python — Max Ghenis, Dec 21 '20 at 06:15

score 4 · Accepted Answer · answered Feb 26 '18 at 05:14

the calculation of mad can be replaced by:

x = np.array([1, 2, 3, 6])
c = np.array([2, 3, 1, 2])

count = np.multiply.outer(c, c)
mad = np.abs(np.subtract.outer(x, x) * count).sum() / count.sum()

np.mean(x) can be replaced by:

np.average(x, weights=c)

Here is the full function:

def gini(x, weights=None):
    if weights is None:
        weights = np.ones_like(x)
    count = np.multiply.outer(weights, weights)
    mad = np.abs(np.subtract.outer(x, x) * count).sum() / count.sum()
    rmad = mad / np.average(x, weights=weights)
    return 0.5 * rmad

to check the result, gini2() use numpy.repeat() to repeat elements:

def gini2(x, weights=None):
    if weights is None:
        weights = np.ones(x.shape[0], dtype=int)    
    x = np.repeat(x, weights)
    mad = np.abs(np.subtract.outer(x, x)).mean()
    rmad = mad / np.mean(x)
    return 0.5 * rmad

If you find yourself here and want a much more efficient way to compute in Python, see https://stackoverflow.com/questions/48999542/more-efficient-weighted-gini-coefficient-in-python — themaninthewoods, May 24 '18 at 20:56

Weighted Gini coefficient in Python

1 Answers1

Linked