How to find most frequent values in numpy ndarray?

Question

I have a numpy ndarray with shape of (30,480,640), the 1th and 2th axis representing locations(latitude and longitute), the 0th axis contains actual data points.I want to use the most frequent value along the 0th axis at each location, which is to construct a new array with shape of (1,480,640).ie:

>>> data
array([[[ 0,  1,  2,  3,  4],
        [ 5,  6,  7,  8,  9],
        [10, 11, 12, 13, 14],
        [15, 16, 17, 18, 19]],

       [[ 0,  1,  2,  3,  4],
        [ 5,  6,  7,  8,  9],
        [10, 11, 12, 13, 14],
        [15, 16, 17, 18, 19]],

       [[40, 40, 42, 43, 44],
        [45, 46, 47, 48, 49],
        [50, 51, 52, 53, 54],
        [55, 56, 57, 58, 59]]])

(perform calculation)

>>> new_data 
array([[[ 0,  1,  2,  3,  4],
        [ 5,  6,  7,  8,  9],
        [10, 11, 12, 13, 14],
        [15, 16, 17, 18, 19]]])

The data points will contain negtive and positive floating numbers. How can I perform such calculations? Thanks a lot!

I tried with numpy.unique,but I got "TypeError: unique() got an unexpected keyword argument 'return_inverse'".I'm using numpy version 1.2.1 installed on Unix and it doesn't support return_inverse..I also tried mode,but it takes forever to process such large amount of data...so is there an alternative way to get the most frequent values? Thanks again.

What do you mean by dominant value? I don't understand the question. — Henry Gomersall, Sep 06 '12 at 09:43
I'll second the comment by @HenryGomersall - I haven't a Scooby what your question is either... — Jon Clements, Sep 06 '12 at 09:57
Sorry for your confusion...I meant the most frequent values.. — oops, Sep 06 '12 at 09:58
Does this answer your question? [Most efficient way to find mode in numpy array](https://stackoverflow.com/questions/16330831/most-efficient-way-to-find-mode-in-numpy-array) — mpx, Oct 03 '21 at 13:50

ecatmur · Accepted Answer · 2012-09-10T11:56:19.700

To find the most frequent value of a flat array, use unique, bincount and argmax:

arr = np.array([5, 4, -2, 1, -2, 0, 4, 4, -6, -1])
u, indices = np.unique(arr, return_inverse=True)
u[np.argmax(np.bincount(indices))]

To work with a multidimensional array, we don't need to worry about unique, but we do need to use apply_along_axis on bincount:

arr = np.array([[5, 4, -2, 1, -2, 0, 4, 4, -6, -1],
                [0, 1,  2, 2,  3, 4, 5, 6,  7,  8]])
axis = 1
u, indices = np.unique(arr, return_inverse=True)
u[np.argmax(np.apply_along_axis(np.bincount, axis, indices.reshape(arr.shape),
                                None, np.max(indices) + 1), axis=axis)]

With your data:

data = np.array([
   [[ 0,  1,  2,  3,  4],
    [ 5,  6,  7,  8,  9],
    [10, 11, 12, 13, 14],
    [15, 16, 17, 18, 19]],

   [[ 0,  1,  2,  3,  4],
    [ 5,  6,  7,  8,  9],
    [10, 11, 12, 13, 14],
    [15, 16, 17, 18, 19]],

   [[40, 40, 42, 43, 44],
    [45, 46, 47, 48, 49],
    [50, 51, 52, 53, 54],
    [55, 56, 57, 58, 59]]])
axis = 0
u, indices = np.unique(arr, return_inverse=True)
u[np.argmax(np.apply_along_axis(np.bincount, axis, indices.reshape(arr.shape),
                                None, np.max(indices) + 1), axis=axis)]
array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14],
       [15, 16, 17, 18, 19]])

NumPy 1.2, really? You can approximate np.unique(return_inverse=True) reasonably efficiently using np.searchsorted (it's an additional O(n log n), so shouldn't change the performance significantly):

u = np.unique(arr)
indices = np.searchsorted(u, arr.flat)

@ ecatmur,I'm using numpy version 1.2.1 and it doesn't support np.unique(return_inverse)..any suggestions? — oops, Sep 10 '12 at 09:52
@oops see above, you'll have to test it yourself as I have no idea where I'd even find such an old version of numpy ;) — ecatmur, Sep 10 '12 at 11:56
For big ndarrays with more dimensions, this approach is not really suitable, since it will allocate an array of size (N_dim1,N_dim2,...,N_unique) which gets out of hand very quickly. — CheshireCat, Apr 29 '20 at 11:39

score 9 · Answer 2 · answered Sep 06 '12 at 20:05

9

Use SciPy's mode function:

import numpy as np
from scipy.stats import mode

data = np.array([[[ 0,  1,  2,  3,  4],
                  [ 5,  6,  7,  8,  9],
                  [10, 11, 12, 13, 14],
                  [15, 16, 17, 18, 19]],

                 [[ 0,  1,  2,  3,  4],
                  [ 5,  6,  7,  8,  9],
                  [10, 11, 12, 13, 14],
                  [15, 16, 17, 18, 19]],

                 [[40, 40, 42, 43, 44],
                  [45, 46, 47, 48, 49],
                  [50, 51, 52, 53, 54],
                  [55, 56, 57, 58, 59]]])

print data

# find mode along the zero-th axis; the return value is a tuple of the
# modes and their counts.
print mode(data, axis=0)

answered Sep 06 '12 at 20:05

Taro Sato

1,444
1
15
19

Thank you Taro Sato,but it takes very very long time to process large arrays..any suggestion to speed it up? – oops Sep 10 '12 at 07:03
Okay, I noticed that you want to do this with floats. To do that, I think you need a slightly different approach, since it doesn't really make sense to ask what is the most frequent float, since there's only a small chance that two floats coincide from repeated experiments. Do you really need to find such a weird thing? I you know (roughly) the distribution of your sample, then there are better measures to compute, such as mean and median, to find out what the most likely number in your sample is. – Taro Sato Sep 10 '12 at 17:54
do people still widely used scipy package? Read somewhere that mean from scipy is deprecated. Just curious to know :) – Mona Jalal Jul 30 '16 at 23:07
It's been a while since this answer was written, so I would not be surprised if the function has been deprecated (in favor of `np.mean` for example)... but I think I was commenting on general approach, not a specific function of a package. – Taro Sato Jul 30 '16 at 23:32

score 1 · Answer 3 · answered Nov 12 '20 at 18:49

A slightly better solution in my opinion is the following

tmpL = np.array([3, 2, 3, 2, 5, 2, 2, 3, 3, 2, 2, 2, 3, 3, 2, 2, 3, 2, 3, 2])
unique, counts = np.unique(tmpL, return_counts=True)
return unique[np.argmax(counts)]

Using np.unique we can get the count of each unique elements. The index of the max element in counts will be the corresponding element in unique.

score 0 · Answer 4 · answered Sep 06 '12 at 10:34

0

flatten your array, then build a collections.Counter from it. As usual, take special care when comparing floating-point numbers.

answered Sep 06 '12 at 10:34

ev-br

24,968
9
65
78

score 0 · Answer 5 · answered Aug 28 '20 at 08:02

Explaining @ecatmurs part

u[np.argmax(np.apply_along_axis(np.bincount, axis, indices.reshape(arr.shape),
                                None, np.max(indices) + 1), axis=axis)]

a little bit more and restructuring it to be more concise when re-reading it (because I used this solution and after a few weeks I was wondering what had happened in this function):

axis = 0
uniques, indices = np.unique(arr, return_inverse=True)

args_for_bincount_fn = None, np.max(indices) + 1
binned_indices = np.apply_along_axis(np.bincount,
                            last_axis, 
                            indices.reshape(arr.shape),
                            *args_for_bincount_fn)

most_common = uniques[np.argmax(binned_indices,axis=axis)]

How to find most frequent values in numpy ndarray?

5 Answers5

Linked