Find most frequent value per coordinate in multiple 2d Arrays

Question

I have multiple 2d arrays like this for example:

A = [[-1, -1, 0, 1, -1], [1, 1, 0, -1, -1], [-1, -1, -1, -1, -1], [-1, 1, -1, -1, 0]]
B = [[-1, -1, 0, 1, -1], [1, -1, 0, -1, -1], [0, 1, -1, 1, -1], [-1, 1, -1, -1, -1]]
C = [[0, -1, 0, 1, -1], [1, -1, 0, -1, -1], [0, 1, -1, 1, -1], [-1, 1, -1, -1, -1]]
D = [[-1, -1, 0, 1, 0], [0, 0, -1, 0, 1], [0, 1, -1, 1, -1], [-1, 1, -1, -1, -1]]

I need to find the most frequent value across each respective coordinate so the output would be like this:

E = [[-1 -1 0 1 -1],[1 -1 0 -1 -1],[0 1 -1 1 -1],[-1 1 -1 -1 -1]]

I can definitely loop through each of these arrays but I was looking for a vectorised approach. The elements can be around 10-11 in number and arrays dimensions are around 900X900.

Is it possible to solve this using list comprehension?

Does this answer your question? [Most efficient way to find mode in numpy array](https://stackoverflow.com/questions/16330831/most-efficient-way-to-find-mode-in-numpy-array) — Mechanic Pig, Jun 12 '22 at 10:52

I'mahdi · Answer 1 · 2022-06-11T10:08:52.853

You can zip all arrays and in each cell of rows and columns of all arrays compute count and find max and save max like below:

import numpy as np
from collections import Counter
def cell_wise_cnt(arrs):
    n_row = 0
    res = np.empty((len(arrs[0]),len(arrs[0][0])))
    for row in zip(*arrs):
        arr = np.array(row)
        num_col = len(arr[0])
        for col in range(num_col):
            res[n_row][col] = Counter(arr[:, col]).most_common()[0][0]
        n_row += 1
    return res

Output:

>>> cell_wise_cnt(arrs = (A,B,C,D))

array([[-1., -1.,  0.,  1., -1.],
       [ 1., -1.,  0., -1., -1.],
       [ 0.,  1., -1.,  1., -1.],
       [-1.,  1., -1., -1., -1.]])

Benchmark on colab:

%timeit cell_wise_cnt(arrs = (A,B,C,D))
# 136 µs per loop

% timeit scipy.stats.mode([A,B,C,D]).mode
# 585 µs per loop

%timeit stats.mode((A,B,C,D)*100_000).mode
# 1.73 s per loop

%timeit cell_wise_cnt(arrs = (A,B,C,D)*100_000)
# 2.38 s per loop

With python 3.9 and Julio_Lopes's Answer we can get a better run_time.:

import statistics
def Julio_Lopes(arrs):
    return [[statistics.mode(j)  for j in zip(*i)] for i in zip(*arrs)]

%timeit Julio_Lopes(arrs = (A,B,C,D))
# 106 µs per loop

%timeit Julio_Lopes(arrs = (A,B,C,D)*100_000)
# 653 ms per loop

That are some unfortunate benchmark results with `scipy` for small arrays. — Michael Szczesny, Jun 11 '22 at 09:43
This question turned out to be more interesting than I initially thought. — Michael Szczesny, Jun 11 '22 at 10:14

Cesar Lopes · Accepted Answer · 2022-06-11T09:24:18.003

1

Using list comprehension gets a little Hacky. Gave some work, but did it.

Basically you have to use nested child list comprehension, and the arrays must be of the same size for this to work.

To work with a matrix, it would need just 1 nested list, but as we are working with a list of matrixes, it'll be 3 dimensional, so 2 nested childs.

The import mode I used to get the most dominant value.

from statistics import mode


A = [[-1, -1, 0, 1, -1], [1, 1, 0, -1, -1], [-1, -1, -1, -1, -1], [-1, 1, -1, -1, 0]]
B = [[-1, -1, 0, 1, -1], [1, -1, 0, -1, -1], [0, 1, -1, 1, -1], [-1, 1, -1, -1, -1]]
C = [[0, -1, 0, 1, -1], [1, -1, 0, -1, -1], [0, 1, -1, 1, -1], [-1, 1, -1, -1, -1]]
D = [[-1, -1, 0, 1, 0], [0, 0, -1, 0, 1], [0, 1, -1, 1, -1], [-1, 1, -1, -1, -1]]

matrixes = [A, B, C, D]

result = [[mode([x[k][j] for x in matrixes]) for j in range(len(matrixes[0][0]))] for k in range(len([x[0][0] for x in matrixes]))]


print(result)

result:

[[-1, -1, 0, 1, -1], [1, -1, 0, -1, -1], [0, 1, -1, 1, -1], [-1, 1, -1, -1, -1]]

edited Jun 11 '22 at 09:24

answered Jun 11 '22 at 09:05

Cesar Lopes

373
1
10

Works with python 3.9, with python 3.7.13 I get a very different result: `[[ModeResult(mode=array([-1]), count=array([3])),...` – Michael Szczesny Jun 11 '22 at 09:30
you can't use 3.9? – Cesar Lopes Jun 11 '22 at 09:35
For example, Google Colab is stuck in python 3.7.x. It's just a reminder of breaking changes between python versions. – Michael Szczesny Jun 11 '22 at 09:39
I'll try to make it work for 3.7, wait a sec. – Cesar Lopes Jun 11 '22 at 09:40
1

Append `mode(...).mode[0]` for python 3.7. – Michael Szczesny Jun 11 '22 at 10:06

Hamzah · Answer 3 · 2022-06-14T09:53:10.200

0

You can simply use scipy.stats.mode:

from scipy import stats

arr = [A,B,C,D]
arr.mode(arr)[0][0].tolist()

#output
 [[-1, -1, 0, 1, -1],
 [1, -1, 0, -1, -1],
 [0, 1, -1, 1, -1],
 [-1, 1, -1, -1, -1]]

edited Jun 14 '22 at 09:53

answered Jun 11 '22 at 18:20

Hamzah

8,175
3
19
43

Find most frequent value per coordinate in multiple 2d Arrays

3 Answers3