Combination of all rows in two 4D numpy arrays

Question

I have two numpy arrays named my_arr1, my_arr2 both of them of size 100x30x30x3. I would like to output the combination of each row of the first array with each row from the second array and result to two new arrays with size 10000x30x30x3. I managed to do so with a simpler example with two array of size 4x2 by following that example:

a1_ = np.array([[1, 1],
           [2, 2],
           [3, 0],
           [4, 2],
           [5, 3]])
a2_ = np.array([[10, 1],
           [11, 2],
           [12, 0],
           [13, 2],
           [14, 3]])


def combine_matrices(a1, a2):

   m1, n1 = a1.shape
   m2, n2 = a2.shape
   out = np.zeros((m1, m2, n1+n2), dtype=int)
   out[:, :, :n1] = a1[:, None, :]
   out[:, :, n1:] = a2
   out.shape = (m1*m2, -1)

 return out

The reult is a 25x4 array which can be split to two arrays 25x2. How can I modify my function in case that a1 = np.zeros((100, 30, 30, 3)) and a2 = np.zeros((100. 30, 30, 3)) with final scope to return two arrays with size (10000, 30, 30 ,3) and not just one.

score 1 · Accepted Answer · answered Oct 16 '18 at 16:17

You could get this done with numpy.outer

You can't just use the function though, and there may be something better, but I think you have to iterate over your other matrix indecies.

# Use transpose to make the empty array shape (10x30x30x1000)
out_arr = numpy.zeros(shape=(1000, 30, 30, 10)).transpose()

for i in range(out_arr.shape[0]):
   for j in range(out_arr.dtype[1]):
      for k in range(out_arr.dtype[2]):
          # Use transpose again to get the data you want 
          # (transposing first before grabbing the index) 
          out_arr[i][j][k] = numpy.outer(arr1.transpose()[i][j][k], arr2.transpose()[i][j][k])

# 1000x30x30x10 array (rather than 10x30x30x1000)
out_arr = out_arr.transpose()

I am using the transposes as a trick because you said the data is (1000, 30, 30, 10).

One thing that trips me up working with this is making the mistake of trying to do:

arr[:][i][j][k]

because that will not grab the memory you would want in this case. It grab the same data as:

arr[i][j][k]

My arrays size is (100x30x30x3) and finally i want one array (10000x30x30x6) or two of 10000x30x30x3 not (1000, 30, 30, 10) — Jose Ramon, Oct 17 '18 at 08:12
Maybe the np.tile could provide me a solution to it. I can replicate 100 times each item of a1 and match it with a2 100 times. — Jose Ramon, Oct 17 '18 at 08:59

score 1 · Answer 2 · answered Oct 16 '18 at 16:40

Not 100% sure this is what you want but it gets the shapes right.

First build diagnostic inputs

>>> a1 = np.empty((2, 2, 2, 3, 4), 'U1')
>>> for i, x in enumerate(np.ogrid[:2, :2, :2, :3]):
...      a1[..., i] = x
... 
>>> a2 = a1 = a1.view('U4').reshape(a1.shape[:-1])
>>> a1
array([[[['0000', '0001', '0002'],
         ['0010', '0011', '0012']],

        [['0100', '0101', '0102'],
         ['0110', '0111', '0112']]],


       [[['1000', '1001', '1002'],
         ['1010', '1011', '1012']],

        [['1100', '1101', '1102'],
         ['1110', '1111', '1112']]]], dtype='<U4')

Next, allocate output

>>> A1, A2 = (np.empty((a.shape[0], *a.shape), a.dtype) for a in (a1, a2))

Fill in using broadcasting

>>> A1[...] = a1[:, None]
>>> A2[...] = a2[None]

Merge first two axes

>>> A1, A2 = (A.reshape(-1, *A.shape[2:]) for A in (A1, A2))

Done

>>> A1
array([[[['0000', '0001', '0002'],
         ['0010', '0011', '0012']],

        [['0100', '0101', '0102'],
         ['0110', '0111', '0112']]],


       [[['0000', '0001', '0002'],
         ['0010', '0011', '0012']],

        [['0100', '0101', '0102'],
         ['0110', '0111', '0112']]],


       [[['1000', '1001', '1002'],
         ['1010', '1011', '1012']],

        [['1100', '1101', '1102'],
         ['1110', '1111', '1112']]],


       [[['1000', '1001', '1002'],
         ['1010', '1011', '1012']],

        [['1100', '1101', '1102'],
         ['1110', '1111', '1112']]]], dtype='<U4')
>>> A2
array([[[['0000', '0001', '0002'],
         ['0010', '0011', '0012']],

        [['0100', '0101', '0102'],
         ['0110', '0111', '0112']]],


       [[['1000', '1001', '1002'],
         ['1010', '1011', '1012']],

        [['1100', '1101', '1102'],
         ['1110', '1111', '1112']]],


       [[['0000', '0001', '0002'],
         ['0010', '0011', '0012']],

        [['0100', '0101', '0102'],
         ['0110', '0111', '0112']]],


       [[['1000', '1001', '1002'],
         ['1010', '1011', '1012']],

        [['1100', '1101', '1102'],
         ['1110', '1111', '1112']]]], dtype='<U4')

Combination of all rows in two 4D numpy arrays

2 Answers2