Form a big 2d array from multiple smaller 2d arrays

Question

The question is the inverse of this question. I'm looking for a generic method to from the original big array from small arrays:

array([[[ 0,  1,  2],
        [ 6,  7,  8]],    
       [[ 3,  4,  5],
        [ 9, 10, 11]], 
       [[12, 13, 14],
        [18, 19, 20]],    
       [[15, 16, 17],
        [21, 22, 23]]])

->

array([[ 0,  1,  2,  3,  4,  5],
       [ 6,  7,  8,  9, 10, 11],
       [12, 13, 14, 15, 16, 17],
       [18, 19, 20, 21, 22, 23]])

I am currently developing a solution, will post it when it's done, would however like to see other (better) ways.

score 39 · Accepted Answer · edited May 23 '17 at 12:32

39

import numpy as np
def blockshaped(arr, nrows, ncols):
    """
    Return an array of shape (n, nrows, ncols) where
    n * nrows * ncols = arr.size

    If arr is a 2D array, the returned array looks like n subblocks with
    each subblock preserving the "physical" layout of arr.
    """
    h, w = arr.shape
    return (arr.reshape(h//nrows, nrows, -1, ncols)
               .swapaxes(1,2)
               .reshape(-1, nrows, ncols))


def unblockshaped(arr, h, w):
    """
    Return an array of shape (h, w) where
    h * w = arr.size

    If arr is of shape (n, nrows, ncols), n sublocks of shape (nrows, ncols),
    then the returned array preserves the "physical" layout of the sublocks.
    """
    n, nrows, ncols = arr.shape
    return (arr.reshape(h//nrows, -1, nrows, ncols)
               .swapaxes(1,2)
               .reshape(h, w))

For example,

c = np.arange(24).reshape((4,6))
print(c)
# [[ 0  1  2  3  4  5]
#  [ 6  7  8  9 10 11]
#  [12 13 14 15 16 17]
#  [18 19 20 21 22 23]]

print(blockshaped(c, 2, 3))
# [[[ 0  1  2]
#   [ 6  7  8]]

#  [[ 3  4  5]
#   [ 9 10 11]]

#  [[12 13 14]
#   [18 19 20]]

#  [[15 16 17]
#   [21 22 23]]]

print(unblockshaped(blockshaped(c, 2, 3), 4, 6))
# [[ 0  1  2  3  4  5]
#  [ 6  7  8  9 10 11]
#  [12 13 14 15 16 17]
#  [18 19 20 21 22 23]]

Note that there is also superbatfish's blockwise_view. It arranges the blocks in a different format (using more axes) but it has the advantage of (1) always returning a view and (2) being capable of handing arrays of any dimension.

edited May 23 '17 at 12:32

Community

1
1

answered Jun 01 '13 at 14:32

unutbu

842,883
184
1,785
1,677

It doesn't work for `c = np.arange(24).reshape((6,4))` `print(unblockshaped(blockshaped(a, 3, 2), 6, 4))` – TheMeaningfulEngineer Jun 02 '13 at 09:18
`blockshaped` returns as expected. The problem is in `unblockshaped` – TheMeaningfulEngineer Jun 02 '13 at 09:22
1

Yes, I had the order of arguments to `reshape` wrong. Try it now. – unutbu Jun 02 '13 at 10:35
Major problem is when `n * nrows * ncols = arr.size` does not hold. For example, for a prime like `size=17` or `size=47`, you can not use this method. – Sudipta Basak Jun 28 '16 at 09:43
@SudiptaBasak: It is not a problem because there are no 2D arrays, `arr`, with `arr.size` equal to a prime number. – unutbu Jun 28 '16 at 09:44
@unutbu you are correct. Should have said that in case nrows/ncols is a prime, this does not work. For example `c=np.arange(42).reshape(6,7)` only has trivial solutions if I wanted to break the columns. – Sudipta Basak Jun 28 '16 at 09:55
@SudiptaBasak: `blockshaped` works as advertised. It is not `blockshaped`'s fault that prime numbers only have trivial factors :). Perhaps you are looking for something like `np.array_split(c, 3, axis=1)`. Notice that the result is a **list of arrays**, not a single array, however. There is no single array equivalent since NumPy arrays can not have rows (or columns) of uneven length. – unutbu Jun 28 '16 at 10:20
@unutbu Well the OP won't be able to turn big columns/rows into smaller columns/rows if that's really required. I came across this as I was looking to tile a big matrix and I need a solution for any combination that the user might throw in. See my post for a solution. – Sudipta Basak Jun 28 '16 at 10:52

score 4 · Answer 2 · answered Jun 27 '16 at 00:04

4

Yet another (simple) approach:

threedarray = ...
twodarray = np.array(map(lambda x: x.flatten(), threedarray))
print(twodarray.shape)

answered Jun 27 '16 at 00:04

Thamme Gowda

11,249
5
50
57

score 1 · Answer 3 · answered Jun 01 '13 at 14:15

1

I hope I get you right, let's say we have a,b :

>>> a = np.array([[1,2] ,[3,4]])
>>> b = np.array([[5,6] ,[7,8]])
    >>> a
    array([[1, 2],
           [3, 4]])
    >>> b
    array([[5, 6],
           [7, 8]])

in order to make it one big 2d array use numpy.concatenate:

>>> c = np.concatenate((a,b), axis=1 )
>>> c
array([[1, 2, 5, 6],
       [3, 4, 7, 8]])

answered Jun 01 '13 at 14:15

0x90

39,472
36
165
245

What I'm searching for i a method which can rebuild a big array (image) from smaller arrays. I need it to be generic so it can be applied to images of different sizes. Like slicing in jpeg. Slice an image to 8×8 blocks, do operations on each block, rebuild the original image from blocks. – TheMeaningfulEngineer Jun 02 '13 at 09:27

score 0 · Answer 4 · answered Jun 02 '13 at 09:54

It works for the images I tested for now. Will if further tests are made. It is however a solution which takes no account about speed and memory usage.

def unblockshaped(blocks, h, w):
    n, nrows, ncols = blocks.shape
    bpc = w/ncols
    bpr = h/nrows

    reconstructed = zeros((h,w))
    t = 0
    for i in arange(bpr):
        for j in arange(bpc):
            reconstructed[i*nrows:i*nrows+nrows,j*ncols:j*ncols+ncols] = blocks[t]
            t = t+1
    return reconstructed

Sudipta Basak · Answer 5 · 2016-06-28T23:42:27.993

Here is a solution that one can use if someone is wishing to create tiles of a matrix:

from itertools import product
import numpy as np
def tiles(arr, nrows, ncols):
    """
    If arr is a 2D array, the returned list contains nrowsXncols numpy arrays
    with each array preserving the "physical" layout of arr.

    When the array shape (rows, cols) are not divisible by (nrows, ncols) then
    some of the array dimensions can change according to numpy.array_split.

    """
    rows, cols = arr.shape
    col_arr = np.array_split(range(cols), ncols)
    row_arr = np.array_split(range(rows), nrows)
    return [arr[r[0]: r[-1]+1, c[0]: c[-1]+1]
                     for r, c in product(row_arr, col_arr)]

Form a big 2d array from multiple smaller 2d arrays

5 Answers5

Linked