NumPy: 2-D array tiled, converted to a 1-D array, and have to become a 2-D array again

Question

Ok, I am no an expert in numpy, so sorry if the answer is obvious, but this has been bugging me off for a few days now, so I have no other option than asking here. So, here is the input array I have:

a = np.array([
    [0, 0, 1, 3, 4,  5,  12, 0, 0,  0, 0,  0  ],
    [0, 0, 4, 0, 13, 0,  0,  2, 0,  0, 0,  0  ],
    [1, 2, 3, 4, 5,  6,  7,  8, 0,  0, 0,  0  ],
    [5, 4, 9, 0, 3,  0,  7,  2, 0,  0, 0,  0  ],
    [0, 0, 0, 0, 0,  0,  0,  0, 0,  0, 0,  0  ],
    [0, 0, 0, 0, 1,  0,  5,  7, 5,  0, 1,  0  ],
    [0, 0, 0, 0, 0,  5,  12, 3, 0,  4, 12, 3  ],
    [0, 0, 0, 0, 5,  14, 0,  9, 10, 2, 0,  15 ]
])

It needs to be split into tiles with a size of 4x4 (which means 16 elements per tile, you'll see why this is important). I tile it up (using Iosif Doundoulakis's np.reshape() method, explained here, big shoutout):

def tiling(arr):
    # 16 - total number of elements getting into a tile
    # 4 - width of a tile
    # 4 - height of a tile
    b = arr.reshape(arr.shape[0] // 4, 4, arr.shape[1] // 4, 4, 1)
    return b.swapaxes(1, 2)

... and, when I call tiles = tiling(a), I get a similar result:

*I've formatted the output for easier reading, the actual output looks different, but it is organised the same way.

[[
 [
  [[ 0] [ 0] [ 1] [ 3]]
  [[ 0] [ 0] [ 4] [ 0]]
  [[ 1] [ 2] [ 3] [ 4]]
  [[ 5] [ 4] [ 9] [ 0]]
 ]
.... this is one tile, there are 5 more ...
]]

which is exactly what I want my tiles to look like. Then, I flatten the tiled array, so it becomes

[ 0  0  1  3  0  0  4  0  1  2  3  4  5  4  9  0  4  5 12  0 13  0  0  2
  5  6  7  8  3  0  7  2  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0
  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  1  0  5  7
  0  5 12  3  5 14  0  9  0  0  0  0  5  0  1  0  0  4 12  3 10  2  0 15]

and every 16 numbers represent a tile. The next step is to pass the flatten array to an external program which returns an array with the same shape - 1 dimensional array. For now, the data is only passed to the external utility and returned by it, so the array keeps it's values.

Knowing the total number of array elements that go into a tile (16), as well as the shape of a tile (4, 4), how can I turn this 1-D array back into tiles and then create a 2-D array from those tiles, which looks like the one from the beginning?

EDIT: I was out for a few days, sorry for the delay! The thing is that I have a flat 1-D array as a result of the external utility which I want to reshape into a form, I get from the original array, so something like:

    arr (with a shape (8, 12, 1))
        |
        |  tile the array (using Iosif 
        |  Doundoulakis's method)
        V 
    tiled_arr = arr.reshape(2, 3, 4, 4, 1)
        |
        | flatten the tiled array
        V 
    tiled_arr.flatten('C')
        |
        | pass to the external utility
        V 
    it returns the same flat array, for now, but it wouldn't in the nearest future, so reformatting tiled_array is not an option
        |
        | pass it to a reshaping function in question
        V 
    It should reshape the flat array back into (8, 12, 1), which is the shape of the original array arr

I came up with this code yesterday:

def reshape(flat_array, original_array):

    a = np.array([np.split(flat_array, 16)]).reshape(original_array.shape[1] // 4, 4, original_array.shape[0] // 4, 4, original_array.shape[2])

    b = a.reshape(2, 3, 4, 4)
    return b.swapaxes(1, 2).reshape(original_array.shape)

... and it works, I get the result I want to. But it seems to me, it could be at least optimised a little bit.

I'm not able to run your `tiling` function. Besides `frame` being not defined, it raises `ValueError: cannot reshape array of size 96 into shape (0,16,3,4,1)` — , Jun 05 '22 at 18:27
I fixed it, should be fine now. Due to copying code and hardcoding some variable values, I messed up the function, but it's fixed now, try again. Thanks for pointing this out and not letting it just sink in :) — Emil Avramov, Jun 05 '22 at 19:14

score 0 · Answer 1 · answered Jun 05 '22 at 19:28

I think this is actually as simple as another swapaxes and reshape. First, you'll need to call swapaxes(1, 2) again to undo the last call. Then just reshape back to a's shape.

>>> tiles
array([[[[[ 0],
          [ 0],
          [ 1],
          [ 3]],

         [[ 0],
          [ 0],
          [ 4],
          [ 0]],
          ...
          ]]])

>>> tiles.swapaxes(1, 2).reshape(a.shape)
array([[ 0,  0,  1,  3,  4,  5, 12,  0,  0,  0,  0,  0],
       [ 0,  0,  4,  0, 13,  0,  0,  2,  0,  0,  0,  0],
       [ 1,  2,  3,  4,  5,  6,  7,  8,  0,  0,  0,  0],
       [ 5,  4,  9,  0,  3,  0,  7,  2,  0,  0,  0,  0],
       [ 0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0],
       [ 0,  0,  0,  0,  1,  0,  5,  7,  5,  0,  1,  0],
       [ 0,  0,  0,  0,  0,  5, 12,  3,  0,  4, 12,  3],
       [ 0,  0,  0,  0,  5, 14,  0,  9, 10,  2,  0, 15]])

If you don't have a.shape, you can probably calculate it with something like (S[0]*S[2], S[1]*S[3]), where S is tiles.shape:

tiles.swapaxes(1, 2).reshape(x.shape[0] * x.shape[2], x.shape[1] * x.shape[3])

Ali_Sh · Answer 2 · 2022-06-06T22:51:40.117

I could not understand what you want exactly, but I guess you are searching for something like my previous answer (which get shape = (6, 4 ,4) for your example):

a.reshape(a.shape[0] // 4, 4, -1, 4).swapaxes(1, 2).reshape(a.size // (4 * 4), 4, 4)

You need to reshape the return array to a.size // (4 * 4), 4, 4) (if you sure about the shapes that can be divided by 4 or 16 …), so the tiling function must change to:

def tiling(arr):
    b = arr.reshape(arr.shape[0] // 4, 4, arr.shape[1] // 4, 4, 1)
    return b.swapaxes(1, 2).reshape(arr.size // (4 * 4), 4, 4)

# [[[ 0  0  1  3]
#   [ 0  0  4  0]
#   [ 1  2  3  4]
#   [ 5  4  9  0]]
# 
#  [[ 4  5 12  0]
#   [13  0  0  2]
#   [ 5  6  7  8]
#   [ 3  0  7  2]]
# 
#  [[ 0  0  0  0]
#   [ 0  0  0  0]
#   [ 0  0  0  0]
#   [ 0  0  0  0]]
# 
#  [[ 0  0  0  0]
#   [ 0  0  0  0]
#   [ 0  0  0  0]
#   [ 0  0  0  0]]
# 
#  [[ 0  0  0  0]
#   [ 1  0  5  7]
#   [ 0  5 12  3]
#   [ 5 14  0  9]]
# 
#  [[ 0  0  0  0]
#   [ 5  0  1  0]
#   [ 0  4 12  3]
#   [10  2  0 15]]]

score 0 · Answer 3 · answered Jun 06 '22 at 02:50

One of the benefits of Iosif Doundoulakis' approach is that the tiled array and the original array share memory.

original_array = np.array([
    [0, 0, 1, 3, 4, 5, 12, 0, 0, 0, 0, 0],
    [0, 0, 4, 0, 13, 0, 0, 2, 0, 0, 0, 0],
    [1, 2, 3, 4, 5, 6, 7, 8, 0, 0, 0, 0],
    [5, 4, 9, 0, 3, 0, 7, 2, 0, 0, 0, 0],
    [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
    [0, 0, 0, 0, 1, 0, 5, 7, 5, 0, 1, 0],
    [0, 0, 0, 0, 0, 5, 12, 3, 0, 4, 12, 3],
    [0, 0, 0, 0, 5, 14, 0, 9, 10, 2, 0, 15],
])

tile_rows, tile_cols = tile_shape = (2, 4)

grid_rows, grid_cols = grid_shape = (original_array.shape[0] // tile_shape[0],
                                     original_array.shape[1] // tile_shape[1])

tiled_array = original_array \
    .reshape((grid_rows, tile_rows, grid_cols, tile_cols, 1)) \
    .swapaxes(1, 2)

print(np.shares_memory(tiled_array, original_array))  # True

tiled_array is a view of original_array; any in-place modifications of tiled_array also apply to original_array:

tiled_array[0, 0] = 99
print(original_array)

prints

[[99 99 99 99  4  5 12  0  0  0  0  0]
 [99 99 99 99 13  0  0  2  0  0  0  0]
 [ 1  2  3  4  5  6  7  8  0  0  0  0]
 [ 5  4  9  0  3  0  7  2  0  0  0  0]
 [ 0  0  0  0  0  0  0  0  0  0  0  0]
 [ 0  0  0  0  1  0  5  7  5  0  1  0]
 [ 0  0  0  0  0  5 12  3  0  4 12  3]
 [ 0  0  0  0  5 14  0  9 10  2  0 15]]

As others have pointed out, the transformation can be reversed by swapping the axes back and reshaping,

recovered_original_array = tiled_array \
    .swapaxes(1, 2) \
    .reshape((grid_rows * tile_rows, grid_cols * tile_cols))

print(np.all(recovered_original_array == original_array))  # True

The three arrays all share memory.

print(np.shares_memory(recovered_original_array, original_array)) # True

However, this useful property is destroyed when you flatten the array with the flatten method, because flatten always returns a copy rather than a view,

flat_array = tiled_array.flatten()
print(np.shares_memory(flat_array, original_array))  # False

The reshape method, called with -1 as its argument, will return a flattened view if possible, but will otherwise return a copy. In this case, it returns a copy.

flat_array = tiled_array.reshape(-1)
print(np.shares_memory(flat_array, original_array))  # False

Either way, you can easily recover tiled_array from flat_array,

recovered_tiled_array = flat_array.reshape((grid_rows, grid_cols, tile_rows, tile_cols, 1))
print(np.shares_memory(recovered_tiled_array, tiled_array))  # False
print(np.all(recovered_tiled_array == tiled_array))  # True

As far as I know, there is no way to make a flat array view of the original array with the order you require. You can, however, make a flat view of the tiled array, and work on that. To do so, make a copy of tiled_array

tiled_array = original_array \
    .reshape((grid_rows, tile_rows, grid_cols, tile_cols, 1)) \
    .swapaxes(1, 2).copy()

print(np.shares_memory(tiled_array, original_array))  # False

allowing a flat view of the same,

flat_array = tiled_array.reshape(-1)

print(np.shares_memory(flat_array, tiled_array))  # True

flat_array[:] = function_on_flat_array(flat_array)  # changes apply automatically to flat_array

NumPy: 2-D array tiled, converted to a 1-D array, and have to become a 2-D array again

3 Answers3