1

Following up on this answer by jorgeca:

def patchify(img, patch_shape):
    img = np.ascontiguousarray(img)  # won't make a copy if not needed
    X, Y = img.shape
    x, y = patch_shape
    shape = ((X-x+1), (Y-y+1), x, y) # number of patches, patch_shape
    # The right strides can be thought by:
    # 1) Thinking of `img` as a chunk of memory in C order
    # 2) Asking how many items through that chunk of memory are needed when indices
    #    i,j,k,l are incremented by one
    strides = img.itemsize*np.array([Y, 1, Y, 1])
    return np.lib.stride_tricks.as_strided(img, shape=shape, strides=strides)

How can those overlapping arrays be merged back again to the original image?

Community
  • 1
  • 1
Durin
  • 15
  • 5

1 Answers1

0

Approach #1

Here's one approach after converting the 4D array of patches into 2D and then simply slicing and stacking the leftover rows and columns -

def unpatchify(img_patches, block_size):
    B0, B1 = block_size
    N = np.prod(img_patches.shape[1::2])
    patches2D = img_patches.transpose(0,2,1,3).reshape(-1,N)

    m,n = patches2D.shape
    row_mask = np.zeros(m,dtype=bool)
    col_mask = np.zeros(n,dtype=bool)
    row_mask[::B0]= 1
    col_mask[::B1]= 1
    row_mask[-B0:] = 1
    col_mask[-B1:] = 1
    return patches2D[np.ix_(row_mask, col_mask)]

Sample run -

In [233]: img = np.random.randint(0,255,(16,25))
     ...: block_size = (4,8)
     ...: 

In [234]: np.allclose(img, unpatchify(patchify(img, block_size), block_size))
Out[234]: True

Approach #2

In the previous approach, use of transpose on the big 4D array would force a copy and as such that transpose operation might prove costly. To avoid that, here's another approach making heavy usage of slicing -

def unpatchify_v2(img_patches, block_size):
    B0, B1 = block_size
    m,n,r,q = img_patches.shape
    shp = m + r - 1, n + q - 1

    p1 = img_patches[::B0,::B1].swapaxes(1,2)
    p1 = p1.reshape(-1,p1.shape[2]*p1.shape[3])
    p2 = img_patches[:,-1,0,:]
    p3 = img_patches[-1,:,:,0].T
    p4 = img_patches[-1,-1]

    out = np.zeros(shp,dtype=img_patches.dtype)
    out[:p1.shape[0],:p1.shape[1]] = p1
    out[:p2.shape[0],-p2.shape[1]:] = p2
    out[-p3.shape[0]:,:p3.shape[1]] = p3
    out[-p4.shape[0]:,-p4.shape[1]:] = p4
    return out

Runtime test

In [16]: img = np.random.randint(0,255,(1024,1024))
    ...: block_size = (3,3)
    ...: img_patches = patchify(img, block_size)
    ...: 

In [17]: %timeit unpatchify(img_patches, block_size)
    ...: %timeit unpatchify_v2(img_patches, block_size)

10 loops, best of 3: 22.9 ms per loop
100 loops, best of 3: 2.25 ms per loop

In [18]: img = np.random.randint(0,255,(1024,1024))
    ...: block_size = (8,8)
    ...: img_patches = patchify(img, block_size)
    ...: 

In [19]: %timeit unpatchify(img_patches, block_size)
    ...: %timeit unpatchify_v2(img_patches, block_size)
    ...: 
10 loops, best of 3: 114 ms per loop
1000 loops, best of 3: 1.5 ms per loop
Divakar
  • 218,885
  • 19
  • 262
  • 358