Cannot unflat numpy arrays properly

Question

I am triying to 'unflat' numpy arrays for a tensorflow topic. I need to take a NxN matrix, for example 27x27, and then, row by row take all the row elements (27 each time) and reshape this to 3x3x3 maps (with the 27 columns I will get 3x3x3 x 27maps), and I did the following function:

def unflat_pca(flated_patches, depth=3, verbose=False):
    # tensor with shape [components_width, components_height]
    p_width = flated_patches.shape[0]
    p_height = flated_patches.shape[1]

    # Utilizo 3x3 por la ventana de la convolucion
    res = np.empty((3,3, depth, p_width))

    for one_map in range(p_width):

        map_unflat = np.empty((3,3, depth))

        current_indx = 0
        for d in range(depth):
            # flated_patches matriz cuadrada de pca (PxP)
            map_unflat[:,:,d] = flated_patches[one_map, current_indx:(current_indx+(3*3))].reshape(3,3)
            current_indx += 3*3
            res[:,:, d, one_map] = map_unflat[:,:,d]

    if verbose:
        print("\n-- unflat_pca function --")
        print("The initial shape was: " + str(flated_patches.shape))
        print("The output shape is: " + str(res.shape) + "\n")

    return res #[width, height, depth, pca_maps]

Then when I try to test the function, I pass an easy to follow array (0,1,2...) to try to observe if the function works properly...

utest =  unflat_pca(np.arange(0, 27*27).reshape(27,27), verbose=True)

And I get

-- unflat_pca function -- The initial shape was: (27, 27) The output shape is: (3, 3, 3, 27)

Perfect! But now, when I inspect the result, for example with utest[:,:,:,0], I expect that all the numbers in the same array go as 1,2,3.... but got

 array([[[  0.,   9.,  18.],
        [  1.,  10.,  19.],
        [  2.,  11.,  20.]],

       [[  3.,  12.,  21.],
        [  4.,  13.,  22.],
        [  5.,  14.,  23.]],

       [[  6.,  15.,  24.],
        [  7.,  16.,  25.],
        [  8.,  17.,  26.]]])

But If I only inspect the first channel I got what I expected.

> array([[ 0.,  1.,  2.],
       [ 3.,  4.,  5.],
       [ 6.,  7.,  8.]])

I am confused because later I use the unflated maps and I am getting bad results, I think that its due to the first result where I obtain the numbers not correctly (by columns?!). Could you help me? Sorry about my english :P

PS: Expected Value of utest[:,:,:,0] -> the 3x3x3 maps ordered (width, height, depth):

 array([[[  0.,   1.,  2.],
        [  3.,  4.,  5.],
        [  6.,  7.,  8.]],

       [[  9.,  10.,  11.],
        [  12.,  13.,  14.],
        [  15.,  16.,  17.]],

       [[  18.,  19.,  20.],
        [  21.,  22.,  23.],
        [  24.,  25.,  26.]]])

PS2: Example for first row via paper: First row result

can you provide the example how `utest` must look, at least `utest[:,:,:,:3]` — aiven, Jan 09 '18 at 19:05

score 2 · Accepted Answer · answered Jan 09 '18 at 20:09

2

Reshape & permute axes -

a.reshape(-1,3,3,3).transpose(1,2,3,0)

Sample run -

In [482]: a = np.arange(27*27).reshape(27,27)

In [483]: out = a.reshape(-1,3,3,3).transpose(1,2,3,0)

# Verify output shape
In [484]: out.shape
Out[484]: (3, 3, 3, 27)

# Verify output values for the first slice
In [485]: out[...,0]
Out[485]: 
array([[[ 0,  1,  2],
        [ 3,  4,  5],
        [ 6,  7,  8]],

       [[ 9, 10, 11],
        [12, 13, 14],
        [15, 16, 17]],

       [[18, 19, 20],
        [21, 22, 23],
        [24, 25, 26]]])

answered Jan 09 '18 at 20:09

Divakar

218,885
19
262
358

I think its perfect :D and I will not need do so many things... But I don't understand one thing: Why utest[:,:,0,0] returns this array([[ 0, 3, 6], [ 9, 12, 15], [18, 21, 24]]) and not this array([[[ 0, 1, 2], [ 3, 4, 5], [ 6, 7, 8]]) – Mario Parreño Jan 10 '18 at 09:41
@MarioParreño You might be interpreting the axes wrongly. See here - https://stackoverflow.com/a/41507480/3293881. Basically, with `utest[:,:,0,0] `, you are extracting the first row off each 2D slice from your expected 3D array, the one you posted in your question. – Divakar Jan 10 '18 at 09:52
I saw your post, and due that my results are the same I want question you the last one. My final purpouse its to do PCA to a flatened array and get a 27*27 matrix (this works well), and then get row by row (27 elements), and unflat them to build a array with the shape [filter_height, filter_width, in_channels, out_channels] (the same as the parameter 'filter' in https://www.tensorflow.org/api_docs/python/tf/nn/conv2d), but I think that I am not doing well this unflat -> Your unflat flated_patches.reshape(-1,w_patch,h_patch,depth).transpose(1,2,3,0) – Mario Parreño Jan 10 '18 at 16:32
@MarioParreño Not clear about the format. But I would think play around with the first three numbers in `transpose(1,2,3,0)`, i.e. try `(1,3,2,0)` and so on and see if one of those work. – Divakar Jan 10 '18 at 19:36

Cannot unflat numpy arrays properly

1 Answers1