ValueError: total size of new array must be unchanged (Reshape from keras)

Question

For this toy model:

from keras.layers import Input, Dense, Reshape
from keras.models import Model

# this is the size of our encoded representations
compression = 10  

input_img = Input(shape=(28,28, ), name= "28x28")
encoded = Dense(int(np.floor(28*28/compression)), activation='relu', 
                name= "encoder_" + str(compression))(input_img)
decoded = Dense(units = 28*28, activation='sigmoid',
                name = "28.28_decoder")(encoded)
reshape = Reshape(target_shape = (28,28), name = "28x28_reshape")(decoded)
autoencoder = Model(input_img, reshape)

I get the error:

ValueError                                Traceback (most recent call last)
<ipython-input-27-04b835543369> in <module>
     12 decoded = Dense(units = 28*28, activation='sigmoid',
     13                 name = "28.28_decoder")(encoded)
---> 14 reshape = Reshape(target_shape = (28,28), name = "28x28_reshape")(decoded)
     15 autoencoder = Model(input_img, reshape)
     16 

~/.local/lib/python3.6/site-packages/keras/engine/base_layer.py in __call__(self, inputs, **kwargs)
    472             if all([s is not None
    473                     for s in to_list(input_shape)]):
--> 474                 output_shape = self.compute_output_shape(input_shape)
    475             else:
    476                 if isinstance(input_shape, list):

~/.local/lib/python3.6/site-packages/keras/layers/core.py in compute_output_shape(self, input_shape)
    396             # input shape known? then we can compute the output shape
    397             return (input_shape[0],) + self._fix_unknown_dimension(
--> 398                 input_shape[1:], self.target_shape)
    399 
    400     def call(self, inputs):

~/.local/lib/python3.6/site-packages/keras/layers/core.py in _fix_unknown_dimension(self, input_shape, output_shape)
    384             output_shape[unknown] = original // known
    385         elif original != known:
--> 386             raise ValueError(msg)
    387 
    388         return tuple(output_shape)

ValueError: total size of new array must be unchanged

I have been trying to figure out why but I do not get it. Looking at the help page it rather straightforward, since the previous layer is the adequate shape to perform the reshape.

In the documentation for `Dense` it says `Output shape: nD tensor with shape: (batch_size, ..., units). For instance, for a 2D input with shape (batch_size, input_dim), the output would have shape (batch_size, units).` — Ardweaden, Aug 08 '19 at 11:03
@Ardweaden But the documentation says *target_shape: Target shape. Tuple of integers, does not include the samples dimension (batch size).* — D1X, Aug 08 '19 at 11:05
[The `Dense` layer is applied on the last dimension](https://stackoverflow.com/a/52092176/2099607). Therefore, `encoded` would have a shape of `(None, 28, 78)` and the `decoded` would have a shape of `(None, 28, 784)`. Therefore you cannot resize a tensor of shape `(28,784)` into `(28,28)`. One way to solve this is to use `Flatten` layer right after the input layer. — today, Aug 08 '19 at 16:57

score 0 · Answer 1 · answered Aug 08 '19 at 11:19

0

Your decoded dense layer has 28*28 units, this will be make its output dim to be (,28,784) which won't reshape to 28*28.

answered Aug 08 '19 at 11:19

Arsal

447
2
6

*For instance, for a 2D input with shape (batch_size, input_dim), the output would have shape (batch_size, units)* - From the Dense layer help page. I do not get it. Could you elaborate a little bit? – D1X Aug 08 '19 at 11:24
Only the last dimension contributes to the size of the weights which is 28 in your case (https://github.com/keras-team/keras/blob/88af7d0c97497b5c3a198ee9416b2accfbc72c36/keras/layers/core.py#L880). Your decoded layer input shape is (None, 28, 78), replacing the last dim with 28*28, your decoded layer output dim turns out to be (None, 28, 784). – Arsal Aug 08 '19 at 11:50

score 0 · Answer 2 · answered Aug 08 '19 at 11:20

Based on what you want to do, you are supposed to flatten the array beforehand. This should work:

flatten = Flatten()(input_img)
encoded = Dense(int(np.floor(28*28/compression)), activation='relu',  name= "encoder_" + str(compression))(flatten)
decoded = Dense(units = 28*28, activation='sigmoid', name = "28.28_decoder" (encoded)
reshape = Reshape(target_shape = (28,28), name = "28x28_reshape")(decoded)
autoencoder = Model(input_img, reshape)

I assume that the initial reshape didn't work because you are passing 3D tensor through a dense layer, and the initial dimension gets messed up. Therefore, you can get it back into the initial shape.

ValueError: total size of new array must be unchanged (Reshape from keras)

2 Answers2