Convert a tensor to numpy array in keras

Question

I want to convert the tensor of shape (?,224,224,3) to a numpy array in keras.

Tensor("add_1/add_3:0", shape=(?, 224, 224, 3), dtype=float32)

My basic CNN code is as below:

final_model = Model(input=[img_input], output=[act1,act2,act3,act4,act5])
addops(final_model)    

def addops(model):
        output = model.output
        factor = 2
        for i in range(len(output)):
                output[i] = UpSampling2D(size = (factor**i,factor**i),interpolation='bilinear')(output[i])
                output[i] = (ZeroPadding2D((1,1)))(output[i])
                output[i] = (Conv2D(3,(3,3)))(output[i])
        out = Add()([output[0],output[1],output[2],output[3],output[4]])
        print(out)
        sess = tf.InteractiveSession()
        outarray = tf.Variable(out).eval()
        print(outarray)

Basically, I am taking a pretrained model and capturing the feature maps of a few intermediate layers and doing those transformations on them to reshape them to the size of the input image. Later, i am adding those reshaped feature maps to a single tensor. I want to convert this tensor which is named as 'out' in the above program to a numpy array. This is part of the project I am trying to implement but I am struck here.

When I run this code, I get the below error

ValueError: initial_value must have a shape specified: Tensor("add_1/add_3:0", shape=(?, 224, 224, 3), dtype=float32)

The program gets compiled if I don't initialize the session and take just the tensor 'out' as output. I tried several other ways and have referred to various questions on Stack Overflow, but none of them gave me the required results. What is the best way to convert this tensor 'out' in the above code to a numpy array.

Note: I want to convert this numpy array to an image later. I am happy to know if there are any methods to directly convert this tensor to an image as well.

EDIT1: I passed an image to fill in the initial_value in the shape, but now I see another error which is basically an issue with converting this tensor to a numpy array.

    def addops(model,image):
        image = load_img(image,target_size=(224,224))
        image = img_to_array(image)
        image = np.expand_dims(image,axis=0)
        val =model.predict(image)
#       print(val)
        output = val
        factor = 2
        for i in range(len(output)):
                output[i] = tf.convert_to_tensor(output[i],np.float32)
                output[i] = UpSampling2D(size = (factor**i,factor**i),interpolation='bilinear')(output[i])
                output[i] = (ZeroPadding2D((1,1)))(output[i])
                output[i] = (Conv2D(3,(3,3)))(output[i])
        out = Add()([output[0],output[1],output[2],output[3],output[4]])
        print(out)
        sess = tf.InteractiveSession()
        sess.run(tf.global_variables_initializer())
        sess.run(tf.local_variables_initializer())
        outarray = tf.Variable(out).eval()
        print(outarray)

The error message is:

    Using TensorFlow backend.
WARNING:tensorflow:From /home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
2019-09-23 02:39:52.684145: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-09-23 02:39:52.690019: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3493110000 Hz
2019-09-23 02:39:52.692198: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x51c36f0 executing computations on platform Host. Devices:
2019-09-23 02:39:52.692254: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
WARNING:tensorflow:From /home/ssindhu/deeplab_env/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
/home/ssindhu/hypercolumns.py:107: UserWarning: Update your `Model` call to the Keras 2 API: `Model(inputs=[<tf.Tenso..., outputs=[<tf.Tenso...)`
  final_model = Model(input=[img_input], output=[act1,act2,act3,act4,act5])
Frontend weights loaded.
Tensor("add_1/add_3:0", shape=(1, 224, 224, 3), dtype=float32)
Traceback (most recent call last):
  File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
    return fn(*args)
  File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value Variable
         [[{{node _retval_Variable_0_0}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "training.py", line 18, in <module>
    gethypercols(model,'JPEGImages/2007_000033.jpg')
  File "/home/ssindhu/hypercolumns.py", line 34, in gethypercols
    outarray = tf.Variable(out).eval()
  File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 1695, in eval
    return self._variable.eval(session=session)
  File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 695, in eval
    return _eval_using_default_session(self, feed_dict, self.graph, session)
  File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 5181, in _eval_using_default_session
    return session.run(tensors, feed_dict)
  File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
    run_metadata)
  File "/home/ssindhu/deeplab_env/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value Variable
         [[{{node _retval_Variable_0_0}}]]

It says that I am trying to use an uninitialized value variable. I have seen that we have to initialize the variables in the session for tensorflow and so I have included those lines as well.But still I see the same error. How could I resolve this.

Answer 1

If you want to compute the value of a tensor (here out is a tensor), you just run the tensor in a session, there is no need to create a variable out of it. You can just run out in a session.

Variables are for storing trainable (and other) parameters of your model. What you do is that you create a new variable that should be initialized with out and run it without initialization.