Implement ConvND in Tensorflow

Question

So I need a ND convolutional layer that also supports complex numbers. So I decided to code it myself.

I tested this code on numpy alone and it worked. Tested with several channels, 2D and 1D and complex. However, I have problems when I do it on TF.

This is my code so far:

def call(self, inputs):
    with tf.name_scope("ComplexConvolution_" + str(self.layer_number)) as scope:
        inputs = self._verify_inputs(inputs)            # Check inputs are of expected shape and format
        inputs = self.apply_padding(inputs)             # Add zeros if needed
        output_np = np.zeros(                           # I use np because tf does not support the assigment
            (inputs.shape[0],) +                        # Per each image
            self.output_size,                           # Image out size
            dtype=self.input_dtype                      # To support complex numbers
        )
        img_index = 0
        for image in inputs:
            for filter_index in range(self.filters):
                for i in range(int(np.prod(self.output_size[:-1]))):  # for each element in the output
                    index = np.unravel_index(i, self.output_size[:-1])
                    start_index = tuple([a * b for a, b in zip(index, self.stride_shape)])
                    end_index = tuple([a+b for a, b in zip(start_index, self.kernel_shape)])
                    # set_trace()
                    sector_slice = tuple(
                        [slice(start_index[ind], end_index[ind]) for ind in range(len(start_index))]
                    )
                    sector = image[sector_slice]
                    new_value = tf.reduce_sum(sector * self.kernels[filter_index]) + self.bias[filter_index]
                    # I use Tied Bias https://datascience.stackexchange.com/a/37748/75968
                    output_np[img_index][index][filter_index] = new_value  # The complicated line
                    img_index += 1
        output = apply_activation(self.activation, output_np)
    return output

input_size is a tuple of shape (dim1, dim2, ..., dim3, channels). An 2D rgb conv for example will be (32, 32, 3) and inputs will have shape (None, 32, 32, 3).

The output size is calculated from an equation I found in this paper: A guide to convolution arithmetic for deep learning

out_list = []
for i in range(len(self.input_size) - 1):   # -1 because the number of input channels is irrelevant
    out_list.append(int(np.floor((self.input_size[i] + 2 * self.padding_shape[i] - self.kernel_shape[i]) / self.stride_shape[i]) + 1))
out_list.append(self.filters)

Basically, I use np.zeros because if I use tf.zeros I cannot assign the new_value and I get: TypeError: 'Tensor' object does not support item assignment

However, in this current state I am getting:
NotImplementedError: Cannot convert a symbolic Tensor (placeholder_1:0) to a numpy array.

On that same assignment. I don't see an easy fix, I think I should change the strategy of the code completely.

I even have to do that ugly `index = 0` and `index += 1` instead of `enumerate` because of [this](https://stackoverflow.com/a/62467248/5931672). With TF 2 they are at least on the right track but they have yet some work to do before they do it more "pythonic" — J Agustin Barrachina, Jun 19 '20 at 10:04

score 0 · Accepted Answer · answered Jun 22 '20 at 15:59

In the end, I did it in a very inefficient way based in this comment, also commented here but at least it works:

new_value = tf.reduce_sum(sector * self.kernels[filter_index]) + self.bias[filter_index]
indices = (img_index,) + index + (filter_index,)
mask = tf.Variable(tf.fill(output_np.shape, 1))
mask = mask[indices].assign(0)
mask = tf.cast(mask, dtype=self.input_dtype)
output_np = array * mask + (1 - mask) * new_value

I say inefficient because I create a whole new array for each assignment. My code is taking ages to compute for the moment so I will keep looking for improvements and post here if I get something better.

Implement ConvND in Tensorflow

1 Answers1