How to specify filter in keras conv2d

Question

I am trying to implement into my Keras model a conv2D layer that uses a specific Gaussian filter. I have code that makes the filter, although the existing Keras Conv2D does not have a parameter for the filter itself. Instead, there is a filters parameter, which is an integer that specifies the dimensionality of the output space, and the kernel_size, which is a tuple that specifies the dimensions of the kernel.

I have tried to use the Keras backend stuff to solve this problem, as the conv2D function here allows you to input a specific filter as I would like to do. The problem is that I do not know how to get this back into my model, so again, I am stuck. I have also found questions similar to mine like this one which suggests writing my own layer, but I am not really sure how I would go about doing that, and would hope to avoid it if there is an easier way.

As always, any help is much appreciated.

Answer 1

Long answer short:

You don't need a convolution layer at all.

The purpose of convolution layers is to find the right filters for you. As you already know which filter to use, you can happily skip the whole convolution stuff and jump straight to the fully connected layers.

1. Apply the Gaussian filters to your image.

2. Use the Flatten() layer to feed the images directly to a MLP.

Hope this help.

Answer 2

I have just posted an answer to the linked question but it may be useful here as an example of how to apply a custom filter in Keras. For your example of a Gaussian, use this adapted for 2D to get a filter,

import numpy as np
import scipy.stats as st

def gkern(kernlen=[21,21], nsig=[3, 3]):
    """Returns a 2D Gaussian kernel array."""

    assert len(nsig) == 2
    assert len(kernlen) == 2
    kern1d = []
    for i in range(2):
        interval = (2*nsig[i]+1.)/(kernlen[i])
        x = np.linspace(-nsig[i]-interval/2., nsig[i]+interval/2., kernlen[i]+1)
        kern1d.append(np.diff(st.norm.cdf(x)))

    kernel_raw = np.sqrt(np.outer(kern1d[0], kern1d[1]))
    kernel = kernel_raw/kernel_raw.sum()
    return kernel

import matplotlib.pyplot as plt
plt.imshow(gkern([7,7]), interpolation='none')
plt.show()

You can then set this as an initial filter and freeze that layer so it no longer trains, which would look something like this,

from keras.models import Sequential
from keras.layers import Conv2D

#Set Some Image
image = [[4,3,1,0],[2,1,0,1],[1,2,4,1],[3,1,0,2]]

# Pad to "channels_last" format 
# which is [batch, width, height, channels]=[1,4,4,1]
image = np.expand_dims(np.expand_dims(np.array(image),2),0)

#Initialise to set kernel to required value
def kernel_init(shape):
    kernel = np.zeros(shape)
    kernel[:,:,0,0] = gkern([shape[0], shape[1]])
    return kernel 

#Build Keras model
model = Sequential()
#We would freeze training of the layers if we
# wanted to keep a Gaussian filter
Gausslayer = Conv2D(1, [3,3], kernel_initializer=kernel_init, 
                    input_shape=(4,4,1), padding="valid")
Gausslayer.trainable = False
model.add(Gausslayer)
#Add some more layers here
#model.add(Conv2D(...)
model.build()

# To apply existing filter, we use predict with no training
out = model.predict(image)
print(out[0,:,:,0])

and can be adapted to add more layers which are trainable.