Spatial reflection padding in Caffe

Question

Any ideas how to implement Spatial Reflection Padding in Caffe like in Torch?

  (x): nn.SpatialReflectionPadding(l=1, r=1, t=1, b=1)
  (x): nn.SpatialConvolution(64 -> 64, 3x3)
  (x): nn.ReLU

Answer 1

One way to do this would be using the Python Layer of Caffe. You can then set the functions yourself and customize based on your needs. However, this layer can only run in the CPU, so it might slow down your model especially if you use it in the middle of the network.

In the following, I have defined a layer to zero pad input using the Python layer, which you can modify to suit your needs:

import caffe
import numpy as np

class SpatialReflectionPadding(caffe.Layer):

def setup(self,bottom,top):
    if len(bottom) != 1: # check that a single bottom blob is given
        raise Exception("Expected a single blob")       
    if len(bottom[0].shape) != 4: # check that it is 4D
        raise Exception("Expected 4D blob")
    params = eval(self.param_str) # get the params given in the prototxt
    self.l = params["l"]
    self.r = params["r"]
    self.t = params["t"]
    self.b = params["b"]

def reshape(self,bottom,top):
    top[0].reshape(bottom[0].shape[0],bottom[0].shape[1],bottom[0].shape[2]+self.t+self.b,bottom[0].shape[3]+self.r+self.l) # set the shape of the top blob based on the shape of the existing bottom blob

def forward(self,bottom,top):
    for i in range(0,top[0].shape[2]):
        for j in range(0,top[0].shape[3]):
            if (i < self.t or i >= self.t+bottom[0].shape[2]) or (j < self.l or j >= self.l+bottom[0].shape[3]):
                top[0].data[:,:,i,j] = 0 # for the padded part, set the value to 0
            else:
                top[0].data[:,:,i,j] = bottom[0].data[:,:,i-self.t,j-self.l] # for the rest, copy the value from the bottom blob

def backward(self,top,propagate_down,bottom):
    bottom[0].diff[...] = np.full(bottom[0].shape,1) * top[0].diff[:,:,self.t:self.t+bottom[0].shape[2],self.l:self.l+bottom[0].shape[3]] # set the gradient for backward pass

Then, in your prototxt file, you can use it as:

layer {
    name: "srp" # some name
    type: "Python"
    bottom: "some_layer" # the layer which provides the input blob
    top: "srp"
    python_param {
        module: "caffe_srp" # whatever is your module name
        layer: "SpatialReflectionPadding"
        param_str: '{ "l": 1, "b": 1, "t": 1, "r": 1}'
    }
}

I am not 100% sure that it works correctly, though when I used it, it appeared to do so. In any case, it should give an idea and a starting point on how one could proceed. Also, you could refer to this question and its answers.