1

When convoluting a multi-channel image into one channel image, usually you can have only one bias variable(as output is one channel). If I want to set local biases, that is, set biases for each pixel of the output image, how shall I do this in caffe and torch?

In Tensorflow, this is very simple. your just set a bias matrix, for example:

data is 25(height)X25(width)X48(channels)

weights is 3X3(kernel size)X48(input channels)X1(output channels)

biases is 25X25,

then, 

hidden = tf.nn.conv2d(data, weights, [1, 1, 1, 1], padding='SAME')    
output = tf.relu(hidden+biases)

Is there a similar solution in caffe ortorch?

Shai
  • 111,146
  • 38
  • 238
  • 371
Grant Tao
  • 41
  • 5

1 Answers1

3

For caffe, here is a scale layer post: Scale layer in Caffe. Scale layer can only provide one variable bias. The answer is Bias layer. bias layer can have a weight matrix, treat it as biases.

For torch, torch has a nn.Add() layer, almost like the tensorflow's tf.add() function, so nn.Add() layer is the solution.

All these have been proved by actual models.

But still thank you very much @Shai

Community
  • 1
  • 1
Grant Tao
  • 41
  • 5
  • 1
    You are obviously right `"Bias"` layer is the right way to go. However, if you pay close attention, `"Scale"` layer with `bias_term: true` has a `"Bias"` layer build into it, thus allowing for a 2D ("local") bias terms. – Shai Jul 11 '16 at 05:14