1

For faster inference one model, I want to merge 'Conv-BN-Scale' into a single 'Conv' layer for my tensorflow model, but I can not find some useful complete example about how to do it?
Anyone can give some advises or complete code example?

Thanks!

JustingGong
  • 45
  • 1
  • 8
  • I am not sure about your question, but you can always write a Python class called something like MyLayer and define whatever order of operations you'd like to have. – Richard_wth Jan 28 '19 at 02:49
  • @Richard_wth thanks your reply, can you provide some complete example code? – JustingGong Jan 28 '19 at 03:31
  • I understand conv is convolution, BN is batch normalization, but by scale do you mean activation like ReLU or something else? – anand_v.singh Jan 28 '19 at 03:55
  • @anand_v.singh thanks for your comment, scale means `beta` and `gamma` of `batch normalization`. Please refer to https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization – JustingGong Jan 28 '19 at 05:03
  • @RayTing So basically you want a function that performs convolution `tf.nn.conv2d()` and `tf.layers.batch_normalization()` together and want to pass the parameters for `tf.layers.batch_normalization()` to the function? – anand_v.singh Jan 28 '19 at 05:09
  • @anand_v.singh yes, and the function also contains minus `moving_mean` and divide `moving_variance`. – JustingGong Jan 28 '19 at 05:21

1 Answers1

0

To merge two layers, you will need to pass a Tensor and get a tensor back that is after both the layers are applied, suppose your input tensor is X.

def MlConvBnScale(X ,kernel,strides , padding = 'SAME' , scale = False, beta_initializer = 0.1, gamma_initializer = 0.1, moving_mean_initializer = 0.1, moving_variance_initializer = 0.1):
    convLout = tf.nn.conv2d(X,
                           filter = Kernel, 
                           strides = strides,
                           padding = padding)
    return tf.nn.batch_normalization(convLout,
                                     scale = scale,
                                     beta_initializer = beta_initializer, 
                                     gamma_initializer = gamma_initializer,
                                     moving_mean_initializer = moving_mean_intializer, 
                                     moving_variance_initializer = moving_variance_initializer )

And that will return a tensor after performing both the operations, I have taken default values of variables but you can modify them in your function call, and in case your input is not already a tensor but a numpy array you can use tf.convert_to_tensor() from this link https://www.tensorflow.org/api_docs/python/tf/convert_to_tensor, and in case you are struggling with kernel/filter and its application, check out this thread. What does tf.nn.conv2d do in tensorflow?

If you have any queries or run into trouble implementing it, comment down below and we will see.

anand_v.singh
  • 2,768
  • 1
  • 16
  • 35
  • thanks for your advise, but I do mean this. I mean `merge conv layer and bn layer to one conv layer` and we need to surgery the graph and parameter. The similar method can refer to https://tkv.io/posts/fusing-batchnorm-and-conv/ and https://github.com/sanghoon/pva-faster-rcnn/commit/39570aab8c6513f0e76e5ab5dba8dfbf63e9c68c/ – JustingGong Jan 28 '19 at 07:20
  • So you want these equations filter weights: \mathbf{W}=\mathbf{W}_{BN}\cdot \mathbf{W}_{conv}W=W BN ​ ⋅W conv ​ ; bias: \mathbf{b}=\mathbf{W}_{BN}\cdot\mathbf{b}_{conv}+\mathbf{b}_{BN}b=W BN ​ ⋅b conv ​ +b BN ​ . Implemented by a function? – anand_v.singh Jan 28 '19 at 08:00
  • yes, surgery of my pre-trained ckpt and meta file for merging bn layer into previous conv layer. – JustingGong Jan 28 '19 at 08:14