How can I I initialize the weights in slim.conv2d() with the value of existing model

Question

I use slim.conv2d to set up VGG-net

with slim.arg_scope([slim.conv2d, slim.max_pool2d], padding='SAME'):
    conv1_1 = slim.conv2d(img, 64, [3, 3], scope='conv1')
    conv1_2 = slim.conv2d(conv1_1, 64, [3, 3], scope='conv1_1')
    pool1 = slim.max_pool2d(conv1_2, [2, 2], 2, scope='pool1_2')

    conv2_1 = slim.conv2d(pool1, 128, [3, 3], 1, scope='conv2_1')
    conv2_2 = slim.conv2d(conv2_1, 128, [3, 3], 1, scope='conv2_2')
    pool2 = slim.max_pool2d(conv2_2, [2, 2], 2, scope='pool2')

    conv3_1 = slim.conv2d(pool2, 256, [3, 3], 1, scope='conv3_1')
    conv3_2 = slim.conv2d(conv3_1, 256, [3, 3], 1, scope='conv3_2')
    conv3_3 = slim.conv2d(conv3_2, 256, [3, 3], 1, scope='conv3_3')
    pool3 = slim.max_pool2d(conv3_3, [2, 2], 2, scope='pool3')

    conv4_1 = slim.conv2d(pool3, 512, [3, 3], scope='conv4_1')
    # print conv4_1.shape
    conv4_2 = slim.conv2d(conv4_1, 512, [3, 3], scope='conv4_2')
    conv4_3 = slim.conv2d(conv4_2, 512, [3, 3], scope='conv4_3')  # 38

If I want to initialize the variables of conv1 or conv2 from existing VGG-model.

How can I do it?

score 2 · Answer 1 · edited Apr 08 '18 at 14:36

You can also use assign_from_values as suggested here: Github - Initialize layers.convolution2d from numpy array

sess = tf.Session()
with sess.as_default():

    init = tf.global_variables_initializer()
    sess.run(init)

    path = pathlib.Path('./assets/classifier_weights.npz')
    if(path.is_file()):
        print("Initilize Weights from Numpy Array")
        init_weights = np.load(path)
        assign_op, feed_dict_init = slim.assign_from_values({
            'conv1/weights' : init_weights['conv1_w'],
        })
        sess.run(assign_op, feed_dict_init)

score 0 · Answer 2 · answered May 08 '17 at 16:50

I'm assuming you have a checkpoint of your existing VGG model.

One way to do this using TF Slim is to restore from a checkpoint, but specify a custom mapping between the variable names in the checkpoint and variables in your model. See the comments here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/slim/python/slim/learning.py#L146

score 0 · Answer 3 · answered Jun 16 '17 at 09:32

0

I've replaced the slim.conv2d with tf.nn.conv2d(input, kernel...), where the kernel was created using tf.get_variable, and assigned using tf.assign.

answered Jun 16 '17 at 09:32

Tal

127
1
7

How can I I initialize the weights in slim.conv2d() with the value of existing model

3 Answers3