0

According to Andrej's blog - enter image description here

Where he says that for a Convolutional Layer, with parameter sharing, it introduces F x F x D weights per filter, for a total of (F x F x D) x K weights and K biases.

In my tensorflow code, I have an architecture like this (where D=1)

  1. conv1 : F = 3, K = 32, S = 1, P = 1.

  2. pool1 :

  3. conv2 and so on...

According to the formula,

  1. A model generated with F=3 for conv1 should have 9K weights ,i.e. smaller model, and

  2. A model generated with F=5 should have 25K weights i.e. bigger model

Question In my code, when I write out the model files for both these cases, I see that the .ckpt file is about 380MB (F=3) and 340MB (F=5). Am I missing something?

Code: Here's the reference code for saving the variables to a model and printing its size.

    ''' Run the session and save the model'''

#Add a saver here
saver = tf.train.Saver()

# Run session 
sess.run(tf.initialize_all_variables())
for i in range(201):
  batch = mnist.train.next_batch(50)
  if i%100 == 0:
    train_accuracy = accuracy.eval(feed_dict={x:batch[0], y_: batch[1], keep_prob: 1.0})
    print("step %d, training accuracy %g"%(i, train_accuracy))
  train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 1.0})

# Save model 
save_path = saver.save(sess, "/Users/voladoddi/Desktop/dropmodel.ckpt")
print("Model saved in file: %s" % save_path)

# Test 
print("test accuracy %g"%accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))


# Print model size. 
vars = 0
for v in tf.all_variables():
    vars += np.prod(v.get_shape().as_list())
print(vars*4)/(1024**2),"MB"
Raaj
  • 1,180
  • 4
  • 18
  • 36
  • Those calculations apply only for convolutional layers, if your model has some fully connected layers, then a bigger filter size could produce smaller feature maps, reducing the number of parameters in the FC layers, and depending on the specific model configuration, this could lead to a smaller number of parameters. – Dr. Snoopy Apr 17 '17 at 10:42
  • Hi @MatiasValdenegro, I guess according to this link - http://stackoverflow.com/questions/42786717/how-to-calculate-learnable-parameters-for-neural-network?noredirect=1&lq=1 , I got the terms "model size" and "number of parameters" mixed. You mean to say that #parameters =/= model size. Is this correct? – Raaj Apr 17 '17 at 14:12
  • No, I am talking about something completely different. – Dr. Snoopy Apr 17 '17 at 14:53
  • Printing the name and size of all variables should help you understand what is going on here (it will be easier to debug that than check model sizes) – Alexandre Passos Apr 17 '17 at 16:32

0 Answers0