24

I want to design a convolutional neural network which occupy GPU resource no more than Alexnet.I want to use FLOPs to measure it but I don't know how to calculate it.Is there any tools to do it,please?

StalkerMuse
  • 1,001
  • 3
  • 11
  • 22
  • duplicate of: http://stackoverflow.com/q/41996593/1714410 – Shai Apr 19 '17 at 09:07
  • 3
    @Shai: that doesn't answer the question. The resolution of that link is that half the problem is an open request in TF. This is Caffe. – Prune Apr 19 '17 at 15:50
  • For online tool see http://dgschwend.github.io/netscope/#/editor . For alexnet see http://dgschwend.github.io/netscope/#/preset/alexnet . This supports most wide known layers. For custom layers you will have to calculate yourself. – nomem Apr 19 '17 at 09:31
  • As of the day of this comment, this webpage (http://dgschwend.github.io/netscope/#/preset/alexnet) doesn't seem to show the FLOps. has NaNs for all the macc entries – MonsieurBeilto Jun 07 '19 at 17:01
  • If you're using Keras, you could just use the patch in this pull request: https://github.com/fchollet/keras/pull/6203 Then just call print_summary() and you'll see both the flops per layer and the total. Even if not using Keras, it may be worth it to recreate your nets in Keras just so you can get the flops counts. – Alex I Apr 19 '17 at 09:36

2 Answers2

13

For future visitors, if you use Keras and TensorFlow as Backend then you can try the following example. It calculates the FLOPs for the MobileNet.

import tensorflow as tf
import keras.backend as K
from keras.applications.mobilenet import MobileNet

run_meta = tf.RunMetadata()
with tf.Session(graph=tf.Graph()) as sess:
    K.set_session(sess)
    net = MobileNet(alpha=.75, input_tensor=tf.placeholder('float32', shape=(1,32,32,3)))

    opts = tf.profiler.ProfileOptionBuilder.float_operation()    
    flops = tf.profiler.profile(sess.graph, run_meta=run_meta, cmd='op', options=opts)

    opts = tf.profiler.ProfileOptionBuilder.trainable_variables_parameter()    
    params = tf.profiler.profile(sess.graph, run_meta=run_meta, cmd='op', options=opts)

    print("{:,} --- {:,}".format(flops.total_float_ops, params.total_parameters))
mic
  • 1,190
  • 1
  • 17
  • 29
Tobias Scheck
  • 643
  • 8
  • 16
  • How it's related to https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation ? – mrgloom Sep 09 '18 at 16:41
  • If I run this on Mobilenet V2, I get flops of 7.02 Million but in the paper are 300 Million MACs? So there should be a little difference between MACs and FLOP values but it is quite huge difference. – Awais Hussain Mar 13 '20 at 06:28
  • I've changed the code to fit the tf 2.0 api. If I use the implementation (https://gist.github.com/scheckmedia/cadc5eb3d74ed57a4f3d78011a9f6f7c) I get 3504872 total parameters and 608625165 total flops. The flops are multiplications and additions, to get the MACs value you should divide the result by 2. – Tobias Scheck Mar 13 '20 at 08:21
0

Tobias Scheck's answer works if you are using TensorFlow v1.x, but if you are using TensorFlow v2.x you can use the following code:

import tensorflow as tf

def get_flops(model_h5_path):
    session = tf.compat.v1.Session()
    graph = tf.compat.v1.get_default_graph()
        

    with graph.as_default():
        with session.as_default():
            model = tf.keras.models.load_model(model_h5_path)

            run_meta = tf.compat.v1.RunMetadata()
            opts = tf.compat.v1.profiler.ProfileOptionBuilder.float_operation()
        
            # We use the Keras session graph in the call to the profiler.
            flops = tf.compat.v1.profiler.profile(graph=graph,
                                                  run_meta=run_meta, cmd='op', options=opts)
        
            return flops.total_float_ops

The above function takes the path of a saved model in h5 format. You can save your model and use the function this way:

model.save('path_to_my_model.h5')
tf.compat.v1.reset_default_graph()
print(get_flops('path_to_my_model.h5'))

Note that we use tf.compat.v1.reset_default_graph() for not to accumulate FLOPS each time we call the fuction.

al2
  • 119
  • 7