1

Hello Everyone,

I am working on a Image classification problem using tensorflow and Convolution Neural Network. My model is having following layers.

  • Input image of size 2456x2058
  • 3 convolution Layer {Con1-shape(10,10,1,32); Con2-shape(5,5,32,64); Con3-shape(5,5,64,64)}
  • 3 max pool 2x2 layer
  • 1 fully connected layer.

I have tried using the NVIDIA-SMI tool but it shows me the GPU memory consumption as the model runs.
I would like to know if there is any method or a way to find the estimate of memory before running the model on GPU. So that I can design models with the consideration of available memory.
I have tried using this method for estimation but my calculated memory and observed memory utilisation are no where near to each other.

Thank you all for your time.

Community
  • 1
  • 1
AKSHAY MANKAR
  • 63
  • 9

3 Answers3

1

As far as I understand, when you open a session with tensorflow-gpu, it allocates all the memory in the GPUS that are available. So, when you look at the nvidia-smi output, you will always see the same amount of used memory, even if it actually uses only a part of it. There are options when opening a session to force tensorflow to allocate only a part of the available memory (see How to prevent tensorflow from allocating the totality of a GPU memory? for instance)

Pietro Tortella
  • 1,084
  • 1
  • 6
  • 13
  • thanks for the information. But I have one question, isn't it that if I restrict my TF to some part of GPU memory it will use 100% of that part and again I will not be able to solve the problem of estimation. – AKSHAY MANKAR May 31 '17 at 09:00
  • Yes, the point is that for the problem of the estimation you cannot use tools such as nvidia-smi... one way is to compute it theoretically as you mentioned in the link, another way could be to catch all tensors created by the graph, fetch their shapes and estimate the size of each tensor it creates (but then I don't know if actually tensorflow needs more memory to make the computations, or if it frees some memory while making the computations, or what else...) – Pietro Tortella May 31 '17 at 09:14
  • thanks for the share I will look into this idea of yours to catch the tensors. and if any thing productive comes up I will tell you. – AKSHAY MANKAR May 31 '17 at 09:24
  • also, here https://datascience.stackexchange.com/questions/12649/how-to-calculate-the-mini-batch-memory-impact-when-training-deep-learning-models and the links therein face this problem – Pietro Tortella May 31 '17 at 09:47
  • Thanks for this new link, lets hope this will solve our problem.I will inform you accordingly. – AKSHAY MANKAR May 31 '17 at 10:58
  • So how to "estimate memory usage" ? You never answer the question. – huang Nov 04 '20 at 09:27
0

NVIDIA-SMI ... shows me the GPU memory consumption as the model run

TF preallocates all available memory when you use it, so NVIDIA-SMI would show nearly 100% memory usage ...

but my calculated memory and observed memory utilisation are no where near to each other.

.. so this is unsurprising.

MWB
  • 11,740
  • 6
  • 46
  • 91
  • I agree with you that SMI tool shows the 100% memory usage. So do you have any method to estimate the memory usage so that I will not get the Resource Exhaust Error. – AKSHAY MANKAR May 31 '17 at 09:05
0

You can control the memory allocation of GPU in TensorFlow. Once you calculated your memory requirements for your Deep learning model you can use tf.GPUOptions.

For example if you want to allocate 4 GB(approximately) of GPU memory out of 8 GB.

config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.4
session = tf.Session(config=config, ...)

Once done pass it in tf.Session using config parameter

The per_process_gpu_memory_fraction is used to bound the available amount of GPU memory.

Here's the link to documentation :-

https://www.tensorflow.org/tutorials/using_gpu

WaterRocket8236
  • 1,442
  • 1
  • 17
  • 27
  • Thankyou for reply, but if I use Config Proto method I will be restricting the usage of GPU, What I need is estimation of expected memory that will be used, so I can design the model that will be a perfect fit, which will lead me to the following points: 1) ability to use the entire GPU memory 2) not to be struck by Resource Exhaust Error. – AKSHAY MANKAR Aug 23 '17 at 05:57