0

I'm trying to run the model scoring (inference graph) from tensorflow objec detection API to run it on multiple GPU's, tried specifying the GPU number in the main, but it runs only on single GPU.placed GPU utilization snapshot here

Using tensorflow-gpu==1.13.1, can you kindly point me what I'm missing here. 

for i in range(2):
        with tf.device('/gpu:{}' . format(i)):
            tf_init()
            init = tf.global_variables_initializer
           with detection_graph.as_default():
              with tf.Session(config=tf.ConfigProto(log_device_placement=True)) as session:
                   call to #run_inference_multiple_images function
Lavanya
  • 11
  • 4

1 Answers1

0

The responses to this question should give you a few options for fixing this.

Usually TensorFlow will occupy all visible GPUs unless told otherwise. So if you haven't already tried, you could just remove the with tf.device line (assuming you only have the two GPUs) and TensorFlow should use them both.

Otherwise, I think the easiest is setting the environment variables with os.environ["CUDA_VISIBLE_DEVICES"] = "0,1".

nheise
  • 311
  • 1
  • 8
  • Thanks for your response, I tried the specified methods, but still it runs on only one GPU, also tried adding below config when starting the session `tf.ConfigProto(gpu_options.allow_growth = True)` `tf.ConfigProto(device_count = {'GPU': 2})` During model training, when num_clones set to 4, it has utilized all the GPUs in the system, not sure what I'm missing here. Is it something related to tensorflow-gpu version? – Lavanya Oct 15 '19 at 04:59
  • TF allocates all the GPUs, but might not use them all. Maybe it has decided it doesn't need the second for inference? You could try running a larger model. Also, using the ```with tf.device``` option requires that you manually specify where you want to run the tensors... the device will automatically be chosen for tensors not specified by you. Maybe [this question](https://stackoverflow.com/questions/34834714/does-tensorflow-by-default-use-all-available-gpus-in-the-machine) will help. – nheise Oct 15 '19 at 05:46