3

I'm running a Python script using GPU-enabled Tensorflow. However, the program doesn't seem to recognize any GPU and starts using CPU straight away. What could be the cause of this?

Onno Kampman
  • 1,112
  • 1
  • 6
  • 13
  • Did you install it with GPU support? – rmeertens Feb 16 '17 at 09:37
  • Yes. It used to run 2 weeks ago but suddenly it just jumps to CPU. – Onno Kampman Feb 16 '17 at 10:16
  • how do you know it's not using GPU? – Yaroslav Bulatov Feb 16 '17 at 17:57
  • Please run the session with `session_config=tf.ConfigProto(log_device_placement=True)`. It's possible that the operators that you are requesting do not have a GPU implementation. – drpng Feb 16 '17 at 18:36
  • I know it's not using GPU because when it does it displays information about the GPU. I have bypassed the issue by creating a new virtual environment with Anaconda, and somehow it works in that environment. Not sure why exactly, but I got what I want... Thanks everybody! – Onno Kampman Feb 17 '17 at 08:27

1 Answers1

0

Just want to add to the discussion that tensorflow may stop seeing a GPU due to CUDA initialization failure, in other words, tensorflow detects the GPU, but can't dispatch any op onto it, so it falls back to CPU. In this case, you should see an error in the log, like this

E tensorflow/stream_executor/cuda/cuda_driver.cc:481] failed call to cuInit: CUDA_ERROR_UNKNOWN

The cause is likely to be the conflict between different processes using GPU simultaneously. When that is the case, the most reliable way I found to get tensorflow working is to restart the machine. In the worst case, reinstalling tensorflow and / or NVidia driver.

See also one more case when GPU suddenly stops working.

Maxim
  • 52,561
  • 27
  • 155
  • 209