why Tensorflow-gpu is still using cpu

Question

I am using Keras with tensorflow-gpu in backend, I don't have tensorflow (CPU - version) installed, all the outputs show GPU selected but tf is using CPU and system memory

when i run my code the output is: output_code

I even ran device_lib.list_local_device() and the output is: list_local_devices_output

After running the code I tried nvidia-smi to see the usage of gpu and the output is: nvidia-smi output

Tensorflow-gpu = "1.12.0"
CUDA toolkit = "9.0"
cuDNN = "7.4.1.5"

Environment Variables contain:

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\libnvvp;
C:\WINDOWS\system32;
C:\WINDOWS;
C:\WINDOWS\System32\Wbem;
C:\WINDOWS\System32\WindowsPowerShell\v1.0\;
C:\WINDOWS\System32\OpenSSH\;
C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;
D:\Anaconda3;D:\Anaconda3\Library\mingw-w64\bin
D:\Anaconda3\Library\usr\bin;
D:\Anaconda3\Library\bin;
D:\Anaconda3\Scripts;D:\ffmpeg\bin\;

But still when i check for memory usage in task manager the output is

CPU utilization 51%, RAM utilization 86% GPU utilization 1%, GPU-RAM utilization 0% Task_manager_Output So, I think it is still using CPU instead of GPU.

System Configuration:

Windows-10 64 bit; IDE: Liclipse; Python: 3.6.5

Its using both GPU and CPU, not all operations do run in the GPU, and some data processing is run in the CPU in parallell to the GPU, like for example, feeding data to the GPU. I don't see a problem here. — Dr. Snoopy, Dec 21 '18 at 16:27
If that's the case then i guess got nothing to worry about, should i post my code as well for clarification — Hrishikesh Vichore, Dec 21 '18 at 17:35

score 8 · Accepted Answer · edited Sep 06 '21 at 14:15

8

It is using the GPU, as you can see in logs. The problem is, that a lot of things can not be done on the GPU and as long your data is small and your complexity is low, you will end up with low GPU usage.

Maybe the batch_size is to low -> Increase until you run into OOM Errors
Your data loading is consuming a lot of time and your gpu has to wait (IO Reads)
Your RAM is to low and the application uses Disk as a fallback
Preprocsssing is to slow. If you are dealing with image try to compute everything as a generator or on the gpu if possible
You are using some operations, which are not GPU accelerated

Here is some more detailed explanation.

edited Sep 06 '21 at 14:15

bastien enjalbert

337
1
3
18

answered Dec 21 '18 at 20:03

ixeption

1,972
1
13
19

1. If you are saying the RAM of GPU it is 6.33 GB available and my task is not that heavy to fallback, if you are referring to system memory, it is 8 GB total. Secondly, I am dealing with audio files and all the features are already stored in a pickle file which is directly loaded. Simultaneously, I am creating a CSV file to log the results. So is this thing consuming more RAM as it is I/O operation? – Hrishikesh Vichore Dec 23 '18 at 17:58

why Tensorflow-gpu is still using cpu

1 Answers1

Linked