How to check if GPU memory is available using PyOpenCL

Question

I would like to know if there is a way to check how much GPU memory is available before a function uses it. I have code that often uses 1.5 GB of GPU memory or more, and if something else is using the GPU when my program wants to use it, I get a MemoryError exception or something similar.

I would like to implement some sort of code so that I can check to see if the GPU has enough memory available, and if it does, go ahead and run, but if not, wait until it IS available.

(Preferably, I would like to check before trying to use the GPU rather than using a try-except loop and just retrying if it fails)

I checked the PyOpenCL documentation to see if there was something relevant under device_info, but I couldn't find any actual descriptions.

score 3 · Accepted Answer · edited May 23 '17 at 12:10

This is not possible, and is actually a limitation of OpenCL, not just PyOpenCL. See here.

On NVIDIA devices, you can use nvidia-ml-py. Then you can do something like this:

from pynvml import *
nvmlInit()
for i in range(nvmlDeviceGetCount()):
    handle = nvmlDeviceGetHandleByIndex(i)
    meminfo = nvmlDeviceGetMemoryInfo(handle)
    print("%s: %0.1f MB free, %0.1f MB used, %0.1f MB total" % (
        nvmlDeviceGetName(handle),
        meminfo.free/1024.**2, meminfo.used/1024.**2, meminfo.total/1024.**2))
nvmlShutdown()

score 1 · Answer 2 · answered Sep 22 '18 at 21:03

I tried using the above answer with python3 and I did not work, however I was able to work around that using the following code.

from py3nvml.py3nvml import *
nvmlInit()
deviceCount = nvmlDeviceGetCount()
for i in range(deviceCount):
    handle = nvmlDeviceGetHandleByIndex(i)
    gpuUtilization = nvmlDeviceGetUtilizationRates(handle)
    print(f'{nvmlDeviceGetName(handle)} Utilization: {gpuUtilization.gpu}% Memory: {gpuUtilization.memory}%')
nvmlShutdown()

How to check if GPU memory is available using PyOpenCL

2 Answers2