6

Anybody tried to use numba in google collaboratory? I just can not figure out how to set it up in this environment. At the moment, I'm stuck with the error library nvvm not found.

hynekcer
  • 14,942
  • 6
  • 61
  • 99
Algis
  • 312
  • 4
  • 14

3 Answers3

9

Copy this code into cell. It works for me.

!apt-get install nvidia-cuda-toolkit
!pip3 install numba

import os
os.environ['NUMBAPRO_LIBDEVICE'] = "/usr/lib/nvidia-cuda-toolkit/libdevice"
os.environ['NUMBAPRO_NVVM'] = "/usr/lib/x86_64-linux-gnu/libnvvm.so"

from numba import cuda
import numpy as np
import time

@cuda.jit
def hello(data):
    data[cuda.blockIdx.x, cuda.threadIdx.x] = cuda.blockIdx.x

numBlocks = 5
threadsPerBlock = 10

data = np.ones((numBlocks, threadsPerBlock), dtype=np.uint8)

hello[numBlocks, threadsPerBlock](data)

print(data)
Algis
  • 312
  • 4
  • 14
  • Do you need `!apt-get -y` as well? – korakot Feb 16 '18 at 10:18
  • Also don't forget to use GPU Runtime too. Otherwise, you will get an error. – korakot Feb 16 '18 at 10:40
  • 2
    Dependencies can be simplified by skipping "recommended" packages by `!apt install -y --no-install-recommends -q nvidia-cuda-toolkit`. It is still huge and slow install, but only 100 packages are installed instead of more than 600, like fonts etc. – hynekcer Feb 17 '18 at 17:42
  • 1
    I can confirm that it can not be minimized more. If the package `nvidia-cuda-toolkit` is removed, `numba` will be broken after kernel restart even if all other 99 dependent packages remain installed. – hynekcer Feb 17 '18 at 23:26
  • 1
    Installation of the toolkit did not work for me, but luckily when I used the smaller install of @hynekcer it does work somehow. – Maarten-vd-Sande Mar 31 '18 at 16:14
3

I didn't have to install the packages @Algis suggested, but the paths to the drivers were different. So I had to do the following.

First determine the correct paths for the drivers

!find / -iname 'libdevice'
!find / -iname 'libnvvm.so'

# Output:
# /usr/local/cuda-9.2/nvvm/lib64/libnvvm.so
# /usr/local/cuda-9.2/nvvm/libdevice

Then set the paths as @Algis described

import os
os.environ['NUMBAPRO_LIBDEVICE'] = "/usr/local/cuda-9.2/nvvm/libdevice"
os.environ['NUMBAPRO_NVVM'] = "/usr/local/cuda-9.2/nvvm/lib64/libnvvm.so"
Stan
  • 250
  • 2
  • 9
1

You can do @Stan's work in one simple sweep if you have this block at the beginning of your colab notebook (this also automatically updates as CUDA gets updated)

import os
dev_lib_path = !find / -iname 'libdevice'
nvvm_lib_path = !find / -iname 'libnvvm.so'
assert len(dev_lib_path)>0, "Device Lib Missing"
assert len(nvvm_lib_path)>0, "NVVM Missing"
os.environ['NUMBAPRO_LIBDEVICE'] = dev_lib_path[0]
os.environ['NUMBAPRO_NVVM'] = nvvm_lib_path[0]
kmader
  • 1,319
  • 1
  • 10
  • 13