Install RAPIDS library on Googe Colab notebook

Question

I was wondering if I could install RAPIDS library (executing machine learning tasks entirely on GPU) in Google Colaboratory notebook?

I've done some research but I've not been able to find the way to do that...

Answer 1

This is now possible with the new T4 instances https://medium.com/rapids-ai/run-rapids-on-google-colab-for-free-1617ac6323a8

To enable cuGraph too, you can replace the wget command with:

!conda install -c nvidia/label/cuda10.0 -c rapidsai/label/cuda10.0 -c pytorch \ -c numba -c conda-forge -c numba -c defaults \ boost cudf=0.6 cuml=0.6 python=3.6 cugraph=0.6 -y

Answer 2

Dec 2019 update

New process for RAPIDS v0.11+

Because

• RAPIDS v0.11 has dependencies (pyarrow) which were not covered by the prior install script,
• the notebooks-contrib repo, which contains RAPIDS demo notebooks (e.g. colab_notebooks) and the Colab install script, now follows RAPIDS standard version-specific branch structure*
• and some Colab users still enjoy v0.10,

our honorable notebooks-contrib overlord taureandyernv has updated the script which now:

If running v0.11 or higher, updates pyarrow library to 0.15.x.

Here's the code cell to run in Colab for v0.11:

# Install RAPIDS
!wget -nc https://raw.githubusercontent.com/rapidsai/notebooks-contrib/890b04ed8687da6e3a100c81f449ff6f7b559956/utils/rapids-colab.sh
!bash rapids-colab.sh

import sys, os

dist_package_index = sys.path.index("/usr/local/lib/python3.6/dist-packages")
sys.path = sys.path[:dist_package_index] + ["/usr/local/lib/python3.6/site-packages"] + sys.path[dist_package_index:]
sys.path
if os.path.exists('update_pyarrow.py'): ## This file only exists if you're using RAPIDS version 0.11 or higher
  exec(open("update_pyarrow.py").read(), globals())

For a walk thru setting up Colab & implementing this script, see How to Install RAPIDS in Google Colab

-* e.g. branch-0.11 for v0.11 and branch-0.12 for v0.12 with default set to the current version

Answer 3

Looks like various subparts are not yet pip-installable so the only way to get them on colab would be to build them on colab, which might be more effort than you're interested in investing in this :) https://github.com/rapidsai/cudf/issues/285 is the issue to watch for rapidsai/cudf (presumably the other rapidsai/ libs will follow suit).

Answer 4

Latest solution;

!wget -nc https://github.com/rapidsai/notebooks-extended/raw/master/utils/rapids-colab.sh
!bash rapids-colab.sh

import sys, os

sys.path.append('/usr/local/lib/python3.6/site-packages/')
os.environ['NUMBAPRO_NVVM'] = '/usr/local/cuda/nvvm/lib64/libnvvm.so'
os.environ['NUMBAPRO_LIBDEVICE'] = '/usr/local/cuda/nvvm/libdevice/'

was pushed a few days ago, see issues #104 or #110, or the full rapids-colab.sh script for more info.

Note: instillation currently requires a Tesla T4 instance, checking for this can be done with;

# check gpu type
!nvidia-smi

import pynvml

pynvml.nvmlInit()
handle = pynvml.nvmlDeviceGetHandleByIndex(0)
device_name = pynvml.nvmlDeviceGetName(handle)

# your dolphin is broken, please reset & try again
if device_name != b'Tesla T4':
  raise Exception("""Unfortunately this instance does not have a T4 GPU.
    
    Please make sure you've configured Colab to request a GPU instance type.
    
    Sometimes Colab allocates a Tesla K80 instead of a T4. Resetting the instance.

    If you get a K80 GPU, try Runtime -> Reset all runtimes...""")
  
# got a T4, good to go 
else:
  print('Woo! You got the right kind of GPU!')