0

I have to build yolact++ in docker enviromment (i'm using sagemaker notebook). Like this

ARG PYTORCH="1.3"
ARG CUDA="10.1"
ARG CUDNN="7"
 
FROM pytorch/pytorch:${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel

And i want to run this

COPY yolact/external/DCNv2/setup.py /opt/ml/code/external/DCNv2/setup.py
RUN cd /opt/ml/code/external/DCNv2 && \
python setup.py build develop

But i got this error :

No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
Traceback (most recent call last):
File "setup.py", line 64, in <module>
ext_modules=get_extensions(),
File "setup.py", line 41, in get_extensions
raise NotImplementedError('Cuda is not available')
NotImplementedError: Cuda is not available

But the enviromment supports CUDA. Anyone have an idea where is the problem ?

Thank you.

Keytich
  • 11
  • 3
  • The error message indicate CUDA is not considered installed by the python compiler. Did you try to check the presence (and the path) of the CUDA installation ? – XouDo May 18 '21 at 08:07
  • I cheched if cuda is available it returns False, but when i check the version of cuda is returns me 10.1, i set CUDA_PATH to CUDA_HOME="/usr/local/cuda-10.1". – Keytich May 18 '21 at 09:17
  • Ok so if it still says `using CUDA_HOME='/usr/local/cuda'` then your setting of `CUDA_HOME` does not work. – XouDo May 18 '21 at 09:25
  • And How i can solve this ? – Keytich May 18 '21 at 09:34

2 Answers2

1

SOLUTION :

i edit the /etc/docker/daemon.json with content:

{
"runtimes": {
    "nvidia": {
        "path": "/usr/bin/nvidia-container-runtime",
        "runtimeArgs": []
     } 
},
"default-runtime": "nvidia" 
}

Then i Restart docker daemon:

sudo system restart docker

it solved my problem.

Dharman
  • 30,962
  • 25
  • 85
  • 135
Keytich
  • 11
  • 3
0

You should try setting your CUDA_HOME variable in your dockerfile like this :

ARG PYTORCH="1.3"
ARG CUDA="10.1"
ARG CUDNN="7"

FROM pytorch/pytorch:${PYTORCH}-cuda${CUDA}-cudnn${CUDNN}-devel
ARG CUDA_HOME="/usr/local/cuda-10.1"

...and see if the python compiler sees it then.

XouDo
  • 945
  • 10
  • 19
  • it still says : No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda-10.1' – Keytich May 18 '21 at 10:25
  • So it's not a problem with the path. You said `cuda.is_available()` returns false... then you have to figure out why. Maybe you can check https://stackoverflow.com/questions/48152674/how-to-check-if-pytorch-is-using-the-gpu – XouDo May 18 '21 at 11:07