-1

I've been attempting to use theano for pthon on my GPU (a 980m) with GCC 6.2.0 and G++ 6.2.0 for Python 3.5.1 on Ubuntu 16.04. I've installed all prerequisites as per the instructions and Theano given no errors on import. However, when I run theano.test() I get the following rather verbose ad cryptic error.

===============================
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
In file included from mod.cu:10:0:
/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/cuda/cuda_ndarray.cuh:17:0: warning: "PyString_Check" redefined
 #define PyString_Check PyUnicode_Check
/usr/lib/gcc/x86_64-linux-gnu/6/include/stddef.h(436): error: expected a ";"
/usr/include/x86_64-linux-gnu/c++/6/bits/c++config.h(202): error: expected a ";"
/usr/include/c++/6/exception(63): error: expected a ";"
/usr/include/c++/6/exception(69): error: expected a ";"
/usr/include/c++/6/exception(77): error: expected a ";"
/usr/include/c++/6/exception(85): error: expected a ";"
/usr/include/c++/6/exception(95): error: expected a "{"
/usr/include/c++/6/bits/exception_ptr.h(64): error: function "std::current_exception" returns incomplete type "std::__exception_ptr::exception_ptr"
/usr/include/c++/6/bits/exception_ptr.h(64): error: expected a "{"
/usr/include/c++/6/bits/exception_ptr.h(71): error: namespace "std" has no member "rethrow_exception"
/usr/include/c++/6/bits/exception_ptr.h(81): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(83): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(84): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(86): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(88): error: declaration is incompatible with previous "std::current_exception"
(64): here
/usr/include/c++/6/bits/exception_ptr.h(88): error: use of a local type to declare a function
/usr/include/c++/6/bits/exception_ptr.h(88): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(89): error: use of a local type to declare a function
/usr/include/c++/6/bits/exception_ptr.h(92): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(94): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(97): error: incomplete type is not allowed
/usr/include/c++/6/bits/exception_ptr.h(97): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(118): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(128): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(145): error: use of a local type to declare a function
/usr/include/c++/6/bits/exception_ptr.h(146): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(149): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(154): error: use of a local type to declare a function
/usr/include/c++/6/bits/exception_ptr.h(155): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(158): error: use of a local type to declare a function
/usr/include/c++/6/bits/exception_ptr.h(159): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(162): error: use of a local type to declare a function
/usr/include/c++/6/bits/exception_ptr.h(163): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(171): error: expected a ";"
/usr/include/c++/6/bits/exception_ptr.h(197): error: expected a ";"
/usr/include/c++/6/bits/move.h(47): error: expected a ";"
/usr/include/c++/6/type_traits(71): error: identifier "constexpr" is undefined
/usr/include/c++/6/type_traits(71): error: template parameter "_Tp" may not be redeclared in this scope
/usr/include/c++/6/type_traits(71): error: expected a ";"
/usr/include/c++/6/type_traits(72): error: member "std::integral_constant<_Tp, __v>::_Tp" is not a type name
/usr/include/c++/6/type_traits(73): error: member "std::integral_constant<_Tp, __v>::_Tp" is not a type name
/usr/include/c++/6/type_traits(74): error: identifier "constexpr" is undefined
/usr/include/c++/6/type_traits(79): error: identifier "constexpr" is undefined
/usr/include/c++/6/type_traits(79): error: "value_type" has already been declared in the current scope
/usr/include/c++/6/type_traits(79): error: expected a ";"
/usr/include/c++/6/type_traits(84): error: identifier "constexpr" is undefined
/usr/include/c++/6/type_traits(84): error: "_Tp" is not a function or static data member
/usr/include/c++/6/type_traits(93): error: expected a declaration
/usr/include/c++/6/type_traits(93): error: expected a ";"
/usr/include/c++/6/type_traits(126): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
/usr/include/c++/6/type_traits(235): error: identifier "char16_t" is undefined
/usr/include/c++/6/type_traits(239): error: identifier "char32_t" is undefined
/usr/include/c++/6/type_traits(239): error: class "std::__is_integral_helper<<error-type>>" has already been defined
/usr/include/c++/6/type_traits(337): error: identifier "__float128" is undefined
/usr/include/c++/6/type_traits(453): error: expected a ">"
/usr/include/c++/6/type_traits(457): error: expected a ">"
/usr/include/c++/6/type_traits(465): error: expected a ">"
/usr/include/c++/6/type_traits(469): error: expected a ">"
/usr/include/c++/6/type_traits(477): error: expected a ">"
/usr/include/c++/6/type_traits(481): error: expected a ">"
/usr/include/c++/6/type_traits(489): error: expected a ">"
/usr/include/c++/6/type_traits(493): error: expected a ">"
/usr/include/c++/6/type_traits(501): error: expected a ">"
/usr/include/c++/6/type_traits(505): error: expected a ">"
/usr/include/c++/6/type_traits(513): error: expected a ">"
/usr/include/c++/6/type_traits(517): error: expected a ">"
/usr/include/c++/6/type_traits(525): error: expected a ">"
/usr/include/c++/6/type_traits(529): error: expected a ">"
/usr/include/c++/6/type_traits(537): error: expected a ">"
/usr/include/c++/6/type_traits(541): error: expected a ">"
/usr/include/c++/6/type_traits(551): error: namespace "std" has no member "nullptr_t"
/usr/include/c++/6/type_traits(572): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
/usr/include/c++/6/type_traits(578): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
/usr/include/c++/6/type_traits(585): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
/usr/include/c++/6/type_traits(592): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
/usr/include/c++/6/type_traits(602): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
/usr/include/c++/6/type_traits(628): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
/usr/include/c++/6/type_traits(738): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
/usr/include/c++/6/type_traits(752): error: expected a ";"
/usr/include/c++/6/type_traits(767): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
/usr/include/c++/6/type_traits(777): error: expected a ")"
/usr/include/c++/6/type_traits(788): error: an explicit template argument list is not allowed on this declaration
/usr/include/c++/6/type_traits(788): error: expected a type specifier
/usr/include/c++/6/type_traits(788): error: function returning function is not allowed
/usr/include/c++/6/type_traits(788): error: expected a ";"
/usr/include/c++/6/type_traits(794): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
/usr/include/c++/6/type_traits(795): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
/usr/include/c++/6/type_traits(825): error: identifier "noexcept" is undefined
/usr/include/c++/6/type_traits(825): error: function call is not allowed in a constant expression
/usr/include/c++/6/type_traits(825): error: function call is not allowed in a constant expression
/usr/include/c++/6/type_traits(825): error: this operator is not allowed in a template argument expression
/usr/include/c++/6/type_traits(836): error: an explicit template argument list is not allowed on this declaration
/usr/include/c++/6/type_traits(836): error: expected a type specifier
/usr/include/c++/6/type_traits(836): error: function returning function is not allowed
/usr/include/c++/6/type_traits(836): error: expected a ";"
/usr/include/c++/6/type_traits(842): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
/usr/include/c++/6/type_traits(843): error: space required between adjacent ">" delimiters of nested template argument lists (">>" is the right shift operator)
/usr/include/c++/6/type_traits(868): error: function returning function is not allowed
/usr/include/c++/6/type_traits(879): error: an explicit template argument list is not allowed on this declaration
Error limit reached.
100 errors detected in the compilation of "/tmp/tmpxft_00002ddd_00000000-9_mod.cpp1.ii".
Compilation terminated.
ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu: ('nvcc return status', 4, 'for cmd', 'nvcc -shared -O3 -m64 -Xcompiler -D_FORCE_INLINES,-DCUDA_NDARRAY_CUH=mc72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,-fPIC,-fvisibility=hidden -Xlinker -rpath,/home/sean/.theano/compiledir_Linux-4.4--generic-x86_64-with-Ubuntu-16.04-xenial-x86_64-3.5.2-64/cuda_ndarray -I/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/cuda -I/home/sean/.local/lib/python3.5/site-packages/numpy/core/include -I/usr/include/python3.5m -I/home/sean/.local/lib/python3.5/site-packages/theano/gof -o /home/sean/.theano/compiledir_Linux-4.4--generic-x86_64-with-Ubuntu-16.04-xenial-x86_64-3.5.2-64/cuda_ndarray/cuda_ndarray.so mod.cu -L/usr/lib -lcublas -lpython3.5m -lcudart')
/home/sean/.local/lib/python3.5/site-packages/theano/misc/pycuda_init.py:34: UserWarning: PyCUDA import failed in theano.misc.pycuda_init
  warnings.warn("PyCUDA import failed in theano.misc.pycuda_init")
/home/sean/.local/lib/python3.5/site-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
  "downsample module has been moved to the theano.tensor.signal.pool module.")
......WARNING (theano.tensor.blas): We did not found a dynamic library into the library_dir of the library we use for blas. If you use ATLAS, make sure to compile it with dynamics library.
..............S........................................S..................WARNING (theano.tensor.blas): We did not found a dynamic library into the library_dir of the library we use for blas. If you use ATLAS, make sure to compile it with dynamics library.
======================================================================
ERROR: Failure: RuntimeError (('Wrong major API version for gpuarray:', 0, 'Make sure Theano and libgpuarray/pygpu are in sync.'))
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/sean/.local/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
    raise self.exc_val.with_traceback(self.tb)
  File "/home/sean/.local/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/home/sean/.local/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/home/sean/.local/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/usr/lib/python3.5/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/usr/lib/python3.5/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 693, in _load
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/gpuarray/tests/test_abstractconv.py", line 9, in <module>
    from .config import mode_with_gpu, test_ctx_name
  File "/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/gpuarray/tests/config.py", line 10, in <module>
    theano.sandbox.gpuarray.init_dev('cuda')
  File "/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/gpuarray/__init__.py", line 46, in init_dev
    "Make sure Theano and libgpuarray/pygpu "
RuntimeError: ('Wrong major API version for gpuarray:', 0, 'Make sure Theano and libgpuarray/pygpu are in sync.')

======================================================================
ERROR: Failure: RuntimeError (('Wrong major API version for gpuarray:', 0, 'Make sure Theano and libgpuarray/pygpu are in sync.'))
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/sean/.local/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
    raise self.exc_val.with_traceback(self.tb)
  File "/home/sean/.local/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/home/sean/.local/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/home/sean/.local/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/usr/lib/python3.5/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/usr/lib/python3.5/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 693, in _load
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/gpuarray/tests/test_basic_ops.py", line 25, in <module>
    from .config import mode_with_gpu, mode_without_gpu, test_ctx_name
  File "/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/gpuarray/tests/config.py", line 10, in <module>
    theano.sandbox.gpuarray.init_dev('cuda')
  File "/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/gpuarray/__init__.py", line 46, in init_dev
    "Make sure Theano and libgpuarray/pygpu "
RuntimeError: ('Wrong major API version for gpuarray:', 0, 'Make sure Theano and libgpuarray/pygpu are in sync.')

======================================================================
ERROR: Failure: RuntimeError (('Wrong major API version for gpuarray:', 0, 'Make sure Theano and libgpuarray/pygpu are in sync.'))
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/sean/.local/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
    raise self.exc_val.with_traceback(self.tb)
  File "/home/sean/.local/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/home/sean/.local/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/home/sean/.local/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/usr/lib/python3.5/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/usr/lib/python3.5/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 693, in _load
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/gpuarray/tests/test_blas.py", line 13, in <module>
    from .config import mode_with_gpu
  File "/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/gpuarray/tests/config.py", line 10, in <module>
    theano.sandbox.gpuarray.init_dev('cuda')
  File "/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/gpuarray/__init__.py", line 46, in init_dev
    "Make sure Theano and libgpuarray/pygpu "
RuntimeError: ('Wrong major API version for gpuarray:', 0, 'Make sure Theano and libgpuarray/pygpu are in sync.')

======================================================================
ERROR: Failure: RuntimeError (('Wrong major API version for gpuarray:', 0, 'Make sure Theano and libgpuarray/pygpu are in sync.'))
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/sean/.local/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
    raise self.exc_val.with_traceback(self.tb)
  File "/home/sean/.local/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/home/sean/.local/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/home/sean/.local/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/usr/lib/python3.5/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/usr/lib/python3.5/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 693, in _load
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/gpuarray/tests/test_dnn.py", line 19, in <module>
    from .config import mode_with_gpu, mode_without_gpu, test_ctx_name
  File "/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/gpuarray/tests/config.py", line 10, in <module>
    theano.sandbox.gpuarray.init_dev('cuda')
  File "/home/sean/.local/lib/python3.5/site-packages/theano/sandbox/gpuarray/__init__.py", line 46, in init_dev
    "Make sure Theano and libgpuarray/pygpu "
RuntimeError: ('Wrong major API version for gpuarray:', 0, 'Make sure Theano and libgpuarray/pygpu are in sync.')

Given the syntax-like errors, I'm guessing it's a compiler issue of some sort, or perhaps Theano not knowing where to find my GCC or G++, but rather than mucking about aimlessly, I was wondering if anyone could shed some light on this.

Sean Corbett
  • 39
  • 2
  • 5
  • 2
    CUDA 8.0 is generally incompatible with GCC 6.x compilers. The compatibility and support is outlined in the [linux install guide for CUDA 8](http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#system-requirements). The 5.3.x compilers that ship with some versions of Ubuntu 16.04 are compatible. Take a look [here](http://stackoverflow.com/questions/41157739/compiling-with-nvcc-causes-errors-with-c-includes-path-headers) for example. I think your question is basically a duplicate of that and others. – Robert Crovella Dec 28 '16 at 22:34
  • your problem is related to your Cuda version. – Feras Dec 28 '16 at 22:49
  • @feras: no the problem is related to the gcc version. – talonmies Dec 29 '16 at 07:42
  • 1
    http://stackoverflow.com/q/6622454/681865 – talonmies Dec 29 '16 at 08:52

1 Answers1

1

I've installed all prerequisites as per the instructions.

Except that you haven't. If you consult the installation documentation for CUDA 8, you will see that GCC 6 is not currently supported. You will need to install a supported compiler version if you wish to compile code with the CUDA toolchain (which is apparently a prerequisite for running Theano).

You might also want to consult this question, which outlines a number of different strategies for managing and using an alternate compiler version for the CUDA toolkit.

Community
  • 1
  • 1
talonmies
  • 70,661
  • 34
  • 192
  • 269