0

I want to use a dictionary inside nogil. I've read this question and the comments by @DavidW:

Cythonize python dictionary object

So I tried doing this in file.pyx:

# cython: language_level=3
cimport cpython.dict
cdef cpython.dict some_cdict

And my setup.py file looks like this:

from setuptools import setup
from Cython.Build import cythonize
import numpy as np

setup(
    ext_modules=cythonize('file.pyx', annotate=True),
    include_dirs=[np.get_include()]
)

And this is my call to compile it:

python setup.py build_ext --inplace

But I get this error:

file.pyx:3:5: 'dict' is not a type identifier

What am I doing wrong?

I also saw this solution but I would much rather use objects from CPython than libcpp:

https://stackoverflow.com/a/32267243/1884520

waynemystir
  • 323
  • 3
  • 11
  • cc commenter from the linked question: @DavidW – waynemystir May 27 '20 at 05:40
  • Does this answer your question? [Using a dictionary in Cython , especially inside nogil](https://stackoverflow.com/questions/32266444/using-a-dictionary-in-cython-especially-inside-nogil) – ead May 27 '20 at 08:42
  • 1) [`cpython.dict`](https://github.com/cython/cython/blob/master/Cython/Includes/cpython/dict.pxd) is a Cython .pxd file containing wrappers for the [C API interface for dict objects](https://docs.python.org/3/c-api/dict.html). It is not a datatype. 2) You can't use Python `dict` without the GIL since everything you're doing is manipulating Python objects! I've edited https://stackoverflow.com/a/32267243/1884520 because that obviously wasn't clear enough. – DavidW May 27 '20 at 09:15
  • Thank you @DavidW for clarifying this. And apologies for my misunderstanding. So the only way to use a dictionary with nogil is to use libcpp.map or unordered_map? If so, is there an alternative to prange for iterating in paraellel through a map or unordered_map? – waynemystir May 27 '20 at 16:23
  • Probably not the only way (you could always find another suitable C or C++ dictionary library to wrap). But definitely the easiest way. I can't think of a good way to iterate in parallel I'm afraid. – DavidW May 27 '20 at 16:35
  • Thanks again @DavidW. So I guess it's not possible to iterate through a dictionary in a multithreaded way in Python/Cython? – waynemystir May 27 '20 at 17:48
  • You _might_ be able to do the dictionary access in a `with gil:` block, and then do whatever processing you want to do with the data in a nogil section. But it'd be hard and depend a lot on what the data is. It may not be _impossible_ but there's nothing obvious I can recommend. – DavidW May 27 '20 at 18:09
  • OK, thank you @DavidW. When you get a moment, would you looking at this related question: https://stackoverflow.com/questions/62049533/is-it-possible-to-use-a-cython-extension-type-in-a-nogil-section – waynemystir May 27 '20 at 18:21

0 Answers0