4

I am iterating using prange over a list like this:

from cython.parallel import  prange, threadid

cdef int tid
cdef CythonElement tEl
cdef int a, b, c

# elList: python list of CythonElement instances is passed via function call
for n in prange(nElements, schedule='dynamic', nogil=True):
    with gil:
        tEl = elList[n]
        tid =  threadid()
        a = tEl.a
        b = tEl.b
        c = tEl.c 

        print("thread {:} elnumber {:}".format(tid, tEl.elNumber))

   #nothing is done here

    with gil:
        print("thread {:} elnumber {:}".format(tid, tEl.elNumber))

    # some other computations based on a, b and c here ...

I expect an output like this:

thread 0 elnumber 1
thread 1 elnumber 2
thread 2 elnumber 3
thread 3 elnumber 4
thread 0 elnumber 1
thread 1 elnumber 2
thread 2 elnumber 3
thread 3 elnumber 4

But i get:

thread 1 elnumber 1
thread 0 elnumber 3
thread 3 elnumber 2
thread 2 elnumber 4
thread 3 elnumber 4
thread 1 elnumber 2
thread 0 elnumber 4
thread 2 elnumber 4

So, somehow the thread local variable tEl becomes overwritten across the threads? What am i doing wrong ? Thank you!

mneuner
  • 433
  • 5
  • 25
  • You're right. It doesn't seem to be making `tEl` thread-local (have a look at the generated C file, and search for `lastprivate` to check). If you change it to a basic type (like `double`) it works but doesn't seem to with Cython object types. I don't know an obvious solution, but it might be worth filing a bug on github – DavidW Feb 09 '17 at 20:29

2 Answers2

5

It looks like Cython deliberately chooses to exclude any Python variables (including Cython cdef classes) from the list of thread-local variables. Code

I suspect this is deliberate to avoid reference counting issues - they'd need to drop the reference count of all the thread-local variables at the end of the loop (it wouldn't be an insurmountable problem, but might be a big change). Therefore I think it's unlikely to be fixed, but a documentation update might be helpful.

The solution is to refactorise your loop body into a function, where every variable ends up effectively "local" to the function so that it isn't an issue:

cdef f(CythonElement tEl):
    cdef int tid
    with nogil:
        tid = threadid()
        with gil:
            print("thread {:} elnumber {:}".format(tid, tEl.elNumber))

        with gil:
            print("thread {:} elnumber {:}".format(tid, tEl.elNumber))

   # I've trimmed the function a bit for the sake of being testable

# then for the loop:
for n in prange(nElements, schedule='dynamic', nogil=True):
    with gil:
        f()
DavidW
  • 29,336
  • 6
  • 55
  • 86
  • Ok, i understand. Unfortunately, it seems that parallelized calling of cdef extensions is just not possible. Refactoring to pure cdef function calls is not realistic at the moment. In your example, f() is declared without the nogil flag, and thus, is not executed parallely? – mneuner Feb 10 '17 at 07:45
  • It needs the gil at the start and end if the call to refcount tEl. It can release the gil inside and that bit will run in parallel. It should be no worse than the code in your question (obviously in this toy example there almost no "gil-free" work). – DavidW Feb 10 '17 at 08:05
0

Cython provides parallelism based on threads. The order in which threads are executed is not guaranteed, hence the disordered values for thread.

If you want tEl to be private to the thread, you should not define it globally. Try moving cdef CythonElement tEl within the prange. see http://cython-devel.python.narkive.com/atEB3yrQ/openmp-thread-private-variable-not-recognized-bug-report-discussion (part on private variables).

Pierre de Buyl
  • 7,074
  • 2
  • 16
  • 22
  • Thanks for the quick response. However, compiling gives me: >>> cdef statement not allowed here <<< ( This issue is also mentioned in the posted link?) – mneuner Feb 09 '17 at 19:38
  • Ok, I had a number of tabs open, I can't find the place where I had seen the cdef moved. Re-reading your example, I don't see a function. Typically, cython files are compiled to modules and the resulting functions are called. Is the code above the full file? – Pierre de Buyl Feb 09 '17 at 19:47
  • It was in the link actually. Search "cdef w = Worker(n) # block-scoped cdef" – Pierre de Buyl Feb 09 '17 at 19:48
  • tEl is an instance of a cython extension (cdef class). One of its nogil methods is called afterwards, but this is independent of the posted issue. Basically, this snippet produces the error. – mneuner Feb 09 '17 at 19:51
  • >>> It was in the link actually. Search "cdef w = Worker(n) # block-scoped cdef <<< Exactly, but 5 lines below this line it is stated that this is not legal? – mneuner Feb 09 '17 at 19:55
  • I am trying to run the code to debug it but it is not complete. Can you provide a full pyx file and a runner script? – Pierre de Buyl Feb 09 '17 at 20:15
  • A first test shows that even `tid` is not preserved across `with gil` blocks. – Pierre de Buyl Feb 09 '17 at 20:27
  • Sorry, I got carried out. I would advise to stick to something closer to the examples show in the docs of Cython. – Pierre de Buyl Feb 09 '17 at 20:36