2

I have the following code and since it is very slow for larger l i am asking myself if there is an easy possibility to parallelize this loop. I also tried to parallize it by my own, but because I never did it before I am not able to manage it. I would be happy if you can help me out

print('Create 2.0L%.3frec%.3f.npy' % (l, r))
for x1 in range(a**l):
    for x2 in range(a**l):
        for x3 in range(a**l):
            f11 = 0
            if numpy.ndarray.sum(numpy.absolute(numpy.subtract(ws[x1:x1+1], ws[x2:x2+1]))) == 0:
                f11 += 1
            if numpy.ndarray.sum(numpy.absolute(numpy.subtract(ws[x1:x1+1], ws[x3:x3+1]))) == 0:
                f11 += 1
            re[x1][x2][x3] = 1.0*r/(a**l-2)*(numpy.product(numpy.absolute(numpy.subtract((2*ws[x1:x1+1]+ws[x2:x2+1]+ws[x3:x3+1]), 2)))-f11)

            cv1 = numpy.ndarray.sum(numpy.absolute(numpy.subtract(ws[x1:x1+1], ws[x2:x2+1])))
            cv2 = numpy.ndarray.sum(numpy.absolute(numpy.subtract(ws[x1:x1+1], ws[x3:x3+1])))
            c2 = 0
            if cv1 == 0:
                c2 += 1
            if cv2 == 0:
                c2 += 1
            c2 *= 1.0*(1-r)/2
            re[x1][x2][x3] += c2
numpy.save('2.0L%.3frec%.3f' % (l, r), re)
print('Saved 2.0L%.3frec%.3f.npy' % (l, r))

So since all entries of re are independent of the others, there should be a way. I guess I would be helped if I know a solution how to parllize a python program like:

for x1 in range(a):
    for x2 in range(a):
        for x3 in range(a):
            re[x1][x2][x3] = 5*3
HighwayJohn
  • 881
  • 1
  • 9
  • 22

1 Answers1

2

I don't fully understand what you are trying to calculate, but I'll give it a shot. For your latter question, you can do this as follows:

re = np.empty([a]*3) 
x = np.indices([a]*3)
re[x] = 5*3

You can vectorize your code as follows: 

x = np.indices([a**l]*3)
cv1 = (ws[x[0]] == ws[x[1]]).astype(float)
cv2 = (ws[x[0]] == ws[x[2]]).astype(float)
f11 = cv1 + cv2
re = 1.0*r/(a**l-2)*(np.absolute(2*ws[x[0]]+ws[x[1]]+ws[x[2]]-2)-f11)    
re += f11*1.0*(1-r)/2

(I removed superfluous sums and products, and used the == operator to check for equal values, and used that c2 and f11 are the same thing). In theory this should do the same as your code.

Forzaa
  • 1,465
  • 4
  • 15
  • 27
  • That looks really nice, I will try it out soon :) – HighwayJohn Apr 27 '16 at 12:34
  • Traceback (most recent call last): File "/2.3.1.py", line 113, in wp[:,0] = numpy.tensordot(numpy.tensordot(re, wp[:,0], axes=1), wp[:,0], axes=1) File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/numpy/core/numeric.py", line 1246, in tensordot if as_[axes_a[k]] != bs[axes_b[k]]: IndexError: tuple index out of range I am getting this error now. Apparently the re tensor has now a different size? – HighwayJohn Apr 29 '16 at 00:22
  • It's shape should still be the same as before, namely `(a**l,a**l,a**l)`. – Forzaa Apr 29 '16 at 07:47
  • Does it have to be re[x] ? – HighwayJohn Apr 29 '16 at 09:13
  • I tried it on an example `a=2`,`l=3`, `r=0.3` and `ws = np.ones(a**l)`. There it returns the same answer as your code (namely all entries of re are 0.7). – Forzaa Apr 29 '16 at 11:46
  • Maybe next time specify `ws` and the expected output so it is easier to test. Most of the ideas for vectorizing your code are there, so I leave the rest to you. – Forzaa May 09 '16 at 08:47
  • You are right..I will do it better next time. The problem is solved now here: http://stackoverflow.com/questions/37081288/performance-issue-in-python-with-nested-loop Nevertheless, thank you for the effort! – HighwayJohn May 09 '16 at 09:10