python multiprocessing for euclidean distance loop

Question

i have euclidean distance code but its running very slow since the data is big. can i get faster with python multiprocessing ?

this is my code

def euclidean_distance(self,x1, x2):
    distance = 0.0
    for i in range(len(x1)):
        distance += pow( x1[i] - x2[i], 2)
    return math.sqrt(distance)

how to use multipocessing pool ? can i go with pool.map for this function ?

#multiprocessing
pool = multiprocessing.pool(processes=2)
        dist = pool.map(self.euclidean_distance(test.vector,point.vector),range(len(x1))

it doesnt work since range(len(x1) is outside def. any help appreciated for quicken that function, thanks

https://stackoverflow.com/questions/1401712/how-can-the-euclidean-distance-be-calculated-with-numpy — Burhan Khalid, Aug 06 '17 at 06:38
What are `test` and `point` in your example code? I'm a bit doubtful that using `multiprocessing` can help you with this code since the amount of work you do for each function call is very small. I'd expect the overhead of serializing and copying the data between processes would be much more than the cost of doing the actual calculation in the main process. — Blckknght, Aug 06 '17 at 06:38
How long are these vectors? As Burhan suggest, you should try to speed things up with Numpy before you try multiprocessing, since the overheads of using multiprocessing probably outweigh its benefits. — PM 2Ring, Aug 06 '17 at 06:39
its an array, i am calculating distance between array, okay thanks — Elucist, Aug 06 '17 at 06:40
Yes, we understand that you're trying to get the euclidean distance between pairs of 1D arrays. But you haven't told us how many elements are in each of these arrays. To see how to do this using Numpy please take a look at the link that Burhan Khalid posted in the top comment. — PM 2Ring, Aug 06 '17 at 06:43
If Burhan Khalid's link answers your question we can close this one as a duplicate. — PM 2Ring, Aug 06 '17 at 06:44

python multiprocessing for euclidean distance loop

0 Answers0