3

I am curious as to why the following cdist differ so much in time even though they produce the same results:

import numpy as np
from scipy.spatial.distance import cdist

x = np.random.rand(10_000_000, 50)
y = np.random.rand(50)

result_1 = cdist(x, y[np.newaxis, :])

result_2 = cdist(x, y[np.newaxis, :], `minkowski`, p=2.)

The result_1 is significantly faster than result_2.

slaw
  • 6,591
  • 16
  • 56
  • 109
  • 1
    Checking from [source, line 614](https://github.com/scipy/scipy/blob/v1.5.2/scipy/spatial/distance.py#L2403-L2784) you can see that the (default) euclidean distance calls `minkowski(u, v, p=2, w=w)`. Hence I am following this question. – FBruzzesi Sep 24 '20 at 07:46

1 Answers1

3

The C implementation of the Euclidean distance, source lines 50-66, uses multiplication and a sqrt() call while the Minkowski distance, source lines 381-391 is based on the much slower calls to the pow() function.

For reference, see discussion here and here comparing pow to multiplication and sqrt.

So despite the appearance that the Euclidean norm just calls the Minkowski norm, source line 614, cdist actually calls directly through to the C implementation where the code is different. The python euclidean function is not called in the actual execution.

Alex
  • 1,042
  • 1
  • 8
  • 17