in scipy.optimize.minimize - could not be broadcast together data with shapes (2,) (2,5)

Question

I cannot cope with shapes mismatch in scipy.optimize.minimize here in this example:

import numpy as np
from scipy.spatial import distance_matrix
from scipy.optimize import minimize

# Create the matrices
X = np.array([[1,2,3,4,5],[2,1,0,3,4]])
y = np.array([[0,0,0,0,1],[1,1,1,1,0]])

# Display the matrices
print("matrix x:\n", X)
print("matrix y:\n", y)

# compute the distance matrix
dist_mat = distance_matrix(X, y, p=2)

# display distance matrix
print("Distance Matrix:\n", dist_mat)

loss_res = lambda z: 0.5 * z ** 2 * (np.abs(z) <= 1) + (np.abs(z) - 0.5) * (np.abs(z) > 1)
f_to_optMin = lambda w: np.sum(loss_res(X @ w.ravel() - y)) # (2,) 10, (2,5)

res= minimize(f_to_optMin, (10,10,10,10,10,))
print(res.x)

seems, that problem/solution is rather simple, but I don't see how to make it working. Can somebody help me, please?

Traceback (most recent call last):
  File "E:\NEW docs\dist_mat.py", line 33, in <module>
    res= minimize(f_to_optMin, ([1,1,1,1,1,]))
  File "C:\Users\adm\AppData\Local\Programs\Python\Python310-32\lib\site-packages\scipy\optimize\_minimize.py", line 618, in minimize
    return _minimize_bfgs(fun, x0, args, jac, callback, **options)
  File "C:\Users\adm\AppData\Local\Programs\Python\Python310-32\lib\site-packages\scipy\optimize\optimize.py", line 1201, in _minimize_bfgs
    sf = _prepare_scalar_function(fun, x0, jac, args=args, epsilon=eps,
  File "C:\Users\adm\AppData\Local\Programs\Python\Python310-32\lib\site-packages\scipy\optimize\optimize.py", line 261, in _prepare_scalar_function
    sf = ScalarFunction(fun, x0, args, grad, hess,
  File "C:\Users\adm\AppData\Local\Programs\Python\Python310-32\lib\site-packages\scipy\optimize\_differentiable_functions.py", line 140, in __init__
    self._update_fun()
  File "C:\Users\adm\AppData\Local\Programs\Python\Python310-32\lib\site-packages\scipy\optimize\_differentiable_functions.py", line 233, in _update_fun
    self._update_fun_impl()
  File "C:\Users\adm\AppData\Local\Programs\Python\Python310-32\lib\site-packages\scipy\optimize\_differentiable_functions.py", line 137, in update_fun
    self.f = fun_wrapped(self.x)
  File "C:\Users\adm\AppData\Local\Programs\Python\Python310-32\lib\site-packages\scipy\optimize\_differentiable_functions.py", line 134, in fun_wrapped
    return fun(np.copy(x), *args)
  File "E:\NEW docs\dist_mat.py", line 31, in <lambda>
    f_to_optMin = lambda w: np.sum(loss_res(X @ w.ravel() - y)) # 2, 10, [2,5]
ValueError: operands could not be broadcast together with shapes (2,) (2,5)

added... error arises in Sum of Squared Errors func `f_to_optMin` — JeeyCi, Aug 15 '23 at 12:11
`X` is already the same shape as `y`, so what is your intention by `X @ w`? (Ignoring the `.ravel()` part, since `w` is 1-dimensional already.) — 9769953, Aug 15 '23 at 12:35
intention was to model multidimensional multivariate/multioutput/multilabel Classification (I corrected - changed 2 to 0 in one place of y) - if it is possible, though sounds complicated (everywhere multi - but not this is the sence) & sample is small, but logics is needed in the code as an example of *Distance Metric Solutions* — JeeyCi, Aug 15 '23 at 12:40
well, to be more precise: not classification, but Clustering as of distance metric solution (minimized sum of squared errors ) — JeeyCi, Aug 15 '23 at 12:54
I recognized the incorrectness of my solution - because Clustering is Unsupervised & cannot be projected to y here. As it, I suppose, ought to be Model-free, not Model-based solution. Is scipy.optimize.minimize inappropriate for finding min sum of Euclidean distances? probably, yes?.. and will the sollution be the center of any of binary class ? perhaps not... I'm stucked a little bit with the applicability of `minimized sum of Euclidean distances` in real tasks! what could be good aim to use it for (in any usefull example) ? and if this problem could be stated as Covex optimization problem ? — JeeyCi, Aug 15 '23 at 15:24
and besides, if to use [K-means](https://stackoverflow.com/a/53641024/15893581). Though I previously thought like "K-means optimizes the (sum of within-cluster-) sum of squares aka variance aka sum of squared Euclidean distances." — JeeyCi, Aug 15 '23 at 16:49
.. [geometrical median to minimize Euclidean distances.](https://stackoverflow.com/a/57277493/15893581) with scipy.optimize.minimize - is it possible ? — JeeyCi, Aug 15 '23 at 18:02

score 2 · Accepted Answer · answered Aug 15 '23 at 12:32

2

The problem is with

X @ w.ravel() - y

The left side of the subtraction has shape (2) while the right has shape (2,5). To reconcile that, according to the numpy broadcasting rules, you could replace y by its transpose, and have the subtraction applied by rows. I tested it:

import numpy as np
from scipy.spatial import distance_matrix
from scipy.optimize import minimize

# Create the matrices
X = np.array([[1,2,3,4,5],[2,1,0,3,4]])
y = np.array([[0,0,0,0,1],[1,1,1,1,2]])

# Display the matrices
print("matrix x:\n", X)
print("matrix y:\n", y)

# compute the distance matrix
dist_mat = distance_matrix(X, y, p=2)

# display distance matrix
print("Distance Matrix:\n", dist_mat)

loss_res = lambda z: 0.5 * z ** 2 * (np.abs(z) <= 1) + (np.abs(z) - 0.5) * (np.abs(z) > 1)
f_to_optMin = lambda w: np.sum(loss_res(X @ w.ravel() - y.T)) # (2,) 10, (2,5)

res= minimize(f_to_optMin, (1,1,1,1,1,))
print(res.x) # [ 0.91638545  0.41921018 -0.07796579 -0.0237285  -0.2451989 ]

and it seems to work. Not knowing what you are trying to solve, it's impossible to know if this is what you want or need though.

answered Aug 15 '23 at 12:32

Learning is a mess

7,479
7
35
71

1

oh, really - I wanted to solve for minimum of SSE of Euclidean_distances, but used X, y instead of distances - that's my fault. But for corrections with Transpose - Thanks! – JeeyCi Aug 15 '23 at 12:36
does your answer means that we need to broadcast (2,) to (5,2) shapes instead of my faulty (2,) to (2,5) ? – JeeyCi Aug 15 '23 at 12:47
1

Yeah that's pretty much what it means – Learning is a mess Aug 15 '23 at 12:50
1

yes, and I probably was incorrect in Loss (Huber) & Metric used here, more suitable for *regression* problem, than my over-complicated attempt to try the *clustering* problem of minimizing spatial_distances Withing-Group & maximizing spatial_distances Between-Group – JeeyCi Aug 15 '23 at 16:58
OFF: multi-class *classification* - possible [softmax loss_fx](https://gist.github.com/tsaith/290471a7ef508850139a) algo – JeeyCi Aug 16 '23 at 08:42
OFF: [minimize(logistic_loss...)](https://github.com/UBC-MDS/DSCI_572_sup-learn-2/blob/master/lectures/lecture2_gradient-descent.ipynb) – JeeyCi Aug 20 '23 at 17:55
1

P.S. [clustering-algorithms](https://neptune.ai/blog/clustering-algorithms) – JeeyCi Aug 24 '23 at 13:58

in scipy.optimize.minimize - could not be broadcast together data with shapes (2,) (2,5)

1 Answers1