0

This set of exercises has the student use a QP solver to solve an SVM in R. The suggested solver is the quadprog package. The quadratic problem is given as:

enter image description here

From the remark about the linear SVM, $K=XX'$, $K$ is a singular matrix usually, at most rank $p$ where $X$ is $n\times p$. But the solver quadprog requires a positive definite matrix, not just PSD, in the place of $K$, as mentioned many places (and verified). Any ideas what the instructor had in mind?

Hasse1987
  • 243
  • 1
  • 9
  • Does this answer the question? https://stackoverflow.com/questions/33088363/how-to-use-r-package-quadprog-to-solve-svm – IRTFM Nov 13 '18 at 07:29

1 Answers1

0

I think the workaround would be to add a small number (such as 1e-7) to the diagonal elements of the matrix which is supposed to be positive definite. I am not certain about the math behind it, but the sources below, as well as my experience, suggest that this solution works.

source: https://stats.stackexchange.com/questions/179900/optimizing-a-support-vector-machine-with-quadratic-programming

source: https://teazrq.github.io/stat542/hw/HW6.pdf

Jariani
  • 438
  • 4
  • 6