1

I have to compute the smallest magnitude eigenvalue and it's associated eigenvector of a non symmetric matrix using PySpark libraries.

The size of is very high and I want the computation to be distributed among the cluster's workers.

The problem is that i didn't find any API to compute eigenvalues in PySpark 2.3 documentation.

I have identified two paths, but I want to avoid them:

  1. to reimplement eigen value decomposition trough QR algorithm using QRDecomposition available in PySpark API
  2. to compute eigen value decomposition trough scala version class as described in this question on Stack Overflow

Is there a simpler or better way then this last two?

I already know the existence of this post, but they are conceptually different.

Gennaro
  • 138
  • 1
  • 2
  • 16
  • Why you marked as duplicate? I don't think it is a duplicate question. Can you explain to me how compute the smallest eigenvalue and its associated eigenvector thorugh PCA? – Gennaro Jun 09 '18 at 12:31
  • I'm having troubles understanding your requirements. Have you read all the answers ? It's not always just about the accepted answer. Did you read [this](https://stackoverflow.com/a/33500704/3415409) per example ? – eliasah Jun 09 '18 at 13:32

0 Answers0