1

+I'm trying to implement a matlab algorithm in C++.

This is the matlab code:

p = 3;
K = [3 4 5; 4 5 6; 7 8 9];
e = ones(p,1);
K2 = K - (1/p)*K*ones(p) - 1/p + (1/p^2)*sum(K(:))
[V_K,D_K] = eig(K2);

While this is the analogous C++ code using OpenCV:

float data[] = {3, 4, 5,
                4, 5, 6,
                7, 8, 9};
cv::Mat m(3, 3, CV_32F, data);
float p = K.rows;
cv::Mat CK = K - (1/p)*K*cv::Mat::ones(p,p,CV_32F) - 1/p + (1/std::pow(p,2))*cv::sum(K)[0];
cv::Mat eigenvalues(1,p,CK.type()), eigenvectors(p,p,CK.type());
cv::eigen(CK,eigenvalues,eigenvectors);

The matlab code print:

CK =

4.3333    5.3333    6.3333
4.3333    5.3333    6.3333
4.3333    5.3333    6.3333

0.5774    0.6100   -0.1960
0.5774   -0.7604   -0.6799
0.5774    0.2230    0.7066

16.0000         0         0
      0   -0.0000         0
      0         0    0.0000

While the C++ code produces:

CK=[4.3333335, 5.3333335, 6.3333335;
 4.3333335, 5.3333335, 6.3333335;
 4.333333, 5.333333, 6.333333]

eigenvectors=[0.53452265, 0.56521076, 0.62834883;
 -0.41672006, 0.8230716, -0.38587254;
 0.73527533, 0.05558794, -0.67548501]

eigenvalues=[17.417906;
 -0.33612049;
 -1.0817847]

As you can see, the values are completely differents (even the ones of CK!). Why this happens and how can I avoid it?

Note that I'm not completely sure that my C++ implementation is correct!

I found this and this question related, but they seem related to slightly differences, while here the error is huge!

UPDATE:

I've tried to follow to suggestions in comments & answers. Unfortunately, none of the solution proposed solved the problem. First of all I tried to use the Eigen library with float precision. This is the code using the Eigen::Map structure as described here:

//in order to map OpenCV matrices to Eigen, they need to be continous
assert(CK.isContinuous());
Eigen::Map<Eigen::Matrix<float, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor>> CKEigenMapped (CK.ptr<float>(), CK.rows, CK.cols);
Eigen::Matrix<float, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor> CKEigen = CKEigenMapped;
Eigen::EigenSolver<Eigen::Matrix<float, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor>> es (CKEigen,true);
std::cout<<"Eigenvalues:"<<std::endl<< es.eigenvalues() << std::endl;
std::cout<<"Eigenvectors:"<<std::endl<< es.eigenvectors() << std::endl;

Then I tried to convert from float to double through CK.convertTo(CK, CV_64F):

//Double precision
CK.convertTo(CK, CV_64F);
Eigen::Map<Eigen::Matrix<double, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor>> CKEigenMappedD (CK.ptr<double>(), CK.rows, CK.cols);
Eigen::Matrix<double, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor> CKEigenD = CKEigenMappedD;
Eigen::EigenSolver<Eigen::Matrix<double, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor>> esD (CKEigenD,true);
std::cout<<"Eigenvalues:"<<std::endl<< esD.eigenvalues() << std::endl;
std::cout<<"Eigenvectors:"<<std::endl<< esD.eigenvectors() << std::endl;

Finally I tried to use the cv2eigen function (I thought that Eigen::Map could have been wrong) as described here:

//Double precision, cv2eigen
Eigen::MatrixXd X=Eigen::MatrixXd(CK.rows,CK.cols);
cv2eigen(CK,X);
Eigen::EigenSolver<Eigen::MatrixXd> esDD (X,true);
std::cout<<"Eigenvalues:"<<std::endl<< esDD.eigenvalues() << std::endl;
std::cout<<"Eigenvectors:"<<std::endl<< esDD.eigenvectors() << std::endl;

And these are the results corresponding to the 3 previous solutions:

Eigenvalues:
(-4.17233e-07,0)
          (16,0)
(-3.37175e-07,0)
Eigenvectors:
(-0.885296,0)   (0.57735,0)  (-0.88566,0)
 (0.328824,0)   (0.57735,0)  (0.277518,0)
 (0.328824,0)   (0.57735,0)  (0.372278,0)
Eigenvalues:
          (16,0)
  (8.9407e-08,0)
(-1.88417e-16,0)
Eigenvectors:
  (0.57735,0)  (0.480589,0)  (0.408248,0)
  (0.57735,0)  (0.480589,0) (-0.816497,0)
  (0.57735,0) (-0.733531,0)  (0.408248,0)
Eigenvalues:
          (16,0)
  (8.9407e-08,0)
(-1.88417e-16,0)
Eigenvectors:
  (0.57735,0)  (0.480589,0)  (0.408248,0)
  (0.57735,0)  (0.480589,0) (-0.816497,0)
  (0.57735,0) (-0.733531,0)  (0.408248,0)

As you can notice:

  1. None of them correspond to the Matlab results ( :'( )
  2. There is a difference between using double and float
  3. There is no difference between using Eigen::Map and cv2eigen

Please note that I'm not expert in Eigen and I could have used Eigen::EigenSolver in a wrong way.

UPDATE 2:

This is starting to be a mess! This is the code using Amradillo. Notice that A has the same values of K2 (CK in C++):

arma::mat A(3,3);
A   << 4.333333333333333 << 5.333333333333333 << 6.333333333333333 <<arma::endr
    << 4.333333333333333 << 5.333333333333333 << 6.333333333333333 <<arma::endr
    << 4.333333333333333 << 5.333333333333333 << 6.333333333333333 <<arma::endr;
arma::cx_vec eigval;
arma::cx_mat eigvec;
eig_gen(eigval,eigvec,A);
std::cout<<"eigval="<<std::endl<<eigval<<std::endl<<"eigvec="<<std::endl<<eigvec<<std::endl;

And these are the printed values:

eigval=
    (+1.600e+01,+0.000e+00)
    (-4.010e-17,+3.435e-16)
    (-4.010e-17,-3.435e-16)

eigvec=
    (+5.774e-01,+0.000e+00)    (-5.836e-02,+3.338e-01)    (-5.836e-02,-3.338e-01)
    (+5.774e-01,+0.000e+00)    (+7.174e-01,+0.000e+00)    (+7.174e-01,-0.000e+00)
    (+5.774e-01,+0.000e+00)    (-5.642e-01,-2.284e-01)    (-5.642e-01,+2.284e-01)

Seriously, what's wrong with all these libraries? They don't even closely agree with each other!

Community
  • 1
  • 1
justHelloWorld
  • 6,478
  • 8
  • 58
  • 138
  • 5
    Have you tried switching to `double`? Looks like it could be a precision or rounding error. – Thomas Matthews Jul 11 '16 at 14:34
  • I am not sure your C++ implementation is correct either, there may be estimation errors in there. – GameOfThrows Jul 11 '16 at 14:36
  • Which library is correct? (Look at the eigenvalues first). – Bathsheba Jul 11 '16 at 14:36
  • @Bathsheba the matlab code is the original one, so it's the correct one – justHelloWorld Jul 11 '16 at 14:37
  • @ThomasMatthews ouch, I was afraid that could be the problem...but this could be a problem since the OpenCV matrices that I'm using are in `float` :( I'll try to convert them to `double`, but I don't know if that's possible – justHelloWorld Jul 11 '16 at 14:38
  • Doubles in matlab are 64 bit Floats in C++. They both use the IEEE 754 format of encoding. – GameOfThrows Jul 11 '16 at 14:44
  • @ThomasMatthews if you run `eig(single(K2))` in MATLAB it will give practically the same result as it does now. Its not that. – Ander Biguri Jul 11 '16 at 14:44
  • Break it down to smaller checks. One line or subexpression at a time. Then it should be easy to see where the two diverge and what is wrong. – doug Jul 11 '16 at 14:50
  • 1
    @justHelloWorld - the Armadillo answer is correct. `+1.600e+01` is `16`. Small eigenvalues such as `(-4.010e-17,+3.435e-16)` are essentially zero. Any eigenvectors corresponding to zero eigenvalues are typically garbage. The garbage is dependent on the nature of each eigendecomposition algorithm. – hbrerkere Jul 12 '16 at 06:57
  • @justHelloWorld - further note: to check the validity of each answer, reconstruct the decomposed matrix and compare it to the original matrix. For example, `[v,l] = eig(K2); K3 = v*l*inv(v); error = norm(K2-K3)`. See a linear algebra textbook for more information. – hbrerkere Jul 12 '16 at 07:14
  • @hbrerkere thanks for your comment, I didn't know that eigenvectors relatives to small eigenvectors are garbage, thanks for the tip. Anyway, if that's case, also the results from `Eigen` are correct. The only weird thing is that for using the `float` precision (first of the three codes in **UPDATE**) the order of the eigenvalues (and relative eigenvectors) is different, while this doesn't happens with `double` precision. – justHelloWorld Jul 12 '16 at 07:32

1 Answers1

6

cv::eigen assumes that the input matrix is symetric, and yours is not. that is why the difference is there.

I believe openCV does not have support for eigenvectors of non-symmetric matrices, you may need to use another library.

Update: PCA (principal component analysis) is an eigenvector decomposition, so you may be able to go that way, but the best would be to use some specific math library, such as EIGEN or ARMADILLO.

Ander Biguri
  • 35,140
  • 11
  • 74
  • 120
  • 2
    See also the [eig_gen()](http://arma.sourceforge.net/docs.html#eig_gen) and [eig_sym()](http://arma.sourceforge.net/docs.html#eig_sym) functions in Armadillo. The eig_gen() function handles general (non-symmetric) matrices, while the eig_sym() is optimized for symmetric matrices. – hbrerkere Jul 11 '16 at 17:19
  • @justHelloWorld Updated – Ander Biguri Jul 12 '16 at 07:34
  • @justHelloWorld Let me look a bit further, but the results are quite close, as your eigenvalues are good in all, and *some* eigenvectors are different. – Ander Biguri Jul 12 '16 at 07:40
  • @AnderBiguri hbrerkere suggested (in the question comments) to ignore the eigenvectors relatives to near-zero eigenvalues, since they are garbage. In that case the results are the same for both `Eigen` and `Armadillo` libraries (even if in the `float` case for some weird the order of the eigenvalues is different). Do you agree? – justHelloWorld Jul 12 '16 at 07:42
  • 1
    @justHelloWorld oh damn, its too early in the morning. Indeed an eigenvector for a nearly zero eigenvalue is rubbish. The results you get in both Armadillo and Eigen are right. – Ander Biguri Jul 12 '16 at 07:48
  • @AnderBiguri I'm sorry dude, in HK the sun is going down already :D I hope to be in your same timezone when I get back home in italy ;) – justHelloWorld Jul 12 '16 at 07:50
  • @justHelloWorld haha ;) Happy to help. Consider accepting the answer if it worked – Ander Biguri Jul 12 '16 at 08:05