Principal Component Analysis using libpca

Question

libpca is a C++ library for Principal Component Analysis that builds upon Armadillo, a linear algebra library.

I am having a problem with it, though. I am comparing its output with the example given by Lindsay Smith in his great tutorial on PCA. When I retrieve the first principal component, I get the same values as Smith in his tutorial but with its signs inverted. For the second principal component the signs and values are correct.

Anyone know why this is?

Code:

#include "pca.h"
#include <iostream>

using namespace std;

int main(int argc, char** argv) {
    stats::pca pca(2);

    double* elements = new double[20]{2.5, 2.4, 0.5, 0.7, 2.2, 2.9, 1.9, 2.2, 3.1, 3.0, 2.3, 2.7, 2, 1.6, 1, 1.1, 1.5, 1.6, 1.1, 0.9};
    for (int i = 0; i < 20; i++) {
        vector<double> record;
        record.push_back(elements[i++]);
        record.push_back(elements[i]);
        pca.add_record(record);
    }

    pca.solve();             

    const vector<double> principal_1 = pca.get_principal(0);
    for (int i = 0; i < principal_1.size(); i++)
        cout << principal_1[i] << " ";
    cout << endl;

    const vector<double> principal_2 = pca.get_principal(1);
    for (int i = 0; i < principal_2.size(); i++)
        cout << principal_2[i] << " ";
    cout << endl;

    delete elements;
    return 0;
}

Output:

0.82797 -1.77758 0.992197 0.27421 1.6758 0.912949 -0.0991094 -1.14457 -0.438046 -1.22382 
-0.175115 0.142857 0.384375 0.130417 -0.209498 0.175282 -0.349825 0.0464173 0.0177646 -0.162675

in most cases only the magnitude of the principal components matters — mtall, Sep 05 '14 at 04:46

score 5 · Accepted Answer · answered Sep 05 '14 at 07:34

5

@mtall already has the core reason: The principal components form a normal basis of a subspace. Regardless of how you created a basis, multiplying any basisvector by -1 forms another basis of the same subspace.

This is fairly easy to see: Multiplying a vector v by any constant does not change its direction. if v is normal to w, then 2*v is normal to 3*w. Multiplying a vector by -1 reverses its direction. If v and w had an angle alpha, then -v and w have an angle (pi - alpha). But if alpha was pi/2, v and w are normal, (pi-pi/2) is still pi/2 and thus -v and w are also normal.

answered Sep 05 '14 at 07:34

MSalters

173,980
10
155
350

Thanks for clearing that up. Follow-up question: if I wanted all vectors to have only positive values, how could I transform them to do that while keeping their relations, so, say distance metrics kept their distances? – kunterbunt Sep 05 '14 at 11:24
2

@randomdude: That doesn't make sense at all. You can multiply a vector by -1, but that multiplies all components. For instance, your PCA could return both [2,2] and [-1,1]. That's two orthogonal vectors. You can multiply the latter to get [1,-1] but there's no way you can sensibly transform the latter to [1,1]. That would be colinear with, not orthogonal to [2,2] – MSalters Sep 05 '14 at 12:59

Principal Component Analysis using libpca

1 Answers1