Principal Component Analysis(PCA) understanding document
0 likes•651 views
PCA is applied to reduce a dataset into fewer dimensions while retaining most of the variation in the data. It works by calculating the covariance matrix of the data and extracting eigenvectors with the highest eigenvalues, which become the principal components. The EJML Java library can be used to perform PCA by adding sample data, computing the basis using eigenvectors, and projecting samples into the reduced eigenvector space. PCA is generally not useful for datasets containing mostly 0s and 1s, as such sparse data is already in a compact format.
![Calculate the covariance matrix :
ccv X Y
X 0.61655556 0.615444444
Y 0.615444444 0.71655556
Calculate the eigenvalues and the eigenvectors.
eigenvalues
0.490833989
1.28402771
eigenvector 1 eigenvector 2
-0.735178656 -0.677873399
0.677873399 -0.735178656
The eigenvector with the highest eigenvalue is the principal component of the data set.
Once eigenvectors are found from the covariance matrix, the next step is to order them by
eigenvalue, highest to lowest. This gives you the components in order of significance. Now, if you
like, you can decide to ignore the components of lesser significance. You do lose some information,
but if the eigenvalues are small, you don’t lose much. If you leave out some components, the final
data set will have less dimensions than the original. To be precise, if you originally have dimensions
in your data, and so you calculate eigenvectors and eigenvalues, and then you choose only the first
eigenvectors, then the final data set has only dimensions.
FeatureVector = [eig1 eig2 eig3.....]
eigenvector 1 eigenvector 2
-0.677873399 -0.735178656
0.735178656 -0.677873399
We can choose to leave out the smaller, less significant component and only have a single column:
eigenvector 1
-0.677873399](https://image.slidesharecdn.com/pcaunderstandingdocument-160613230230/85/Principal-Component-Analysis-PCA-understanding-document-2-320.jpg)
![Here we can commercially use efficient-java-matrix-library: (EJML) :
Explaining EJML :
Here is the code which you can use after adding ejml jar to the classpath :
https://code.google.com/p/efficient-java-matrix-library/wiki/PrincipleComponentAnalysisExample
We can write a test component class for this class.
Process:
1. First we have to provide all the data sample by
pca.addSample(sample);
2. Then we have to call : pca.computeBasis(n);
It actually is the main component , here n is the number to which we want our feature to reduce to.
3. Now we can use eigen vectors created to actually get values using function : sampleToEigenSpace(
double[] sampleData )
Points to Note :
PCA will not be pretty useful with the data having 0's and 1's as the data having this feature can be
easily converted to sparse matrix format which will automatically reduces your memory req.
The PCA o/p will never be useful to convert it into sparse matrix format as it will not contain 0's .
So its better not to use PCA if we have data having 0's and 1's.
(We didn't got any java library to give sparse matrix as input format to PCA)
Links :
http://www.cs.otago.ac.nz/cosc453/student_tutorials/principal_components.pdf](https://image.slidesharecdn.com/pcaunderstandingdocument-160613230230/85/Principal-Component-Analysis-PCA-understanding-document-5-320.jpg)
Principal Component Analysis(PCA) understanding document
- 1. PCA Understanding Document Theory : Let the data points be the following on which PCA will be applied. X Y 2.5 2.4 0.5 0.7 2.2 2.9 1.9 2.2 3.1 3.0 2.3 2.7 2 1.6 1 1.1 1.5 1.6 1.1 0.9 Subtract the mean from the dataset from each of the individual axes.The modified dataset is : X Y .69 .49 -1.31 -1.21 .39 .99 .09 .29 1.29 1.09 .49 79 .19 -.31 -.81 -.81 -.31 -.31 -.71 -1.01
- 2. Calculate the covariance matrix : ccv X Y X 0.61655556 0.615444444 Y 0.615444444 0.71655556 Calculate the eigenvalues and the eigenvectors. eigenvalues 0.490833989 1.28402771 eigenvector 1 eigenvector 2 -0.735178656 -0.677873399 0.677873399 -0.735178656 The eigenvector with the highest eigenvalue is the principal component of the data set. Once eigenvectors are found from the covariance matrix, the next step is to order them by eigenvalue, highest to lowest. This gives you the components in order of significance. Now, if you like, you can decide to ignore the components of lesser significance. You do lose some information, but if the eigenvalues are small, you don’t lose much. If you leave out some components, the final data set will have less dimensions than the original. To be precise, if you originally have dimensions in your data, and so you calculate eigenvectors and eigenvalues, and then you choose only the first eigenvectors, then the final data set has only dimensions. FeatureVector = [eig1 eig2 eig3.....] eigenvector 1 eigenvector 2 -0.677873399 -0.735178656 0.735178656 -0.677873399 We can choose to leave out the smaller, less significant component and only have a single column: eigenvector 1 -0.677873399
- 3. 0.735178656 FinalData = RowFeatureVector * RowDataAdjust where RowFeatureVector is the matrix with the eigenvectors in the columns transposed so that the eigenvectors are now in the rows, with the most significant eigenvector at the top and RowDataAdjust is the mean-adjusted data transposed ie. the data items are in each column, with each row holding a separate dimension. X Y -.827970186 -.175115307 1.77758033 .142857227 -.992197494 .384374989 -.274210416 .130417207 -1.67580142 -.209498461 -.912949103 .175282444 .0991094375 -.349824698 1.14457216 .0464172582 .438046137 .0177646297 1.22382056 -.162675287 Transformed Data (Single eigenvector) X -.827970186 1.77758033 -.992197494 -.274210416 -1.67580142 -.912949103 .0991094375 1.14457216
- 4. .438046137 1.22382056 Eg . We have n features FinalData = SampleData(1*n matrix) * Eigen Vector (n*1 matrix) = 1*1 Matrix i.e( 1st eigen vector of n eigen vectors which are in descending order according to its eigen values is used to get 1st value for features after PCA execution.) To get the Final Data : FinalData = RowFeatureVector * RowDataAdjust Getting back old Data : RowDataAdjust = RowFeatureVector^(-1) * Final Data Java Library : 1 . java-statistical-analysis-tool: (JSAT) https://code.google.com/p/java-statistical-analysis- tool/source/browse/trunk/JSAT/src/jsat/datatransform/PCA.java?spec=svn414&r=414 License : GNU GPL v3 2. efficient-java-matrix-library: (EJML) https://code.google.com/p/efficient-java-matrix-library/wiki/PrincipleComponentAnalysisExample License : GNU Lesser GPL 3. Michael Thomas Flanagan's Java Scientific Library http://www.ee.ucl.ac.uk/~mflanaga/java/PCA.html License : This library is no longer publicly available
- 5. Here we can commercially use efficient-java-matrix-library: (EJML) : Explaining EJML : Here is the code which you can use after adding ejml jar to the classpath : https://code.google.com/p/efficient-java-matrix-library/wiki/PrincipleComponentAnalysisExample We can write a test component class for this class. Process: 1. First we have to provide all the data sample by pca.addSample(sample); 2. Then we have to call : pca.computeBasis(n); It actually is the main component , here n is the number to which we want our feature to reduce to. 3. Now we can use eigen vectors created to actually get values using function : sampleToEigenSpace( double[] sampleData ) Points to Note : PCA will not be pretty useful with the data having 0's and 1's as the data having this feature can be easily converted to sparse matrix format which will automatically reduces your memory req. The PCA o/p will never be useful to convert it into sparse matrix format as it will not contain 0's . So its better not to use PCA if we have data having 0's and 1's. (We didn't got any java library to give sparse matrix as input format to PCA) Links : http://www.cs.otago.ac.nz/cosc453/student_tutorials/principal_components.pdf