libPLS: an Integrated Library for Partial Least Squares Regression and Linear Discriminant Analysis
Featured in Model Population Aanlysis (MPA) approaches
Join our mailing list for getting updates.
NEW: deliver your comments/questions to my Blog.
Updates:
2. Customer prior is allowed in LDA to weigh each class of samples in version 1.98.
1. Elastic Component Regression(ECR) added in version 1.95.
1. Overview
This library provides a whole set of easy-to-use functions for building partial least squares (PLS) regression (PLSR) and discriminant analysis (PLS-DA) models as well as predictive performance evaluation. Towards building a reliable model, we also implemented a number of commonly used outlier detection and variable selection methods that can be used to "clean" your data by removing potential outliers and using only a sub-set of selected variables.
The algorithms in the current version cover:
| Types | Methods | Abbreviations | Codes | Notes |
|---|---|---|---|---|
| Data pretreat | mean-centering | pretreat.m | ||
| autoscaling | pretreat.m | |||
| Orthogonal Projection to Latent Structures | OPLS | opls.m | ||
| Orthogonal Signal Correctionof Tom Fearn | OSC | oscfearn.m | ||
| Orthogonal Signal Correction of Swante Wold | OSC | oscwold.m | ||
| Sample partition | Kennard-Stone algorithm | KS | ks.m | |
| Model building | Partial Least Squares | PLS | pls.m | |
| Linear Discriminant Analysis | LDA | ldapinv.m | ||
| Partial Least Squares-Linear Discriminant Analysis | PLS-DA | plslda.m | ||
| Elastic Component Regression | ECR | ecr.m | ||
| Model assessment | leave-one-out cross validation | LOOCV | plscv.m, plsldacv.m | |
| K-fold cross validation | K-fold CV | plscv.m, plsldacv.m, ecrcv.m | ||
| double cross validation | DCV | plsdcv.m, plsldadcv.m | ||
| Monte Carlo cross validation | MCCV | plsmccv.m, plsldamccv.m | ||
| Using an independent test set | ||||
| Outlier detection | The Monte Carlo method | mcs.m | ||
| Variable selection | Variable Importance in Projection | VIP | inside pls.m or plslda.m | |
| Target Projection | TP | inside pls.m or plslda.m | ||
| Uninformative Variable Elimination | UVE | mcuvepls.m, mcuveplslda.m | ||
| Competitive Adaptive Reweighted Sampling | CARS | carspls.m, carsplalda.m | ||
| Random Frog | randomfrog_pls.m, randomfrog_plslda.m | |||
| interval Random Frog | iRF | irf.m | ||
| Subwindow Permutation Analysis | SPA | spa.m | ||
| Moving Window Partial Least Squares | MWPLS | mwpls.m | ||
| the Phase Diagram algorithm | PHADIA | phadia.m | ||
| Iteratively Retain Informative Variables | IRIV | iriv.m | ||
| Variable Complementary Network | VCN | vcn.m |
2. Model Population Analysis (MPA)
To build a credible model for a given chemical or biological or clinical data, it may be helpful to first get somewhat better insight into the data itself before modeling and then to present the statistically stable results derived from a large number of sub-models established only on one dataset with the aid of Monte Carlo Sampling (MCS). We proposed a new concept Model Population Analysis (MPA), which is a general framework for designing new data analysis methods by statistically analyzing user-interested outputs (regression coefficients, prediction errors etc) of a number of sub-models generated by introducing data variation in samples or variables or both. New methods are expected to be developed by making full use of the interesting parameter in a novel manner. As described in the left figure, the output of a population of sub-models can be put into four spaces: sample space, variable space, parameter space and model space, which could serve as a guide for algorithm development.
The concept of MPA was originally proposed in J. Chemometr., 24 (2009) 418, and systmatically elucidated and reviewed in TrAC 38 (2012)154-162.
A series of MPA-based methods are available in the libPLS package, which include:
- Subwindow Permutation Analysis: variable selection for classification models; output a variable-interaction-incorporated P-value for assessing the synergistically statistical importance of each variable; this P-value is minus log10-transformed to COSS score; another statistic, called DMEAN, is also provided to assess the effect size of each vairable in predictions.
- Margin Influence Analysis: variable selection specififically designed for support vector machines, output a variable-interaction-incorporated P-value for assessing the synergistically statistical importance of each variable, this P-value is minus log10-transformed to COSS score; for assessing the synergisticaly statistical importance of each variable., this P-value is minus log10-transformed to COSS score ; another statistic, called DMEAN, is also provided to assess the effect size of each vairable in predictions.
- Random Frog: variable selection for both classification and regression, output a selection probability for each variable. In addition, interval Random Frog method was developed for spectral data only
- Monte Carlo Uninformative Variable Elimination: variable elimination for both classification and regression; output a reliability index for each vairable.
- Variable Complementary Network: building Variable Complementary Networks and simultaneously assessing variable importance; output a Total Complementary Information for each variable for importance evaluation.
- The Phase Diagram method: variable selection and visulization for classification; output a diagnostic plot showing which variables are important; output a variable-interaction-incorporated P-value for assessing the synergistically statistical importance of each variable. This method is an extension of MIA and SPA.
- Iteratively Retains Informative Variables: variable selection in regression.
- The Monte Carlo method: outlier detection in regression;output a diagnostic plot showing which samples are likely to be outliers.
A systematic introduction of the MPA idea can be found in our presentation [PDF] .