Journal of Data Science and Modeling

PCA by Shrinkage Estimation: A Comprehensive Mathematical and Statistical Analysis

Document Type : Research Manuscript

Authors

Parviz Nasiri 1
Heydar Mokhtari Farivar 2

1 Payam Noor uinversity

2 Department of Mathematics and Computer Sciences, Iran University of Science and Technology, P.O. Box 16846, Tehran, Iran

https://doi.org/10.22054/jdsm.2025.86705.1074
Abstract
Principal Component Analysis (PCA) is a cornerstone technique for dimensionality
reduction and data analysis. However, classic PCA can exhibit instability in
high-dimensional settings where the number of variables significantly exceeds the
number of observations. Shrinkage-based PCA addresses this limitation by incorporating
regularization into the covariance matrix estimation process, leading to
more stable and interpretable results. This paper provides a robust mathematical
and statistical foundation for shrinkage-based PCA, compares its performance with
classic PCA, and demonstrates its advantages through theoretical analysis, numerical
simulations, and real-world data experiments. It is important to note that using the idea of a contraction estimator increases the efficiency of the estimator. mean time in this paper, it is shown that the covariance matrix estimator resulting from the contraction estimator is very efficient.
It is also worth mentioning that to increase the efficiency of the contraction estimator, the recently discussed interval contraction estimator can be used.
keywords: principal component analysis, Shrinkage-based, Estimation, Covariance Structures, Simulation.

Keywords

Principal Component Analysis (PCA)
Estimation
Mean square error
Subjects
Journal of Data Science and Modeling
Volume 3, Issue 1 - Serial Number 5
December 2024
Pages 29-47

Home

Guide for Authors

Submit Manuscript

Contact Us