Solving Multi-collinearity Problem by Ridge and Eigen value Regression with Simulation

  • Taha Hussein Ali Department of Statistics-College of Administration and Economics / Salahaddin University-Erbil
Keywords: Multiple Regression, Ridge parameter, Multicollinearity, Conditional number, Eigen value Regression


                In this paper, five new methods were proposed to estimate the ridge parameter by inserting the conditional number, which are used to estimate the parameters of the ridge regression model to deal with multicollinearity problem and then compare their efficiency with some classical methods that was studied by several researchers based on Mean square error and comparing them with Eigen value Regression through simulation study (MATLAB language program designed for this purpose), the research shows that the efficiency of the proposed methods in dealing with multicollinearity problem and the advantages of the proposed methods compared with the classical methods of  Ridge and Eigen value Regression.


[1] Batah, F.S., Ramnathan, T., Gore, S.D., 2008. The efficiency of modified jackknife and ridge type regression estimators: a comparison. Surv. Math. App. 24 (2), 157–174.
[2] Draper, N. R. and Smith, H. (1998). Applied Regression Analysis (3rd edition). John Wiley & Sons.
[3] Dorugade, A.V., 2014. New ridge parameters for ridge regression. J. Assoc. Arab Univ. Basic Appl. Sci. 15, 94–99.
[4] Hoerl, A.E., Kennard, R.W., 1970. Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12, 55–67.
[5] Hoerl, A.E., Kennard, R.W., Baldwin, K.F., 1975. Ridge regression: some simulation. Commun. Stat. 4, 105–123.
[6] Khalaf, G., 2012. A proposed ridge parameter to improve the least squares estimator. J. Mod. Appl. Stat. Methods 11, 443–449.
[7] Khalaf, G., Shukur, G., 2005. Choosing ridge parameter for regression problem. Commun. Stat. Theor. Meth. 34, 1177–1182.
[8] Lawrence K. D. and Arthur J. L., 1989. Robust Regression: Analysis and Applications, 1st Edn., Marcel Dekker– New Yourk.
[9] Lawless, J.F., 1978. Ridge and related estimation procedure. Commun. Stat. 7, 139–164.
[10] McDonald, G.C., Galarneau, D.I., 1975. A monte carlo evaluation of some ridge-type estimators. J. Am. Stat. Assoc. 70, 407–416.
[11] Muniz, G., Kibria, B.M.G., 2009. On some ridge regression estimators: an empirical comparison. Commun. Stat. Simul. Comput. 38, 621–630.
[12] Saleh, A.K.Md.E., Kibria, B.M.G., 1993. Performances of some new preliminary test ridge regression estimators and their properties. Commun. Stat. 22, 2747–2764.
[13] Sarkar, N., 1992. A new estimator combining the ridge regression and the restricted least squares method of estimation. Commun. Stat. 21, 1987–2000.
[14] Singh, B., Chaubey, Y.P., 1987. On some improved ridge estimators. Stat. Pap. 28, 53–67.
[15] Swindel, B.F., 1976. Good ridge estimators based on prior information. Commun. Stat. Theor. Methods 11, 1065–1075.
[16] Webster J. T., Gunst R. F. and Mason R. L., 1974, Latent Root Regression Analysis, Technometrics, Vol.16, No.4, pp 513-22.
[17] Yan, X., 2008. Modified nonlinear generalized ridge regression and its application to develop naphtha cut point soft sensor. Comput. Chem. Eng. 32 (3), 608–621.
[18] Yan, X., Zhao, W., 2009. Concentration soft sensor based on modified bak propagation algorithm embedded with ridge regression. Intell. Autom. Soft Comput. 15 (1), 41–51.
[19] Yang, H., Chang, X., 2010. A new two-parameter estimator in linear regression. Comm. Stat. Theor. Methods 39, 923–934.
[20] Zhong, Z., Yang, H., 2007. Ridge estimation to the restricted linear model. Commun. Stat. Theor. Methods 36, 2099–2115.
How to Cite
Ali T. Solving Multi-collinearity Problem by Ridge and Eigen value Regression with Simulation. JAHS [Internet]. 1Nov.2018 [cited 12Jul.2020];22(5):(PP 262 -276). Available from: