Machine Learning Resources

What is Elastic-net? Why is it better in comparison to Ridge and Lasso?

Elastic net uses a weighted combination of the L1 and L2 penalties that are used in both LASSO and Ridge regression, respectively. By accounting for information from both penalties, it is able to harness the strengths of both regularization techniques. The cost function for Elastic net is as follows, where ƛ1 is the regularization parameter corresponding to the LASSO penalty, ƛ2 is the regularization parameter for the Ridge penalty, and ⍺ controls the weight assigned to each penalty. 

One advantage of using Elastic net is in a scenario when multicollinearity is present among the features. In that case, LASSO would tend to pick one out of a pair of correlated variables and shrink the other to 0, while Ridge would tend to shrink both coefficients at the same time. Therefore, Elastic net is better suited to incorporate the appropriate effects of each of the predictors by retaining both of them but still applying an extra degree of regularization by making appropriate use of the L1 component. 

Partner Ad