Feature Selection via Regularization

How different penalties affect coefficient shrinkage

0.30

Ridge (L2)

All Features
Active Features
6/6

Lasso (L1)

Sparse
Active Features
4/6

Elastic Net

Balanced
Active Features
5/6
Ridge L2 Norm
1.84
Lasso L1 Norm
1.52
Elastic Net Combined
1.66
Key Insights
Ridge: Shrinks all coefficients but keeps all features
Lasso: Sets some coefficients to exactly zero
Elastic Net: Compromise between L1 and L2