Ridge vs Lasso Regression
Ridge regression shrinks every coefficient toward zero but keeps them all; lasso drives some coefficients exactly to zero, deleting features from the model entirely. Both add a penalty on coefficient size to ordinary least squares, trading a little bias for a large drop in variance, but the penalty shape differs: ridge uses the L2 penalty, the sum of squared coefficients, while lasso uses the L1 penalty, the sum of absolute values. The L1 penalty's sharp corners are what make exact zeros possible, so lasso does automatic feature selection and ridge does not.
That geometric difference drives the practical guidance. When many features each contribute a little, and especially when features are correlated, ridge tends to predict better because it spreads weight smoothly across the correlated group. When only a few features truly matter and the rest are noise, lasso tends to win because it can silence the noise features completely and hand you a sparse, readable model. Lasso's known weakness is correlated predictors: it tends to pick one from a group somewhat arbitrarily and zero the others, and the survivor can change with a slightly different sample.
Both methods share two non-negotiable mechanics. Features must be standardized first, because the penalty treats all coefficients on the same scale, so an unscaled feature measured in large units gets unfairly crushed. And the penalty strength, usually called alpha or lambda, must be chosen by cross-validation, since the best value depends entirely on your data's signal-to-noise ratio and sample size, and no fixed default works across problems.
Side by side
Core idea
Add the sum of squared coefficients to the least-squares loss, shrinking all weights smoothly toward zero.
Add the sum of absolute coefficient values to the loss, shrinking weights and setting the weakest ones exactly to zero.
Penalty type
L2 penalty; differentiable everywhere, with a closed-form solution available.
L1 penalty; non-differentiable at zero, which is precisely what produces exact zeros, solved by coordinate descent or similar methods.
Effect on coefficients
Proportional-style shrinkage: every coefficient gets smaller, none reach exactly zero, and correlated features share the weight.
Soft thresholding: small coefficients are cut to exactly zero while surviving ones are shrunk by a constant amount.
Feature selection
None; the final model always uses every feature, however tiny its weight.
Built in; the zeroed coefficients remove features, yielding a sparse model you can read and deploy cheaply.
Correlated features
Handles them gracefully by splitting weight across the correlated group, keeping predictions stable.
Handles them poorly; it tends to keep one member of a correlated group arbitrarily and zero the rest, and the choice is unstable across resamples.
When it shines
Dense problems where many features each carry a little signal, and prediction accuracy matters more than a short feature list.
Sparse problems where a few features carry most of the signal and the rest are noise, or when you need an interpretable short list.
Key hyperparameters
One penalty strength alpha, chosen by cross-validation; larger alpha means more shrinkage.
One penalty strength alpha, chosen by cross-validation; larger alpha zeros out more features.
Preprocessing requirements
Standardize features first; unscaled features distort how the penalty distributes shrinkage.
Same requirement, and it matters even more because scale directly decides which coefficients cross the zero threshold.
More features than samples
Works well; the penalty makes the ill-posed least-squares problem solvable and stable.
Works and additionally prunes the feature set, but can select at most as many features as there are samples.
Typical use cases
Multicollinear regression, dense signals such as many weak predictors, and as the default stabilizer for linear models.
High-dimensional screening such as genomics or text, model compression, and any setting where the deliverable is a short feature list.
| Dimension | Ridge Regression (L2) | Lasso Regression (L1) |
|---|---|---|
| Core idea | Add the sum of squared coefficients to the least-squares loss, shrinking all weights smoothly toward zero. | Add the sum of absolute coefficient values to the loss, shrinking weights and setting the weakest ones exactly to zero. |
| Penalty type | L2 penalty; differentiable everywhere, with a closed-form solution available. | L1 penalty; non-differentiable at zero, which is precisely what produces exact zeros, solved by coordinate descent or similar methods. |
| Effect on coefficients | Proportional-style shrinkage: every coefficient gets smaller, none reach exactly zero, and correlated features share the weight. | Soft thresholding: small coefficients are cut to exactly zero while surviving ones are shrunk by a constant amount. |
| Feature selection | None; the final model always uses every feature, however tiny its weight. | Built in; the zeroed coefficients remove features, yielding a sparse model you can read and deploy cheaply. |
| Correlated features | Handles them gracefully by splitting weight across the correlated group, keeping predictions stable. | Handles them poorly; it tends to keep one member of a correlated group arbitrarily and zero the rest, and the choice is unstable across resamples. |
| When it shines | Dense problems where many features each carry a little signal, and prediction accuracy matters more than a short feature list. | Sparse problems where a few features carry most of the signal and the rest are noise, or when you need an interpretable short list. |
| Key hyperparameters | One penalty strength alpha, chosen by cross-validation; larger alpha means more shrinkage. | One penalty strength alpha, chosen by cross-validation; larger alpha zeros out more features. |
| Preprocessing requirements | Standardize features first; unscaled features distort how the penalty distributes shrinkage. | Same requirement, and it matters even more because scale directly decides which coefficients cross the zero threshold. |
| More features than samples | Works well; the penalty makes the ill-posed least-squares problem solvable and stable. | Works and additionally prunes the feature set, but can select at most as many features as there are samples. |
| Typical use cases | Multicollinear regression, dense signals such as many weak predictors, and as the default stabilizer for linear models. | High-dimensional screening such as genomics or text, model compression, and any setting where the deliverable is a short feature list. |
When to use Ridge Regression (L2)
- Your features are strongly correlated and you want stable coefficients and predictions rather than an arbitrary winner from each group.
- You believe many features each contribute a small amount, so deleting any of them would discard real signal.
- Prediction accuracy is the goal and nobody is asking you for a shortlist of variables.
- You want the most forgiving default: ridge rarely does anything surprising, making it the safe first regularizer.
- You need fast, closed-form or highly stable fits inside a larger pipeline, for example repeated refits in cross-validation.
When to use Lasso Regression (L1)
- You suspect only a handful of your many features actually matter and want the model to find and name them.
- Interpretability is a requirement: stakeholders need a short list of variables, not a hundred small weights.
- You have far more features than samples and need aggressive pruning before anything downstream.
- Deployment cost matters and dropping features saves real measurement or computation, such as removing sensors or expensive data fields.
- You are doing exploratory screening to decide which features deserve deeper study, treating the selection as a hypothesis, not a verdict.
The bottom line
If you only care about predictions, start with ridge: it is stable, handles correlated features gracefully, and almost never behaves surprisingly. Choose lasso when sparsity is itself a goal, because you need a short interpretable feature list, or you have vastly more features than samples and must prune. If you want lasso's selection but your features come in correlated groups, skip the dilemma and use elastic net, which blends both penalties and keeps or drops correlated features together. Whichever you choose, standardize your features first and pick the penalty strength by cross-validation; those two steps matter more than the L1-versus-L2 decision in most real problems.