Elastic Net

RegressionAdvanced~6 min

Elastic Net — Blend L1 and L2 penalties to get the best of both.

Elastic net mixes ridge and lasso in one penalty. You get lasso’s feature selection plus ridge’s stability with correlated features — controlled by a single mixing dial.

Coefficient path — drag to set λ

Coefficients at λ = 0.00100, ρ = 0.00 (click one to trace it)

⇄ marks the two strongly correlated features.

Elastic-net controls

Data

Design

x1 and x2 are strongly correlated. Pure lasso keeps one at random; ρ below 1 lets them share.

Model

Constraint (circle-like)

L1 ratio ρ0 · pure L2

Regularization λ0.00100

Largest |w|3.782

Non-zero6 / 6

Playback

What to step through

Step 0 / 20

Speed

Circle (ridge)
Rounded diamond
Diamond (lasso)
↻

Step 0 of 20 — ρ = 0.00 — penalty is 0% L1 and 100% L2, so the constraint region is circle-like; 6 of 6 coefficients survive at λ = 0.00100

Play the ρ mix and watch the constraint region above morph from a circle into a diamond — the corners are what make coefficients hit exactly zero.

Break it

Slide ρ to 1 and λ to the top: pure L1 at full strength either keeps one of the correlated pair at random or deletes every feature.

The idea in plain words

Elastic net simply adds both penalties at once: a slice of lasso’s L1 for sparsity and a slice of ridge’s L2 for stability. One mixing dial ρ slides between them, and you can watch the constraint region morph from a circle to a diamond as it moves.

The payoff shows up with correlated features. Pure lasso arbitrarily keeps one and drops the rest; the L2 part encourages correlated features to share the weight, so selection stays stable — the best of both penalties from a single knob.

Now, the math

Elastic net is a convex blend of the L1 and L2 penalties:

J = \tfrac{1}{n}\sum_i (y_i - \hat{y}_i)^2 + \lambda\Bigl(\rho \sum_j |w_j| + (1-\rho)\sum_j w_j^2\Bigr)

$\rho$: the L1 ratio — 1 is pure lasso, 0 is pure ridge, between is a blend.
$\lambda$: the overall strength of the combined penalty.
$1-\rho$: the share given to the stabilizing L2 term.

▸ Show the derivation

The constraint region ρ‖w‖₁ + (1−ρ)‖w‖² = c interpolates between the diamond (ρ = 1) and the circle (ρ = 0): its corners stay sharp enough to zero out irrelevant features while its sides round out enough to spread weight across correlated ones. Coordinate descent solves it with the same soft-threshold as lasso, divided by an extra 1 + λ(1−ρ) ridge shrinkage factor.

Trace it by hand

Two perfectly correlated features — identical columns x1 and x2, both equal to (-1, -1, 1, 1) — with targets y equal to 3 x1. The data cannot tell the twins apart, so only the penalty decides how weight is shared. All solutions computed with the repo's coordinate-descent and ridge solvers at lambda equal to 1.

Step 1 — the data alone cannot decide
$\mathbf{z} = \tfrac{1}{n}X^\top \mathbf{y} = (3,\ 3): \quad \text{any } w_1 + w_2 = 3 \text{ fits perfectly}$
Plain least squares has no unique answer here; the repo's coordinate descent at lambda 0 happens to land on the split (3, 0).
Step 2 — pure lasso picks one twin, rho equal to 1
$\rho = 1: \quad \mathbf{w} = (2,\ 0)$
Whichever coordinate updates first absorbs the signal; its twin sees nothing left above the threshold. The choice is arbitrary — reorder the features and it flips.
Step 3 — elastic net shares, rho equal to one half
$\rho = 0.5: \quad w_j = \frac{\operatorname{sign}(z_j)\max(|z_j| - \lambda\rho,\ 0)}{1 + \lambda(1-\rho)} \;\Rightarrow\; \mathbf{w} = (1,\ 1)$
Check the fixed point by hand: each weight sees z of 3 minus 1 equals 2, then 2 minus 0.5 over 1.5 equals exactly 1.
Step 4 — the L2 slice is what forces the tie
$\rho = 0.5: \quad \underbrace{0.5\,(2+0) + 0.5\,(4+0)}_{\mathbf{w}=(2,\,0)} = 3 \qquad \text{vs.} \qquad \underbrace{0.5\,(1+1) + 0.5\,(1+1)}_{\mathbf{w}=(1,\,1)} = 2$
Both candidates have the same L1 total of 2, but squaring punishes concentration: 4 plus 0 beats 1 plus 1 on the L2 side, so the even split is cheaper.
Step 5 — same predictions, different story
$\text{lasso: } \hat{y} = 2x_1 + 0\,x_2, \qquad \text{elastic net: } \hat{y} = 1\,x_1 + 1\,x_2$
On these four points the two models output identical predictions — the difference is which features get the credit, and only the elastic net's attribution survives reordering. For comparison, ridge at lambda 1 also lands on (1, 1).

What just happened: With identical features at lambda 1, lasso chose (2, 0) arbitrarily while elastic net at rho one half chose (1, 1): same L1 total, but the L2 slice makes the even split cheaper — penalty 2 versus 3 — so correlated features share the weight. That stability is exactly what elastic net was built for.

Now Break It

Try this: Wrong mixing ratio for the data either over-sparsifies or fails to select at all.

Control: Mixing ratio slider

What happens: Mis-tuned mix! The L1/L2 balance is wrong for this data — either too sparse or not selective enough.

Where elastic net is used

Elastic net blends the L1 and L2 penalties so it can select features like lasso while handling correlated predictors gracefully like ridge, which makes it a strong default in high-dimensional problems. Genomics again is a headline use case: when many genes are correlated within pathways, elastic net keeps whole groups of related genes together rather than arbitrarily dropping all but one, giving biologically sensible models. It is widely used in predictive modeling on wide datasets such as marketing response, credit risk with many overlapping indicators, and text or web features where vocabularies are large and redundant. Practitioners often adopt elastic net whenever they would consider lasso but worry about correlated inputs or datasets with more features than samples, gaining sparsity and stability at once.

A common misconception is that elastic net is simply lasso with a safety net and therefore always superior; in reality it has two knobs, the overall penalty strength and the mixing ratio between L1 and L2, and both must be tuned jointly by cross-validation, which costs more effort. Another pitfall is misunderstanding the mixing parameter: setting it fully toward L1 recovers plain lasso and fully toward L2 recovers ridge, so a poorly chosen mix can throw away the very grouping behavior that motivates elastic net. As with lasso and ridge, features must be standardized first. Finally, elastic net does not remove the need for domain judgment; its selected groups still require interpretation rather than blind trust.

Frequently asked questions

What is elastic net?

Elastic net is a regularized linear regression that combines the L1 penalty of lasso and the L2 penalty of ridge in a single model. The L1 part encourages sparsity by zeroing some coefficients, while the L2 part shrinks and stabilizes coefficients among correlated features. A mixing parameter controls the balance between the two penalties.

How does elastic net differ from lasso and ridge?

Ridge shrinks all coefficients but keeps every feature, lasso zeroes many features but struggles when they are correlated, and elastic net does both by mixing the two penalties. The L2 component lets elastic net keep groups of correlated features together instead of arbitrarily picking one, while the L1 component still yields a sparse model. It is essentially a tunable compromise between ridge and lasso.

What is the mixing parameter in elastic net?

The mixing parameter, often called the L1 ratio or alpha, sets how much of the penalty is L1 versus L2. A value of one makes elastic net behave exactly like lasso, and a value of zero makes it behave like ridge. Intermediate values blend sparsity with the grouping and stability benefits of the L2 term.

When should I choose elastic net?

Choose elastic net when you have many features, expect only some to matter, and suspect those features are correlated, a setting where pure lasso becomes unstable. It is a strong default for wide datasets with more features than samples. If features are largely independent, plain lasso or ridge may be simpler and sufficient.

How do I tune elastic net?

Tune two hyperparameters together: the overall penalty strength and the L1-to-L2 mixing ratio, usually with grid search and cross-validation. Search the penalty strength on a logarithmic scale and try several mixing values between pure ridge and pure lasso. Standardize the features beforehand so both penalties act fairly across them.

← Back to all visualizations Continue on the Learning Path →

Written & reviewed by the ML Visualization team · Last updated August 6, 2026.