Unregularized models overfit — they memorize noise. Ridge, Lasso, and ElasticNet penalize large coefficients to generalize better, and Lasso even selects features.
Why: a model free to use huge coefficients fits training noise and fails on new data — regularization adds a penalty on coefficient size, trading a little training fit for much better generalization. When: use it whenever you have many features or signs of overfitting. Where: the strength alpha controls the trade-off: more alpha, simpler model.
RIDGE (L2) shrinks all coefficients toward zero, keeps them all
-> good when many features each contribute a little
LASSO (L1) drives some coefficients to EXACTLY zero
-> doubles as automatic feature selection
ELASTICNET mix of L1 and L2 (l1_ratio blends them)
-> Lasso's selection + Ridge's stability
Bigger alpha = stronger penalty = simpler model.Why: Ridge shrinks every coefficient to reduce variance, while Lasso zeroes out the useless ones — so Lasso both regularizes and selects features. When: Ridge when all features may matter a little; Lasso when you suspect many are irrelevant. Where: both need scaled features, so use them inside a pipeline with a scaler.
from sklearn.linear_model import Ridge, Lasso
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
ridge = make_pipeline(StandardScaler(), Ridge(alpha=1.0))
ridge.fit(X_train, y_train)
print("Ridge R^2:", ridge.score(X_test, y_test))
lasso = make_pipeline(StandardScaler(), Lasso(alpha=0.1))
lasso.fit(X_train, y_train)
# Count features Lasso kept (non-zero coefficients):
import numpy as np
print("Lasso kept", np.sum(lasso[-1].coef_ != 0), "features")Why: ElasticNet blends L1 and L2 to get Lasso’s selection with Ridge’s stability, and the penalty strength alpha is a hyperparameter you tune, not guess. When: use ElasticNet with correlated features where pure Lasso is unstable. Where: the CV variants search alpha for you via cross-validation (covered later).
from sklearn.linear_model import ElasticNetCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
# ElasticNetCV tunes alpha (and the L1/L2 mix) by cross-validation for you.
model = make_pipeline(
StandardScaler(),
ElasticNetCV(l1_ratio=[0.1, 0.5, 0.9], cv=5),
)
model.fit(X_train, y_train)
print("chosen alpha:", model[-1].alpha_)
print("R^2:", model.score(X_test, y_test))