Many models are distance- or gradient-based and break when features live on wildly different scales. Standardize or normalize — and know which models even need it.
Why: models that use distances (KNN, SVM, K-Means) or gradients (linear/logistic regression, neural nets) let large-magnitude features dominate, so features must share a scale. When: scale for those model families; skip it for tree-based models (decision trees, random forests, gradient boosting), which are scale-invariant. Where: scaling the wrong model wastes effort but rarely hurts — not scaling a distance model quietly ruins it.
NEEDS SCALING DOESN'T NEED SCALING
--------------------------- ------------------------------
KNN, SVM, K-Means Decision Tree
Linear / Logistic Reg. Random Forest
Neural networks / PCA Gradient Boosting (XGBoost, ...)
Distance- and gradient-based models care about scale. Tree splits don't.Why: standardization centers each feature to mean 0 and std 1 (good default, robust to most distributions), while min-max squashes to a fixed [0,1] range (good when you need bounded inputs). When: StandardScaler by default; MinMaxScaler for bounded inputs like image pixels or some neural nets. Where: both are fit on train only, then applied to test.
from sklearn.preprocessing import StandardScaler, MinMaxScaler
std = StandardScaler()
X_train_s = std.fit_transform(X_train[num_cols]) # mean 0, std 1
X_test_s = std.transform(X_test[num_cols])
mm = MinMaxScaler() # squash to [0, 1]
X_train_m = mm.fit_transform(X_train[num_cols])
X_test_m = mm.transform(X_test[num_cols])Why: a few extreme outliers wreck StandardScaler (they inflate the std) and MinMaxScaler (they set the range) — RobustScaler uses the median and interquartile range, which outliers barely move. When: reach for it when a feature has heavy outliers you cannot remove. Where: pair it with the outlier handling from the cleaning step.
from sklearn.preprocessing import RobustScaler
# Uses median and IQR instead of mean and std -> outliers barely shift it.
robust = RobustScaler()
X_train_r = robust.fit_transform(X_train[num_cols])
X_test_r = robust.transform(X_test[num_cols])