Predict a category — spam or not, which digit, which disease. Meet the workhorse classifiers, from logistic regression to random forests and gradient boosting.
Why: no single classifier wins everywhere, so you should know the trade-offs — logistic regression is a fast interpretable baseline, tree ensembles are the strong default for tabular data, SVM and KNN suit specific shapes. When: start with logistic regression, then try a random forest or gradient boosting. Where: for tabular problems, gradient-boosted trees usually top the leaderboard.
MODEL STRENGTH NEEDS SCALING?
Logistic Regression fast, interpretable baseline yes
KNN simple, non-parametric yes
SVM strong on clean, small data yes
Decision Tree interpretable, non-linear no
Random Forest robust default, low tuning no
Gradient Boosting usually best on tabular data no
Start simple, then reach for tree ensembles.Why: thanks to the shared API, training any classifier is the same fit/predict call — so comparing them is trivial. When: use predict for the class and predict_proba for the probability behind it (needed for thresholds and ROC-AUC). Where: probabilities, not just labels, are what let you tune the decision threshold later.
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
X, y, random_state=42, stratify=y)
clf = RandomForestClassifier(n_estimators=200, random_state=42)
clf.fit(X_train, y_train)
print("labels:", clf.predict(X_test[:5]))
print("probabilities:", clf.predict_proba(X_test[:5])[:, 1]) # P(class 1)Why: gradient boosting builds trees sequentially, each correcting the last, and consistently wins on structured/tabular data — it is the model to beat. When: reach for HistGradientBoosting (or XGBoost/LightGBM) when you need top accuracy on tables. Where: it handles mixed scales and non-linearities without feature scaling.
from sklearn.ensemble import HistGradientBoostingClassifier
# Fast, strong, handles unscaled mixed features. The usual winner on tables.
gb = HistGradientBoostingClassifier(max_iter=300, random_state=42)
gb.fit(X_train, y_train)
print("accuracy:", gb.score(X_test, y_test))