Models do math, not words. Turn categories into numbers the right way — one-hot for nominal, ordinal for ordered — without inventing a fake ranking.
Why: encoding a nominal category (city, color) as 1,2,3 invents an order the model will believe — one-hot avoids that, while genuinely ordered categories (small<medium<large) should keep their order. When: one-hot for unordered, ordinal for ordered, and beware high-cardinality columns that explode into thousands of one-hot columns. Where: the choice directly changes what the model can learn.
CATEGORY TYPE ENCODING WHY
nominal (city) one-hot no fake ordering
ordinal (S<M<L) ordinal (0,1,2) preserve the real order
high-cardinality target / hashing one-hot would blow up columns
binary (yes/no) single 0/1 column trivial
Never label-encode a nominal feature as 1,2,3 — the model reads 3 > 1.Why: one-hot turns each category into its own 0/1 column so no ordering is implied — the safe default for nominal features. When: use it for low-cardinality unordered categories. Where: handle_unknown="ignore" lets the test set contain categories the train set never saw without crashing.
from sklearn.preprocessing import OneHotEncoder
ohe = OneHotEncoder(handle_unknown="ignore", sparse_output=False)
ohe.fit(X_train[["city"]]) # learn categories from train
train_enc = ohe.transform(X_train[["city"]]) # e.g. city_NY, city_LA, ...
test_enc = ohe.transform(X_test[["city"]]) # unseen city -> all zeros
print(ohe.get_feature_names_out())Why: ordered categories should map to ordered numbers, and columns with thousands of values (user IDs, zip codes) need target or hashing encoding because one-hot would create thousands of columns. When: OrdinalEncoder with an explicit order for ordered data; target encoding for high cardinality. Where: target encoding uses the label, so it must be fit on train only or it leaks.
from sklearn.preprocessing import OrdinalEncoder
order = [["small", "medium", "large"]] # the REAL order
oe = OrdinalEncoder(categories=order)
X_train["size"] = oe.fit_transform(X_train[["size"]])
X_test["size"] = oe.transform(X_test[["size"]])
# High-cardinality (many unique values): use target encoding
# (category-encoders) — and fit on TRAIN ONLY, since it uses the target.