Before analysis, data needs tidying: missing values, wrong types, and text that needs normalizing. Handle the everyday messes so your numbers are trustworthy.
Why: missing values break math and models, so you first find them, then decide per column whether to drop or fill — never ignore them. When: inspect isna().sum() before anything else. Where: this is a first look; the Data Cleaning course covers imputation strategies in depth.
import pandas as pd, numpy as np
df = pd.DataFrame({"age": [36, np.nan, 45], "city": ["NY", "LA", None]})
print(df.isna().sum()) # count missing per column
df["age"] = df["age"].fillna(df["age"].median()) # fill numeric with median
df["city"] = df["city"].fillna("unknown") # fill text with a marker
# Or drop rows/columns that are mostly empty:
# df = df.dropna(subset=["age"])Why: numbers loaded as text will not do math and inconsistent casing splits the same category into many — fixing types and normalizing strings makes the data usable. When: right after loading, coerce types and clean key text columns. Where: astype converts; the .str accessor gives vectorized string methods.
df = pd.DataFrame({"price": ["10", "20", "30"], "city": [" NY ", "la", "Ny"]})
df["price"] = df["price"].astype(float) # text -> number
df["city"] = df["city"].str.strip().str.lower() # " NY " -> "ny"
print(df["city"].value_counts()) # ny: 2, la: 1 — now consistentWhy: sometimes a transformation is custom logic no built-in covers — map and apply run your function over a Series or DataFrame. When: prefer vectorized operations first; use apply when the logic genuinely needs Python per row. Where: apply is slower than vectorized math, so do not reach for it by default.
df = pd.DataFrame({"salary": [90000, 110000, 60000]})
def bucket(s):
if s < 70000: return "low"
if s < 100000: return "mid"
return "high"
df["band"] = df["salary"].apply(bucket)
print(df)
# Vectorized alternative for simple cases:
# df["band"] = pd.cut(df["salary"], bins=[0, 70000, 100000, np.inf],
# labels=["low", "mid", "high"])