Every PyTorch model trains with the same five-step loop: forward, loss, zero gradients, backward, step. Write it once and you can train anything.
Why: unlike scikit-learn’s single fit(), PyTorch makes you write the loop — but it is always the same five steps, so once you know them you can train any model. When: memorize this order; the zero_grad is the step everyone forgets. Where: an optimizer holds the parameters and applies the update so you do not touch weights by hand.
for each batch:
1. pred = model(x) forward pass
2. loss = criterion(pred, y) how wrong?
3. optimizer.zero_grad() clear old gradients <- easy to forget!
4. loss.backward() compute new gradients
5. optimizer.step() update the weights
Forget step 3 and gradients accumulate across batches -> broken training.Why: putting the five steps in code, with an optimizer, is the entire training procedure — everything else is data loading and evaluation around it. When: this exact shape trains MLPs, CNNs, and Transformers alike. Where: Adam is a safe default optimizer; the learning rate is the most important knob.
import torch, torch.nn as nn
model = MLP(in_features=10, hidden=32, out=3)
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
for epoch in range(10):
for x, y in train_loader: # batches (next lesson)
pred = model(x) # 1. forward
loss = criterion(pred, y) # 2. loss
optimizer.zero_grad() # 3. clear grads
loss.backward() # 4. backward
optimizer.step() # 5. update
print(f"epoch {epoch} loss {loss.item():.4f}")Why: some layers (dropout, batch norm) behave differently during training and inference, so you must switch the model between train() and eval() modes — and disable gradient tracking when evaluating to save memory and time. When: model.train() before the loop, model.eval() + torch.no_grad() when validating or predicting. Where: forgetting eval() silently corrupts validation metrics.
# During training:
model.train()
# ... the training loop ...
# During validation / inference:
model.eval()
with torch.no_grad(): # no gradient tracking -> faster, leaner
for x, y in val_loader:
pred = model(x)
# accumulate validation metrics