The techniques you coded by hand in PyTorch — early stopping, checkpoints, LR schedules — are built-in Keras callbacks. And when fit() isn’t enough, GradientTape gives you the manual loop back.
Why: in PyTorch you wrote early stopping, best-checkpoint saving, and LR scheduling yourself inside the loop — Keras provides these as callbacks you pass to fit(), so common training logic is declarative and reusable. When: attach the callbacks you need; they hook into the loop at the right moments. Where: this is Keras trading the manual control of PyTorch for batteries-included convenience.
from tensorflow import keras
callbacks = [
keras.callbacks.EarlyStopping(patience=5, restore_best_weights=True),
keras.callbacks.ModelCheckpoint("best.keras", save_best_only=True), # best ckpt
keras.callbacks.ReduceLROnPlateau(patience=3), # LR schedule
keras.callbacks.TensorBoard(log_dir="logs"), # logging
]
model.fit(train_ds, validation_data=val_ds, epochs=100, callbacks=callbacks)
# Everything you hand-coded around the PyTorch loop is a callback here.Why: the overfitting and stabilization tools are the same as PyTorch — dropout, batch normalization, and weight decay — just added as Keras layers or optimizer arguments. When: insert Dropout/BatchNormalization layers and use an optimizer with weight_decay, exactly as you reasoned in PyTorch. Where: Keras applies dropout/batch-norm in the correct mode during fit vs. evaluate automatically, so there is no train()/eval() toggle to manage.
from tensorflow import keras
from tensorflow.keras import layers
model = keras.Sequential([
layers.Dense(64, input_shape=(10,)),
layers.BatchNormalization(), # nn.BatchNorm1d
layers.Activation("relu"),
layers.Dropout(0.3), # nn.Dropout — auto on/off per phase
layers.Dense(3),
])
model.compile(optimizer=keras.optimizers.AdamW(1e-3, weight_decay=1e-4), # L2
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True))Why: when fit() is too rigid — custom losses, multiple optimizers, GANs, RL — Keras gives you the PyTorch-style manual loop via tf.GradientTape, which records operations for autodiff just like PyTorch’s autograd. When: write a custom loop for anything fit() cannot express. Where: the structure mirrors PyTorch almost line for line — tape records, compute gradients, apply them — so your PyTorch loop intuition transfers directly.
import tensorflow as tf
optimizer = tf.keras.optimizers.Adam(1e-3)
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
for x, y in train_ds: # the loop is back, like PyTorch
with tf.GradientTape() as tape: # records ops for autodiff (autograd)
logits = model(x, training=True) # forward
loss = loss_fn(y, logits) # loss
grads = tape.gradient(loss, model.trainable_variables) # backward
optimizer.apply_gradients(zip(grads, model.trainable_variables)) # step