A neuron is a weighted sum plus an activation; a network stacks them in layers. Build one by hand to see inside, then the clean way with nn.Module.
Why: every neural network is built from one simple unit — multiply inputs by weights, add a bias, apply a non-linear activation — and understanding that unit demystifies the whole field. When: stacking many of these in layers is what gives networks their power. Where: the non-linearity is essential; without it, stacked layers collapse into a single linear model.
inputs x1 x2 x3
\ | /
(w1 w2 w3) weights + bias b
|
sum = w1*x1 + w2*x2 + w3*x3 + b
|
activation (e.g. ReLU) <- the non-linearity
|
output
A LAYER is many neurons; a NETWORK is stacked layers.Why: doing one layer with raw tensors shows there is no magic — a linear layer is just a matrix multiply plus a bias, then an activation. When: you will never write it this way in practice, but seeing it once makes nn.Module obvious. Where: the weight matrix shape is (in_features, out_features).
import torch
x = torch.randn(1, 3) # one sample, 3 features
W = torch.randn(3, 4) # 3 inputs -> 4 neurons
b = torch.randn(4)
z = x @ W + b # linear combination
a = torch.relu(z) # activation (non-linearity)
print(a.shape) # torch.Size([1, 4])Why: nn.Module manages parameters, moves to the GPU, and composes layers, so you define a network by listing layers and a forward pass — no manual weight bookkeeping. When: define every real model as an nn.Module subclass (or nn.Sequential for simple stacks). Where: nn.Linear creates and tracks its own weights and bias for you.
import torch.nn as nn
class MLP(nn.Module):
def __init__(self, in_features, hidden, out):
super().__init__()
self.net = nn.Sequential(
nn.Linear(in_features, hidden),
nn.ReLU(),
nn.Linear(hidden, out),
)
def forward(self, x):
return self.net(x)
model = MLP(in_features=10, hidden=32, out=3)
print(model)
print(sum(p.numel() for p in model.parameters()), "parameters")