How Neural Networks Learn
Guess, measure the error, nudge every weight — a few million times.
A neural network starts out knowing nothing — its weights are random. Learning is a loop: make a prediction, measure how wrong it was, and shift every weight slightly in the direction that reduces the error.
1Input
→
2Weighted sum
→
3Activation
→
4Prediction
→
5Loss
→
6Backprop
Legend — how to read this diagram
1–nStagesthe ordered steps of the process
1 2 3Walkthroughnumbered steps below run in order
One training step
- Forward pass. Inputs flow through layers of weighted sums and nonlinear activations to a prediction.
- Loss. A loss function scores how far the prediction is from the truth.
- Backprop. The chain rule computes how much each weight contributed to the error.
- Update. Gradient descent nudges each weight opposite its gradient by a small learning rate.
- Repeat. Over many batches, the loss drops and the network generalizes.
Why it generalizes (usually)
Nonlinearity. Activations let stacked layers approximate complex functions.
Regularization. Dropout and weight decay stop it from memorizing the training set.
Data & scale. More diverse data and parameters generally mean better generalization — up to a point.
One-line mental model:
Learning is just error, blamed correctly across millions of knobs, then each knob turned a little.
No comments:
Post a Comment