Episode notes
Series: Weekly AI Papers · Historical classic
Rumelhart, Hinton, and Williams explain how backpropagation trains hidden units to learn useful internal representations.
A forward pass computes the network's outputs; a backward pass uses the chain rule to calculate how each weight contributes to the error.
The weights are adjusted to reduce squared output error, with momentum helping the learning process converge.
Experiments on mirror symmetry and family relationships show hidden units discovering structure that was not supplied as a training target.
The paper extends the method to recurrent networks by unfolding their computation across time.
This 1986 Nature paper also discusses local minima and distinguishes its procedure from a biologically plausible model of learning.
Source: Learning representations by back-propagating errors · David E. Rumelhart, Geoffrey E. Hinton and Ronald J. Williams
Explore: AI & Computer Science · AI Papers
Related episodes: Machines That Think Like People (Lake et al.) · V-JEPA (LeCun)