Chain rule
To differentiate a composition, multiply the outer rate of change by the inner one. Backpropagation is nothing more than this rule applied layer by layer from the loss backwards, reusing each partial result.
- The outer function.
- The inner function.
- The inner function's own derivative.