Derivatives
The derivative as a rate of change you can see, and the chain rule as the reason deep networks are trainable at all.
This lesson is part of a preview roadmap.
The engine around it is real — progress, notes and bookmarks all work — but the video has not been through the sourcing and review protocol a shipped lesson goes through. See the roadmap for what that means.
Derivatives
The player loads from youtube-nocookie.com when you press play. Nothing reaches YouTube before that.
What you'll learn
- A derivative is the best linear approximation to a function at a point — a slope, not an instant.
- Rules like the product and chain rules are statements about how small changes combine, and they can be read off a picture.
- In several variables the same idea becomes a gradient — one slope per input, collected into a vector.
The one calculus idea you cannot skip
The maths a network needs from calculus is narrow: a derivative is a slope, a gradient is a vector of slopes, and the chain rule lets you differentiate a function built out of other functions. That last one is the whole of backpropagation, two nodes from here.
What to watch for
The opening derivation of the area of a circle is a demonstration of method rather than a result worth memorising. It shows the move the rest of the subject runs on: chop something into pieces small enough that each piece is approximately simple, and watch what happens as the pieces shrink.
Where this sits
The path forks here. One branch goes through the network’s architecture, the other through its training signal, and they meet at gradient descent — which needs the shape of the network from one side and the derivative from the other.
Check yourself
What does "instantaneous rate of change" actually mean if nothing moves in an instant?
It is the limit of average rates over shrinking intervals. The value is a property of the limit, not of any single moment.
Why does the chain rule matter more than any other rule here?
Because a network is a composition of functions. Training it means differentiating a composition, and the chain rule is the only tool that does that.
Go deeper
The maths underneath
Notes
Notes are saved to your account.
Everything else on this page works without one — progress and bookmarks keep themselves in this browser and come with you when you sign in. Notes are the one thing worth not trusting to a browser you might clear.
Your notes could not be loaded.