Optimization · Level 2 of 5
Gradient Descent
An iterative method that moves parameters opposite the gradient to reduce an objective locally.
The gradient gives local slope information; step size controls how far to move.
Example
Weights are adjusted a small amount opposite their loss derivatives.
Listen to the definition and example
Audio transcript
Gradient Descent. An iterative method that moves parameters opposite the gradient to reduce an objective locally. The gradient gives local slope information; step size controls how far to move. For example: Weights are adjusted a small amount opposite their loss derivatives.
Explore this concept
A useful analogy
Like taking short downhill steps in fog, using the local slope rather than a map of the entire landscape.
Why it matters
This helps you understand what parameter updates are trying to improve and when they can fail.
Technical detail
θ_next = θ − η ∇L(θ). A small step is locally downhill when the gradient exists; finite steps can overshoot.
Common misconception
Moving opposite the gradient does not guarantee reaching a global minimum.
Start with
Related concepts
Quick recall question
Try answering before looking back at the definition.