Optimization · Level 2 of 5
Learning Rate
The scale applied to a gradient-based parameter update.
Too large a value can destabilize training; too small can slow it.
Example
A trainer multiplies its gradient by 0.001 before updating weights.
Listen to the definition and example
Audio transcript
Learning Rate. The scale applied to a gradient-based parameter update. Too large a value can destabilize training; too small can slow it. For example: A trainer multiplies its gradient by 0.001 before updating weights.
Explore this concept
A useful analogy
The length of each step, rather than the direction you walk.
Why it matters
This helps you understand what parameter updates are trying to improve and when they can fail.
Technical detail
In θ_next = θ − ηg, η is the learning rate and g is the update gradient. Adaptive optimizers also rescale coordinates.
Common misconception
A larger learning rate does not always make training finish sooner.
Start with
Related concepts
Quick recall question
Try answering before looking back at the definition.