Backpropagation

Understand the term. Connect the concept.

A practical dictionary of 512 AI, machine learning, mathematics, and programming terms from HerbDev’s AI Vocabulary app. Start with a definition, explore an example, then follow the concepts behind it.

Linked concepts open in a new tab. Audio reads the definition and example.

Model Training · Level 2 of 5

Computing how a network's loss changes with each parameter by propagating derivatives backward.

It applies the chain rule efficiently through the computation graph.

Example

After a prediction error, derivatives are calculated for every layer's weights.

Listen to the definition and example

Audio transcript

Backpropagation. Computing how a network's loss changes with each parameter by propagating derivatives backward. It applies the chain rule efficiently through the computation graph. For example: After a prediction error, derivatives are calculated for every layer's weights.

Explore this concept

A useful analogy

Like tracing a shared bill backward to determine how each contribution affected the total.

Why it matters

This helps you read a training loop and diagnose what is changing during learning.

Technical detail

Reverse-mode differentiation accumulates derivatives through a directed computation graph. For z = f(y), y = g(x), dz/dx = (dz/dy)(dy/dx).

Common misconception

Backpropagation computes derivatives; the optimizer decides how parameters change.

Start with

Related concepts

Quick recall question

Try answering before looking back at the definition.