Loss Function

Understand the term. Connect the concept.

A practical dictionary of 512 AI, machine learning, mathematics, and programming terms from HerbDev’s AI Vocabulary app. Start with a definition, explore an example, then follow the concepts behind it.

Linked concepts open in a new tab. Audio reads the definition and example.

Model Training · Level 2 of 5

A rule assigning a numerical penalty to predictions or model behavior.

Optimization uses this signal, which may differ from the final evaluation metric.

Example

A squared-error rule penalizes a prediction far from its target.

Listen to the definition and example

Audio transcript

Loss Function. A rule assigning a numerical penalty to predictions or model behavior. Optimization uses this signal, which may differ from the final evaluation metric. For example: A squared-error rule penalizes a prediction far from its target.

Explore this concept

A useful analogy

A scoring rule that tells training how costly a mistake is.

Why it matters

This helps you read a training loop and diagnose what is changing during learning.

Technical detail

A supervised objective often averages ℓ(f_θ(x_i),y_i) over examples, optionally adding regularization. Differentiability affects optimizer choice.

Common misconception

The loss being minimized need not be the metric ultimately reported.

Start with

Function

Related concepts

Quick recall question

Try answering before looking back at the definition.