Model Evaluation · Level 2 of 5
Benchmark
A standardized task, dataset, or procedure used for comparison.
Benchmark success may not transfer to the intended real-world use.
Example
Several models are evaluated on the same held-out task set.
Listen to the definition and example
Audio transcript
Benchmark. A standardized task, dataset, or procedure used for comparison. Benchmark success may not transfer to the intended real-world use. For example: Several models are evaluated on the same held-out task set.
Explore this concept
Why it matters
This helps you ask what a reported score counts and which errors it can hide.
Start with
Quick recall question
Try answering before looking back at the definition.