RNNs · Level 3 of 5
Teacher-Student Model
A setup where a student learns from a teacher model's outputs or representations.
It can transfer useful behavior but also replicate teacher errors.
Example
A small model is trained to imitate a larger model's predictions.
Listen to the definition and example
Audio transcript
Teacher-Student Model. A setup where a student learns from a teacher model's outputs or representations. It can transfer useful behavior but also replicate teacher errors. For example: A small model is trained to imitate a larger model's predictions.
Explore this concept
Why it matters
This helps you trace information and gradient flow through sequence models.
Start with
Related concepts
Quick recall question
Try answering before looking back at the definition.