Functions & Programming Concepts · Level 2 of 5
ReLU
An activation function returning the larger of zero and its input.
Its derivative is zero for negative inputs and one for positive inputs, with a convention at zero.
Example
A unit maps negative three to zero and positive three to three.
Listen to the definition and example
Audio transcript
ReLU. An activation function returning the larger of zero and its input. Its derivative is zero for negative inputs and one for positive inputs, with a convention at zero. For example: A unit maps negative three to zero and positive three to three.
Explore this concept
A useful analogy
A one-way gate that blocks negative signals and passes positive ones.
Why it matters
This helps you distinguish a computation’s contract, inputs, outputs, and behavior.
Technical detail
ReLU(x) = max(0,x). At x = 0 the derivative is not unique; implementations choose a convention.
Common misconception
A unit receiving only negative preactivations can stop receiving useful gradients.
Start with
Related concepts
Quick recall question
Try answering before looking back at the definition.