training

ReLU

Rectified Linear Unit, the most widely used activation function in deep learning that outputs the input directly if positive, or zero if negative. ReLU is computationally efficient and helps mitigate the vanishing gradient problem in deep networks.