Activation Function

Activation functions in neural networks make the models non-linear. Without non-linear functions applied to neurons, neural nets would be just linear models.