Activation function It takes the output of a neuron and decide whether this neuron is going to fire or not, in other words, "should this neuron 'fire' or not?" Activation function Step function Hardly used. Sigmoid function Used in Binary Classification. Used in output layer usually. Vanishing Gradient possible. ↓ Hyperbolic tangent function ReLU(Rectified Linear Unit) function f(x) = max(0,x) I..
