\begin{abstract}
This paper reviews the most common activation functions for convolution neural
networks. They are evaluated on the Asirra, GTSRB, HASYv2, STL-10, CIFAR-10,
CIFAR-100 and MNIST dataset. Possible reasons for the differences in their
performance are given.

New state of the art results are achieved for Asirra, GTSRB, HASYv2 and STL-10.
\end{abstract}