This article discusses the usage of ReLU in a Convolutional Neural Network. There is also an experiment that involves random dropping of Activation Layers based on a hyperparameter in an Object Recognition model and validating it across the accuracy.