Motivated by recent advances in morphological neural networks, we further study the properties of morphological units when incorporated in layers of conventional neural networks. We confirm and extend the observation that a Max-plus layer can be used to select relevant filters and reduce redundancy in its previous layer, without incurring performance loss. We present several experiments in image processing, showing that this filter selection property seems efficient and robust. We also point out the close connection between Maxout networks and our pruned Max-plus networks. The code related to our experiments is available online (https://github.com/yunxiangzhang).