Abstract The proliferation of Deep Neural Networks in various domains has seen an increased need for interpretability these models. Preliminary work done along this line, and papers that surveyed such, are focused on high-level representation analysis. However, a recent branch concentrated at more granular level analyzing neurons within In paper, we survey the neuron analysis including: i) meth...
