Recent research indicates that deep learning has achieved noticeably promising results in a wide range of areas such as computer vision, speech recognition and natural language processing. This paper offers an empirical study on the use of deep learning techniques for hard characters recognition on the notMNIST dataset. The MNIST dataset has been widely used for training and testing in the fiel...

We present a new approach and a novel architecture, termed WSNet, for learning compact and efficient deep neural networks. Existing approaches conventionally learn full model parameters independently at first and then compress them via ad hoc processing like model pruning or filter factorization. Different from them, WSNet proposes learning model parameters by sampling from a compact set of lea...

Deep feedforward neural networks are associated with complicated, nonconvex objective functions. Yet, simple optimization algorithms can identify parameters that generalize well to held-out data. We currently lack detailed descriptions of this learning process, even on a qualitative level. We propose a simple tensor decomposition model to study how hidden representations evolve over learning. T...

The problem of sound classification has been studied in depth and has multiple applications related to identity discrimination, enhanced hearing aids, robotics, and music technology. There are two ways in which the problem of sound classification can be approached. The first method is empirical in nature, and requires a database of sounds to learn from through feature extraction. The second met...

Over the last few years, deep learning has revolutionized the field of machine learning by dramatically improving the state-of-the-art in various domains. However, as the size of supervised artificial neural networks grows, typically so does the need for larger labeled datasets. Recently, crowdsourcing has established itself as an efficient and cost-effective solution for labeling large sets of...

Recent acoustic event classification research has focused on training suitable filters to represent acoustic events. However, due to limited availability of target event databases and linearity of conventional filters, there is still room for improving performance. By exploiting the non-linear modeling of deep neural networks (DNNs) and their ability to learn beyond pre-trained environments, th...

This paper reviews recent studies in emerging directions of understanding neural-network representations and learning neural networks with interpretable/disentangled middle-layer representations. Although deep neural networks have exhibited superior performance in various tasks, the interpretability is always an Achilles' heel of deep neural networks. At present, deep neural networks obtain a h...

x R el u (x ) ReLU function and its derivative ReLU(x) ReLU’(x) h (0) 0 Bias h (0) 1 Input #2 h (0) 2 Input #3 h (0) 3 Input #4 h (0) 4 Input #5 h (0) 5 Input #6 h (1) 0 h (1) 1 h (1) 2 h (1) 3 h (1) 4 h (1) 5 h (1) 6 h (h) 0 h (h) 1 h (h) 2 h (h) 3 h (h) 4 h (h) 5 h (N) 1 Output #1 h (N) 2 Output #2 h (N) 3 Output #3 h (N) 4 Output #4 h (N) 5 Output #5 Hidden layer 1 Input layer Hidden layer h...