C-SVDDNet: An Effective Single-Layer Network for Unsupervised Feature Learning

In this paper, we investigate the problem of learning feature representation from unlabeled data using a single-layer K-means network. A K-means network maps the input data into a feature representation by finding the nearest centroid for each input point, which has attracted researchers’ great attention recently due to its simplicity, effectiveness, and scalability. However, one drawback of this feature mapping is that it tends to be unreliable when the training data contains noise. To address this issue, we propose a SVDD based feature learning algorithm that describes the density and distribution of each cluster from Kmeans with an SVDD ball for more robust feature representation. For this purpose, we present a new SVDD algorithm called CSVDD that centers the SVDD ball towards the mode of local density of each cluster, and we show that the objective of CSVDD can be solved very efficiently as a linear programming problem. Additionally, previous single-layer networks favor a large number of centroids but a crude pooling size, resulting in a representation that highlights the global aspects of the object. Here we explore an alternative network architecture with much smaller number of nodes but with much finer pooling size, hence emphasizing the local details of the object. The architecture is also extended with multiple receptive field scales and multiple pooling sizes. Extensive experiments on several popular object recognition benchmarks, such as MINST, NORB, CIFAR-10 and STL-10, shows that the proposed C-SVDDNet method yields comparable or better performance than that of the previous state of the art methods.