Unsupervised Object Discovery and Co-Localization by Deep Descriptor Transforming

Reusable model design becomes desirable with the rapid expansion of computer vision and machine learning applications. In this paper, we focus on the reusability of pre-trained deep convolutional models. Specifically, different from treating pre-trained models as feature extractors, we reveal more treasures beneath convolutional layers, i.e., the convolutional activations could act as a detector for the common object in the image colocalization problem. We propose a simple yet effective method, termed Deep Descriptor Transforming (DDT), for evaluating the correlations of descriptors and then obtaining the category-consistent regions, which can accurately locate the common object in a set of unlabeled images, i.e., unsupervised object discovery. Empirical studies validate the effectiveness of the proposed DDT method. On benchmark image co-localization datasets, DDT consistently outperforms existing state-of-the-art methods by a large margin. Moreover, DDT also demonstrates good generalization ability for unseen categories and robustness for dealing with noisy data. Beyond those, DDT can be also employed for harvesting web images into valid external data sources for improving performance of both image recognition and object detection. The first two authors contributed equally to this work. This work was done when X.-S. Wei was visiting the University of Adelaide. Corresponding authors: Jianxin Wu and Chunhua Shen Xiu-Shen Wei · Chen-Lin Zhang · Jianxin Wu · Zhi-Hua Zhou Nanjing University, China E-mail: {weixs, zhangcl, wujx, zhouzh}@lamda.nju.edu.cn Chunhua Shen The University of Adelaide, Australia E-mail: [email protected]