Exploiting Geometric Structure of High Dimensional Data for Learning: An Empirical Study

In machine learning, high dimensional data generally should have a high degree of freedom. However, recent experiments in machine learning show that real world data in high dimensions is usually governed by a surprisingly low dimensions. We believe that in high dimensions, geometry information, for example, the “shape” of data distribution, can help learning algorithms to perform better. A geometric transform of high dimensions targeted for learning is attractive for high dimensional machine learning problems. In this paper, we gave an empirical evaluation by experiments comparing how geometric information of high dimensional data can improve learning. We consider two geometric transforms, Laplacian Eigenmaps and Diffusion maps as our general geometric transforms. Distance in spaces after the transform is discussed. We compared classification results from data in original spaces to a new representation of the data after geometric transforms in various application areas, including image, text, acoustic signals, microarray data, and artificial data sets. Results showed that learning algorithms can take advantage of geometric information for most real world data in high dimensions. When labeled examples are extremely few, geometric transforms showed great improvement in learning. We also found cases when these geometric transforms fail in artificial data sets. General conditions when the transforms can result in better classifications are discussed.