Manifold Learning for Semantic Visualization

Visualization of high-dimensional data, such as text documents, is useful to map out the similarities among various data points. In the high-dimensional space, documents are commonly represented as bags of words, with dimensionality equal to the size of the vocabulary. Classical approaches to document visualization directly reduce this into visualizable two or three dimensions, using techniques such as multidimensional scaling. Recent approaches consider an intermediate representation in topic space, between word space and visualization space, which preserves the semantics by topic modeling. While aiming for a good fit between the model parameters and the observed data, previous approaches have not considered the local consistency among data instances in terms of the intrinsic geometric structure of the document manifold. We consider the problem of semantic visualization by jointly modeling topics and visualization on the intrinsic document manifold. Each document has both a topic distribution and visualization coordinate. Specifically, we propose an unsupervised probabilistic model, called Semafore, which aims to preserve the manifold in the lower-dimensional spaces through a regularization framework designed for the semantic visualization task. To validate the efficacy of Semafore, our comprehensive experiments on a number of real-life text datasets of news articles and web pages show that the proposed methods outperform the the state-of-the-art baselines on objective evaluation metrics.