Topic extraction from extremely short texts with variational manifold regularization

With the emerging of massive short texts, e.g., social media posts and question titles from Q&A systems, discovering valuable information them is increasingly significant for many real-world applications content analysis. The family topic modeling can effectively explore hidden structures documents through assumptions latent topics. However, due to sparseness existing models, Dirichlet allocation, lose effectiveness on them. To this end, an effective solution, namely multinomial mixture (DMM), supposing that each text only associated with a single topic, indirectly enriches document-level word co-occurrences. DMM sensitive noisy words, where it often learns inaccurate representations at document level. address problem, we extend novel Laplacian Multinomial Mixture (LapDMM) model texts. basic idea LapDMM preserve local neighborhood enabling spread topical signals among neighboring documents, so as modify representations. This achieved by incorporating variational manifold regularization into objective DMM, constraining close texts similar find nearest neighbors before inference, construct offline graph, distances be computed mover’s distance. We further develop online version LapDMM, Online achieve inference speedup Carrying implications, exploit spirit stochastic optimization mini-batches up-to-date graph efficiently approximate instead. evaluate our compare against state-of-the-art models several traditional tasks, i.e., quality, clustering classification. empirical results demonstrate very performance gains over baseline models.