Visualizing population structure with variational autoencoders

Abstract Dimensionality reduction is a common tool for visualization and inference of population structure from genotypes, but popular methods either return too many dimensions easy plotting (PCA) or fail to preserve global geometry (t-SNE UMAP). Here we explore the utility variational autoencoders (VAEs)—generative machine learning models in which pair neural networks seek first compress then recreate input data—for visualizing genetic variation. VAEs incorporate nonlinear relationships, allow users define dimensionality latent space, our tests better than t-SNE UMAP. Our implementation, call popvae, available as command-line python program at github.com/kr-colab/popvae. The approach yields embeddings that capture subtle aspects humans Anopheles mosquitoes, can generate artificial genotypes characteristic given sample population.