Latent Autoregressive Source Separation

Autoregressive models have achieved impressive results over a wide range of domains in terms generation quality and downstream task performance. In the continuous domain, key factor behind this success is usage quantized latent spaces (e.g., obtained via VQ-VAE autoencoders), which allow for dimensionality reduction faster inference times. However, using existing pre-trained to perform new non-trivial tasks difficult since it requires additional fine-tuning or extensive training elicit prompting. This paper introduces LASS as way vector-quantized Latent Source Separation (i.e., de-mixing an input signal into its constituent sources) without requiring gradient-based optimization modifications models. Our separation method relies on Bayesian formulation autoregressive are priors, discrete (non-parametric) likelihood function constructed by performing frequency counts sums addend tokens. We test our images audio with several sampling strategies ancestral, beam search) showing competitive approaches while offering at same time significant speedups scalability higher dimensional data.