Abstract Dual encoders perform retrieval by encoding documents and queries into dense low-dimensional vectors, scoring each document its inner product with the query. We investigate capacity of this architecture relative to sparse bag-of-words models attentional neural networks. Using both theoretical empirical analysis, we establish connections between dimension, margin gold lower-ranked docum...

Sparse coding is an unsupervised learning algorithm that learns a succinct high-level representation of the inputs given only unlabeled data; it represents each input as a sparse linear combination of a set of basis functions. Originally applied to modeling the human visual cortex, sparse coding has also been shown to be useful for self-taught learning, in which the goal is to solve a supervise...

Acyclic phase-type distributions form a versatile model, serving as approximations to many probability distributions in various circumstances. They exhibit special properties and characteristics that usually make their applications attractive. Compared to acyclic continuous phase-type (ACPH) distributions, acyclic discrete phase-type (ADPH) distributions and their subclasses (ADPH family) have ...

Sparse coding is an unsupervised learning algorithm that learns a succinct high-level representation of the inputs given only unlabeled data; it represents each input as a sparse linear combination of a set of basis functions. Originally applied to modeling the human visual cortex, sparse coding has also been shown to be useful for self-taught learning, in which the goal is to solve a supervise...

Utilization of classification latent space information for downstream reconstruction and generation is an intriguing a relatively unexplored area. In general, discriminative representations are rich in class specific features but too sparse reconstruction, whereas, autoencoders the dense has limited indistinguishable features, making it less suitable classification. this work, we propose modell...

While convolutional sparse representations enjoy a number of useful properties, they have received limited attention for image reconstruction problems. The present paper compares the performance of block-based and convolutional sparse representations in the removal of Gaussian white noise. While the usual formulation of the convolutional sparse coding problem is slightly inferior to the block-b...