Consistent Sparse Deep Learning: Theory and Computation
نویسندگان
چکیده
منابع مشابه
Unsupervised Feature Learning by Deep Sparse Coding
In this paper, we propose a new unsupervised feature learning framework, namely Deep Sparse Coding (DeepSC), that extends sparse coding to a multi-layer architecture for visual object recognition tasks. The main innovation of the framework is that it connects the sparse-encoders from different layers by a sparse-to-dense module. The sparse-to-dense module is a composition of a local spatial poo...
متن کاملSparse Feature Learning for Deep Belief Networks
Unsupervised learning algorithms aim to discover the structure hidden in the data, and to learn representations that are more suitable as input to a supervised machine than the raw input. Many unsupervised methods are based on reconstructing the input from the representation, while constraining the representation to have certain desirable properties (e.g. low dimension, sparsity, etc). Others a...
متن کاملPosterior Concentration for Sparse Deep Learning
Spike-and-Slab Deep Learning (SS-DL) is a fully Bayesian alternative to Dropout for improving generalizability of deep ReLU networks. This new type of regularization enables provable recovery of smooth input-output maps with unknown levels of smoothness. Indeed, we show that the posterior distribution concentrates at the near minimax rate for α-Hölder smooth maps, performing as well as if we kn...
متن کاملDeep Learning with Dynamic Computation Graphs
Neural networks that compute over graph structures are a natural fit for problems in a variety of domains, including natural language (parse trees) and cheminformatics (molecular graphs). However, since the computation graph has a different shape and size for every input, such networks do not directly support batched training or inference. They are also difficult to implement in popular deep le...
متن کاملLearning sparse and deep representations: new algorithms and perspectives
Three papers related to representation learning are discussed. The first paper presents a broad overview of the field of unsupervised representation learning, the second paper presents a stochastic model for images, and the final paper presents a new approach to encoding images. The relationship between these papers is discussed, and a modification to the method of the third paper is proposed.
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of the American Statistical Association
سال: 2021
ISSN: 0162-1459,1537-274X
DOI: 10.1080/01621459.2021.1895175