Deep vs. shallow networks: An approximation theory perspective
نویسندگان
چکیده
منابع مشابه
Deep vs. shallow networks : An approximation theory perspective
The paper briefly reviews several recent results on hierarchical architectures for learning from examples, that may formally explain the conditions under which Deep Convolutional Neural Networks perform much better in function approximation problems than shallow, one-hidden layer architectures. The paper announces new results for a non-smooth activation function – the ReLU function – used in pr...
متن کاملShallow vs. Deep Sum-Product Networks
We investigate the representational power of sum-product networks (computation networks analogous to neural networks, but whose individual units compute either products or weighted sums), through a theoretical analysis that compares deep (multiple hidden layers) vs. shallow (one hidden layer) architectures. We prove there exist families of functions that can be represented much more efficiently...
متن کاملConvolutional Neural Networks for Text Categorization: Shallow Word-level vs. Deep Character-level
This paper reports the performances of shallow word-level convolutional neural networks (CNN), our earlier work (2015) [3, 4], on the eight datasets with relatively large training data that were used for testing the very deep characterlevel CNN in Conneau et al. (2016) [1]. Our findings are as follows. The shallow word-level CNNs achieve better error rates than the error rates reported in [1] t...
متن کاملFrom Deep to Shallow: Transformations of Deep Rectifier Networks
In this paper, we introduce transformations of deep rectifier networks, enabling the conversion of deep rectifier networks into shallow rectifier networks. We subsequently prove that any rectifier net of any depth can be represented by a maximum of a number of functions that can be realized by a shallow network with a single hidden layer. The transformations of both deep rectifier nets and deep...
متن کاملDiscovering Theoretically Grounded Predictors of Deep vs. Shallow Level Learning
We investigated predictors of shallow and deep learning for 192 college students with high vs. low prior knowledge in a game-like intelligent tutoring system, OperationARA that has an eText, multiple-choice tests, case-based reasoning, and adaptive tutorial conversations. Students are expected to learn about 11 topics of research methodology across three modules that target factual information,...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Analysis and Applications
سال: 2016
ISSN: 0219-5305,1793-6861
DOI: 10.1142/s0219530516400042