Training of Structural SVMs involves solving a large Quadratic Program (QP). One popular method for solving this QP is a cutting-plane approach, where the most violated constraint is iteratively added to a working-set of constraints. Unfortunately, training models with a large number of parameters remains a time consuming process. This paper shows that significant computational savings can be a...

We introduce a robust, error-tolerant adaptive training algorithm for generalized learning paradigms in high-dimensional superposed quantum networks, or adaptive quantum networks. The formalized procedure applies standard backpropagation training across a coherent ensemble of discrete topological configurations of individual neural networks, each of which is formally merged into appropriate lin...

The conventional two-stage training algorithm of the fuzzy/neural architecture called FALCON may not provide accurate results for certain type of problems, due to the implicit assumption of independence that this training makes about parameters of the underlying fuzzy inference system. In this correspondence, a training scheme is proposed for this fuzzy/neural architecture, which is based on li...

Overfitting is one of the most critical challenges in deep neural networks, and there are various types of regularization methods to improve generalization performance. Injecting noises to hidden units during training, e.g., dropout, is known as a successful regularizer, but it is still not clear enough why such training techniques work well in practice and how we can maximize their benefit in ...

A commonly used learning rule is to approximately minimize the average loss over the training set. Other learning algorithms, such as AdaBoost and hard-SVM, aim at minimizing the maximal loss over the training set. The average loss is more popular, particularly in deep learning, due to three main reasons. First, it can be conveniently minimized using online algorithms, that process few examples...

We investigate different representational granularities for sub-lexical representation in statistical machine translation work from English to Turkish. We find that (i) representing both Turkish and English at the morpheme-level but with some selective morpheme-grouping on the Turkish side of the training data, (ii) augmenting the training data with “sentences” comprising only the content words...

Fast convergence speed is a desired property for training latent Dirichlet allocation (LDA), especially in online and parallel topic modeling for massive data sets. This paper presents a novel residual belief propagation (RBP) algorithm to accelerate the convergence speed for training LDA. The proposed RBP uses an informed scheduling scheme for asynchronous message passing, which passes fast-co...

Training of Structural SVMs involves solving a large Quadratic Program (QP). One popular method for solving this optimization problem is a cutting-plane approach, where the most violated constraint is iteratively added to a working-set of constraints. Unfortunately, training models with a large number of parameters remains a time consuming process. This paper shows that significant computationa...

A time signal prediction algorithm based on Relative Neighborhood Graph (RNG) localized FIR filters is defined. The RNG connects two nodes, of input space dimension D, if their lune does not contain any other node:. The FIR filters associated with the nodes, are used for local approximation of the training vectors belonging to the lunes formed by the nodes. The predictor training is carried out...

We consider deep neural networks (formally equivalent to sum-product networks [19]), in which the output of each node is a quadratic function of its inputs. Similar to other deep architectures, these networks can compactly represent any function on a finite training set. The main goal of this paper is the derivation of a provably efficient, layer-by-layer, algorithm for training such networks, ...