The question of how to parallelize the stochastic gradient descent (SGD) method has received much attention in the literature. In this paper, we focus instead on batch methods that use a sizeable fraction of the training set at each iteration to facilitate parallelism, and that employ second-order information. In order to improve the learning process, we follow a multi-batch approach in which t...

A practical concern for Active Learning (AL) is the amount of time human experts must wait for the next instance to label. We propose a method for eliminating this wait time independent of specific learning and scoring algorithms by making scores always available for all instances, using old (stale) scores when necessary. The time during which the expert is annotating is used to train models an...

This contribution focuses on the development of a dynamic conceptual model for a batch rectifier having an infinite number of stages without resorting to multistage calculations. For a given instantaneous still composition, the key feature of the method is the estimation of the instantaneous minimum reflux or the instantaneous distillate composition by using the Rectification Body Method. The u...

This paper describes an implementation of the L-BFGS method designed to deal with two adversarial situations. The first occurs in distributed computing environments where some of the computational nodes devoted to the evaluation of the function and gradient are unable to return results on time. A similar challenge occurs in a multi-batch approach in which the data points used to compute functio...

This paper develops an efficient online algorithm for learning multiple consecutive tasks based on the KSVD algorithm for sparse dictionary optimization. We first derive a batch multi-task learning method that builds upon K-SVD, and then extend the batch algorithm to train models online in a lifelong learning setting. The resulting method has lower computational complexity than other current li...

Two parameter estimation methods are presented for online determination of parameter values using a simple charge/discharge model of a Sony 18650 lithium ion battery. Loss of capacity and resistance increase are both included in the model. The first method is a hybrid combination of batch data reconciliation and moving-horizon parameter estimation. A discussion on the selection of tuning parame...

A new isolation and purification method for bioproducts using membrane-encapsulated affinity adsorbents was investigated. The new method involves encapsulation of affinity adsorbents, batch adsorption of the bioproduct from whole fermentation broth and rapid batch desorption after dissolution of the capsule membranes. Recovery of protein A from Staphylococcus aureus was used as the model experi...

We consider regularized empirical risk minimization problems. In particular, we minimize the sum of a smooth empirical risk function and a nonsmooth regularization function. When the regularization function is block separable, we can solve the minimization problems in a randomized block coordinate descent (RBCD) manner. Existing RBCD methods usually decrease the objective value by exploiting th...

The most natural way to speed-up the training of large networks is to use dataparallelism on multiple GPUs. To scale Stochastic Gradient (SG) based methods to more processors, one need to increase the batch size to make full use of the computational power of each GPU. However, keeping the accuracy of network with increase of batch size is not trivial. Currently, the state-of-the art method is t...