Efficient online learning with pairwise loss functions is a crucial component in building largescale learning system that maximizes the area under the Receiver Operator Characteristic (ROC) curve. In this paper we investigate the generalization performance of online learning algorithms with pairwise loss functions. We show that the existing proof techniques for generalization bounds of online a...

Computational grid is a new paradigm in parallel and distributed computing systems for realizing a virtual supercomputer over idle resources available in a wide area network like the Internet. Computational Grids are characterized for exploiting highly heterogeneous resources; so, one of the main concerns in developing computational grids is how to effectively map tasks onto heterogeneous resou...

So far we have mostly studied statistical learning theory, which focuses on batch learning under probabilistic assumptions. More specifically, we have mostly assumed that the learning algorithm has access to a training set S of n examples drawn iid from a distribution D, and we would like the algorithm to produce a hypothesis or predictor that has low error (measured in some way) on fresh sampl...

In contrast to traditional machine learning algorithms, where all data are available in batch mode, the new paradigm of streaming data poses additional difficulties, since data samples arrive in a sequence and many hard decisions have to be made on-line. The problem addressed here consists of classifying streaming data which not only are unlabeled, but also have a number l of attributes arrivin...

While online learning techniques have existed since Rosenblatt’s introduction of the Perceptron in 1957, there has been a renewed interest lately due to the need for efficient classification algorithms. Additionally, kernel techniques have allowed online learning to be extended to problems whose classes are not linearly separable in their native space. Online algorithms typically result in poor...

Online learning, in contrast to batch learning, occurs in a sequence of rounds. At the beginning of a round, an example is presented to the learning algorithm, the learning algorithm uses its current hypothesis to label the example, and then the learning algorithm is presented with the correct label and the hypothesis is updated. It is a different learning paradigm than batch learning where we ...

In this paper, a robust online sequential extreme learning machine (ROS-ELM) is proposed. It is based on the original OS-ELM with an adaptive selective ensemble framework. Two novel insights are proposed in this paper. First, a novel selective ensemble algorithm referred to as particle swarm opt imization selective ensemble (PSOSEN) is proposed. Noting that PSOSEN is a general selective ensembl...

In this paper, we study the classification problem in the situation where large volumes of training data become available sequentially (online learning). In medical imaging, this is typical, e.g., a 3D brain MRI dataset may be gradually collected from a patient population, and not all of the data is available when the analysis begins. First, we describe two common ensemble learning algorithms, ...

Using neural networks to represent value functions in reinforcement learning algorithms often involves a lot of work in hand-crafting the network structure, and tuning the learning parameters. In this paper, we explore the potential of using constructive neural networks in reinforcement learning. Constructive neural network methods are appealing because they can build the network structure base...