This paper investigates stochastic and adversarial combinatorial multi-armed bandit problems. In the stochastic setting, we first derive problemspecific regret lower bounds, and analyze how these bounds scale with the dimension of the decision space. We then propose COMBUCB, algorithms that efficiently exploit the combinatorial structure of the problem, and derive finitetime upper bound on thei...

We consider the problem of online linear regression on arbitrary deterministic sequences when the ambient dimension d can be much larger than the number of time rounds T . We introduce the notion of sparsity regret bound, which is a deterministic online counterpart of recent risk bounds derived in the stochastic setting under a sparsity scenario. We prove such regret bounds for an online-learni...

We investigate the behavior in N of the N–particle entropy functional for Kac’s stochastic model of Boltzmann dynamics, and its relation to the entropy function for solutions of Kac’s one dimensional nonlinear model Boltzmann equation. We prove a number of results that bring together the notion of propagation of chaos, which Kac introduced in the context of this model, with the problem of estim...

This paper presents the first generalization bounds for time series prediction with a non-stationary mixing stochastic process. We prove Rademacher complexity learning bounds for both average-path generalization with non-stationary β-mixing processes and path-dependent generalization with non-stationary φ-mixing processes. Our guarantees are expressed in terms of βor φ-mixing coefficients and a...

Computing deterministic performance guarantees is a defining issue for systems with hard realtime constraints, like reactive embedded systems. In this paper, we use burst-rate constrained arrivals and ratelatency servers to deduce tight worst-case delay bounds in tandem networks under arbitrary multiplexing. We present a constructive method for computing the exact worst-case delay, which we pro...

The success of deep learning has led to a rising interest in the generalization property of the stochastic gradient descent (SGD) method, and stability is one popular approach to study it. Existing works based on stability have studied nonconvex loss functions, but only considered the generalization error of the SGD in expectation. In this paper, we establish various generalization error bounds...

The method of moments has been proposed as a potential means to reduce the dimensionality of the chemical master equation (CME) appearing in stochastic chemical kinetics. However, attempts to apply the method of moments to the CME usually result in the so-called closure problem. Several authors have proposed moment closure schemes, which allow them to obtain approximations of quantities of inte...

The present paper provides some new stochastic inequalities for the characteristics of the M/GI/1/n and GI/M/1/n loss queueing systems. These stochastic inequalities are based on substantially deepen upand down-crossings analysis, and they are stronger than the known stochastic inequalities obtained earlier. Specifically, for a class of GI/M/1/n queueing system, two-side stochastic inequalities...

This paper presents a new method for synthesizing stochastic control Lyapunov functions for a class of nonlinear stochastic control systems. The technique relies on a transformation of the classical nonlinear Hamilton–Jacobi–Bellman partial differential equation to a linear partial differential equation for a class of problems with a particular constraint on the stochastic forcing. This linear ...

Traditional approaches to solving stochastic optimal control problems involve dynamic programming, and solving certain optimality equations. When recast as stochastic programming problems, structural aspects such as convexity are regained, and solution procedures based on decomposition and duality may be exploited. This paper explores a class of stationary, infinite-horizon stochastic optimizat...