We consider the problem of model-free reinforcement learning for infinite-horizon discounted Markov Decision Processes (MDPs) with a continuous state space and unknown transition kernels, when only a single sample path of the system is available. We focus on the classical approach of Q-learning where the goal is to learn the optimal Q-function. We propose the Nearest Neighbor Q-Learning approac...

Given a set of users U , a set of facilities F , and a query facility q, a reverse nearest neighbors (RNN) query retrieves every user u for which q is its closest facility. Since q is the closest facility of u, the user u is said to be influenced by q. In this paper, we propose a relaxed definition of influence where a user u is said to be influenced by not only its closest facility but also ev...

We develop a general framework for margin-based multicategory classification in metric spaces. The basic work-horse is a margin-regularized version of the nearest-neighbor classifier. We prove generalization bounds that match the state of the art in sample size n and significantly improve the dependence on the number of classes k. Our point of departure is a nearly Bayes-optimal finite-sample r...

We present a randomized algorithm for the approximate nearest neighbor problem in d-dimensional Euclidean space. Given N points {x(j)} in R(d), the algorithm attempts to find k nearest neighbors for each of x(j), where k is a user-specified integer parameter. The algorithm is iterative, and its running time requirements are proportional to T·N·(d·(log d) + k·(d + log k)·(log N)) + N·k(2)·(d + l...

This paper presents kinetic data structures (KDS’s) for maintaining the Semi-Yao graph, all the nearest neighbors, and all the (1 + )-nearest neighbors of a set of moving points in R. Our technique provides the first KDS for the SemiYao graph in R. It generalizes and improves on the previous work on maintaining the Semi-Yao graph in R. Our KDS for all nearest neighbors is deterministic. The bes...