We extend Q-learning to a noncooperative multiagent context, using the framework of generalsum stochastic games. A learning agent maintains Q-functions over joint actions, and performs updates based on assuming Nash equilibrium behavior over the current Q-values. This learning protocol provably converges given certain restrictions on the stage games (defined by Q-values) that arise during learn...

This paper describes an approach to reinforcement learning in multiagent general-sum games in which a learner is told to treat each other agent as either a \friend" or \foe". This Q-learning-style algorithm provides strong convergence guarantees compared to an existing Nash-equilibrium-based learning rule.

We extend the potential-based shapingmethod fromMarkov decision processes to multi-player general-sum stochastic games. We prove that the Nash equilibria in a stochastic game remains unchanged after potential-based shaping is applied to the environment. The property of policy invariance provides a possible way of speeding convergence when learning to play a stochastic game.

Often problems arise where multiple self-interested agents with individual goals can coordinate their actions to improve their outcomes. We model these problems as general sum stochastic games. We develop a tractable approximation algorithm for computing subgame-perfect correlated equilibria in these games. Our algorithm is an extension of standard dynamic programming methods like value iterati...

Multi-agent games are becoming an increasingly prevalent formalism for the study of electronic commerce and auctions. The speed at which transactions can take place and the growing complexity of electronic marketplaces makes the study of computationally simple agents an appealing direction. In this work, we analyze the behavior of agents that incrementally adapt their strategy through gradient ...

Over the past few decades the quest for algorithms to compute Nash equilibria in general-sum stochastic games has intensified and several important algorithms (cf. [9], [12], [16], [7]) have been proposed. However, they suffer from either lack of generality or are intractable for even medium sized problems or both. In this paper, we first formulate a non-linear optimization problem for stochast...

SOSTOOLS is a MATLAB toolbox for constructing and solving sum of squares programs. It can be used in combination with semidefinite programming software, such as SeDuMi, to solve many continuous and combinatorial optimization problems, as well as various control-related problems. This paper provides an overview on sum of squares programming, describes the primary features of SOSTOOLS, and shows ...

The general sum-connectivity index is a molecular descriptor defined as [Formula: see text], where [Formula: see text] denotes the degree of a vertex [Formula: see text], and α is a real number. Let X be a graph; then let [Formula: see text] be the graph obtained from X by adding a new vertex [Formula: see text] corresponding to each edge of X and joining [Formula: see text] to the end vertices...

A general abstract duality result is proposed for equations which are governed by the sum of two operators (possibly multivalued). It allows to unify a large number of variational duality principles, including the Clarke-Ekeland least dual action principle and the Singer-Toland duality. Moreover, it offers a new duality approach to some central questions in the theory of variational inequalitie...

In many situations, multiagent systems must deal with partial observability that agents have in the environment. In these cases, finding optimal solutions is often intractable for more than two agents and approximated solutions are often the only way to solve these problems. The models known to represent this kind of problem is Partially Observable Stochastic Game (POSG). Such a model is usuall...