This paper is the rst to introduce an algorithm to compute stationary equilibria in stochastic games, and shows convergence of the algorithm for almost all such games. Moreover, since in general the number of stationary equilibria is overwhelming, we pay attention to the issue of equilibrium selection. We do this by extending the linear tracing procedure to the class of stochastic games, called...

We study the problem of solving simple stochastic games, and give both an interesting new algorithm and a hardness result. We show a reduction from fine approximation of simple stochastic games to coarse approximation of a polynomial sized game, which can be viewed as an evidence showing the hardness to approximate the value of simple stochastic games. We also present a randomized algorithm tha...

For n-person perfect information stochastic games and for n-person stochastic games with Additive Rewards and Additive Transitions (ARAT) we show the existence of pure limiting average equilibria. Using a similar approach we also derive the existence of limiting average "-equilibria for two-person switching control stochastic games. The orderreld property holds for each of the classes mentioned...

We consider a class of fully stochastic and fully distributed algorithms, that we prove to learn equilibria in games. Indeed, we consider a family of stochastic distributed dynamics that we prove to converge weakly (in the sense of weak convergence for probabilistic processes) towards their mean-field limit, i.e an ordinary differential equation (ODE) in the general case. We focus then on a cla...

In this paper we consider a class of hybrid stochastic games with the piecewise openloop information structure. These games are indexed over a parameter ε which represents the time scale ratio between the stochastic (jump process) and the deterministic (differential state equation) parts of the dynamical system. We study the limit behavior of Nash equilibrium solutions to the hybrid stochastic ...

We prove that optimal strategies exist in perfect-information stochastic games with finitely many states and actions and tail winning conditions. Introduction We prove that optimal strategies exist in perfect-information stochastic games with finitely many states and actions and tail winning conditions. This proof is different from the algorithmic proof sketched in [Hor08]. 1. Perfect-Informati...

Design and control of computer systems that operate in uncertain, competitive or adversarial, environments can be facilitated by formal modelling and analysis. In this paper, we focus on analysis of complex computer systems modelled as turn-based 21/2-player games, or stochastic games for short, that are able to express both stochastic and non-stochastic uncertainty. We offer a systematic overv...

Continuous action space games form a natural extension to normal form games with finite action sets. However, whilst learning dynamics in normal form games are now well studied, it is not until recently that their continuous action space counterparts have been examined. We extend stochastic fictitious play to the continuous action space framework. In normal form games the limiting behaviour of ...

We study optimal stochastic control problems under model uncertainty. We rewrite such problems as (zero-sum) stochastic differential games of forward-backward stochastic differential equations. We prove general stochastic maximum principles for such games, both in the zero-sum case (finding conditions for saddle points) and for the non-zero sum games (finding conditions for Nash equilibria). We...

Infinite duration games with regular conditions are one of the crucial tools in the areas of verification and synthesis. In this paper we consider a branching variant of such games – the game contains branching vertices that split the play into two independent sub-games. Thus, a play has the form of an infinite tree. The winner of the play is determined by a winning condition specified as a set...