Partially Observable Markov Decision Processes (POMDPs) provide a rich framework for sequential decision-making under uncertainty in stochastic domains. However, solving a POMDP is often intractable except for small problems due to their complexity. Here, we focus on online approaches that alleviate the computational complexity by computing good local policies at each decision step during the e...

Selecting a seller in e-markets is a tedious task that we might want to delegate to an agent. Many approaches to constructing such agents have been proposed, building upon different foundations (decision theory, trust modeling) and making use of different information (direct experience with sellers, reputation of sellers, trustworthiness of other buyers called advisors, etc.). In this paper, we...

While a large body of empirical results show that temporally-extended actions and options may significantly affect the learning performance of an agent, the theoretical understanding of how and when options can be beneficial in online reinforcement learning is relatively limited. In this paper, we derive an upper and lower bound on the regret of a variant of UCRL using options. While we first a...

This paper studies the extension of Harsanyi’s theorem (Harsanyi, 1955) in a framework involving uncertainty. It seeks to extend the aggregation result to a wide class of Monotonic Bernoullian and Archimedean preferences (Cerreia-Vioglio et al., 2011) that subsumes many models of choice under uncertainty proposed in the literature. An impossibility result is obtained, unless we are in the speci...

We study the loop clusters induced by Poissonian ensembles of Markov loops on a finite or countable graph (Markov loops can be viewed as excursions of Markov chains with a random starting point, up to re-rooting). Poissonian ensembles are seen as a Poisson point process of loops indexed by ‘time’. The evolution in time of the loop clusters defines a coalescent process on the vertices of the gra...

Imperfect knowledge of the world is one of the main issues AI systems have to deal with. In recent years there has been much interest in coping with such imperfections in planning systems. However, most works only emphasize one side of the problem, namely uncertainty. We argue that vagueness (or imprecision) is a fundamental issue which planning systems should also take into account. With this ...

We investigate the spectrum of the infinitesimal generator of the continuous time random walk on a randomly weighted oriented graph. This is the non-Hermitian random n× n matrix L defined by Ljk = Xjk if k 6= j and Ljj = − ∑ k 6=j Ljk , where (Xjk)j 6=k are i.i.d. random weights. Under mild assumptions on the law of the weights, we establish convergence as n → ∞ of the empirical spectral distri...

Researchers in artiicial intelligence and decision analysis share a concern with the construction of formal models of human knowledge and expertise. Historically, however, their approaches to these problems have diverged. Members of these two communities have recently discovered common ground: a family of graphical models of decision theory known as innuence diagrams or as belief networks. Thes...

We study the problem of learning Bayesian network structures from data. Koivisto and Sood (2004) and Koivisto (2006) presented algorithms that can compute the exact posterior probability of a subnetwork, e.g., a single edge, in O(n2n) time and the posterior probabilities for all n(n − 1) potential edges in O(n2n) total time, assuming that the in-degree, i.e., the number of parents per node, is ...