Often the medical decision maker will be faced with a sequential decision problem involving decisions that lead to different outcomes depending on chance. If the decision process involves many sequential decisions, then the decision problem becomes difficult to visualize and to implement. Decision trees are indispensable graphical tools in such settings. They allow for intuitive understanding o...

UNLABELLED Behavioral and neuroscientific data on reward-based decision making point to a fundamental distinction between habitual and goal-directed action selection. The formation of habits, which requires simple updating of cached values, has been studied in great detail, and the reward prediction error theory of dopamine function has enjoyed prominent success in accounting for its neural bas...

A sequential risk-taking paradigm used to identify real-world risk takers invokes both learning and decision processes. This article expands the paradigm to a larger class of tasks with different stochastic environments and different learning requirements. Generalizing a Bayesian sequential risk-taking model to the larger set of tasks clarifies the roles of learning and decision making during s...

! Abstract ! In the Berlin Cognitive Science meetings there was a brief session titled 'Sequential Sampling Models' (session 42) that attracted enormous interest: The room was packed and overflowed by a considerable amount, and both speakers and audience members felt there was much more to be presented and discussed than the few speakers and talks could allow. Quite a few people present agreed ...

8 AI MAGAZINE One of the central challenges in computational sustainability is how to allocate resources in order to optimize long-term objectives. An archetypal application is conservation planning: managers recommend patches of land in order to achieve long-term conservation of biodiversity. In this and similar applications, we typically have to make decisions over time: financial resources (...

We propose a simple analytical model called M time-scale Markov Decision Process (MMDP) for hierarchically structured sequential decision making processes, where decisions in each level in the M -level hierarchy are made in M different time-scales. In this model, the state space and the action space of each level in the hierarchy are non-overlapping with those of the other levels, respectively,...

Rapid “urbanization” (more than 50% of world’s population now resides in cities) coupled with the natural lack of coordination in usage of common resources (ex: bikes, ambulances, taxis, traffic personnel, attractions) has a detrimental effect on a wide variety of response (ex: waiting times, response time for emergency needs) and coverage metrics (ex: predictability of traffic/security patrols...

OF THE DISSERTATION Efficient Learning of Relational Models for Sequential Decision Making by Thomas J. Walsh Dissertation Director: Michael L. Littman The exploration-exploitation tradeoff is crucial to reinforcement-learning (RL) agents, and a significant number of sample complexity results have been derived for agents in propositional domains. These results guarantee, with high probability, ...

We present a new method for carrying out state estimation in multiagent settings that are characterized by continuous or large discrete state spaces. State estimation in multiagent settings involves updating an agent’s belief over the physical states and the space of other agents’ models. We factor out the models of the other agents and update the agent’s belief over these models, as exactly as...

We investigate several problems in scanning of multidimensional data arrays, such as universal scanning and prediction (“scandiction”, for short), and scandiction of noisy data arrays. These problems arise in several aspects of image and video processing, such as predictive coding, filtering and denoising. In predictive coding of images, for example, an image is compressed by coding the predict...