This paper examines approaches to representing uncertainty in reputation systems for electronic markets with the aim of constructing a decision theoretic framework for collecting information about selling agents and making purchase decisions in the context of a social reputation system. A selection of approaches to representing reputation using Dempster-Shafter Theory and Bayesian probability a...

Motivated by the interest in relational reinforcement learning, we introduce a novel representation formalism, called logical Markov decision programs (LOMDPs), that integrates Markov Decision Processes with Logic Programs. Using LOMDPs one can compactly and declaratively represent complex relational Markov decision processes. Within this framework we then develop a theory of reinforcement lear...

Many problems of sequential decision making under uncertainty, whose underlying probabilistic structure has a Markov chain, can be set up as Markov Decision Problems (MDPs). However, when their underlying transition mechanism cannot be characterized by the Markov chain alone, the problems may be set up as Semi-Markov Decision Problems (SMDPs). The framework of dynamic programming has been used ...

We report an experiment on a decision task by SAMUELSON and BAZERMAN (1985). Subjects submit a bid for an item with an unknown value. A winner’s curse phenomenon arises when subjects bid too high and make losses. Learning direction theory can account for this. However, other influences on behaviour can also be identified. We introduce impulse balance theory to make quantitative predictions on t...

This paper considers the optimal control of time varying continuous time Markov chains whose transition rates are themselves Markov processes. In one set of problems the solution of an ordinary differential equation is shown to determine the optimal performance and feedback controls, while some other cases are shown to lead to singular optimal control problems which are more difficult to solve....

We present an approach for designing decision procedures based on the reduction of complex theories to simpler ones. Specifically, we define reduction functions as a tool for reducing the satisfiability problem of a complex theory to the satisfiability problem of a simpler one. Reduction functions allow us to reduce the theory of lists to the theory of constructors, the theory of arrays to the ...

This paper presents a method for the evaluation of a posteriori (historical) multi-variate multi-stage optimal trading under transaction costs and a diversification constraint. Starting from a given amount of money in some currency, we analyze the stage-wise optimal allocation over a time horizon with potential investments in multiple currencies and various assets, such as, for example, assets ...

One of the most difficult aspects of modeling complex dilemmas in decision-analytic terms is composing a diagram of relevance relations from a set of domain concepts. Decision mod­ els in domains such as medicine, however, ex­ hibit certain prototypical patterns that can guide the modeling process. Medical con­ cepts can be classified according to semantic types that have characteristic positio...

In a recent approach, we proposed to model an access control mechanism as a Markov Decision Process, thus claiming that in order to make an access control decision, one can use well-defined mechanisms from decision theory. We present in this paper an implementation of such mechanism, using the open-source solver GLPK, and we model the problem in the GMPL language. We illustrate our approach wit...

Maintenance can be the factor of either increasing or decreasing system's availability, so it is valuable work to evaluate a maintenance policy from cost and availability point of view, simultaneously and according to decision maker's priorities. This study proposes a Partially Observable Markov Decision Process (POMDP) framework for a partially observable and stochastically deteriorating syste...