This paper surveys the theoretical developments in the field of stochastic process algebras, process algebras where action occurrences may be subject to a delay that is determined by a random variable. A huge class of resource-sharing systems — like large-scale computers, client-server architectures, networks — can accurately be described using such stochastic specification formalisms. The main...

In this paper we present a model for the hidden Markovian bandit problem with linear rewards. As opposed to current work on bandits, do not assume that state is known decision maker before making decision. Furthermore, structural side information where knows in advance there are two types of states; one common all arms and evolves according distribution, other unique each arm distributed an i.i...

This paper examines a number of solution methods for decision processes with non-Markovian rewards (NMRDPs). They all exploit a temporal logic specification of the reward function to automatically translate the NMRDP into an equivalent Markov decision process (MDP) amenable to well-known MDP solution methods. They differ however in the representation of the target MDP and the class of MDP solut...

This article describes the batch Markovian arrival process (BMAP), a point process that is characterized by Markov-modulated batch arrivals of random size. The BMAP is a generalization of many well-known processes including the Markovian arrival process (MAP), the Poisson process, and the Markov-modulated Poisson process. It provides a common framework for modeling arrival processes in a variet...

The objective of this paper is to bring systematic methods for scenario tree generation to the attention of the Process Systems Engineering community. In this paper, we focus on a general, data-driven optimization-based method for generating scenario trees, which does not require strict assumptions on the probability distributions of the uncertain parameters. This method is based on the Moment ...

This paper examines a number of solution meth­ ods for decision processes with non-Markovian rewards (NMRDPs). Tlu::y all t:xploit a temporal logic specification of the reward function to au­ tomatically translate the NMROP into an equiv­ alent Markov decision process (MOP) amenable to well-known MOP solution methods. They dif­ fer however in the representation of the target MOP and the class o...

This chapter gives credence to the introduction of optimal control theory into oil spill modeling and develops an optimization process that will aid in the effective decisionmaking in marine oil spill management. The purpose of the optimal control theory is to determine the control policy that will optimize (maximize or minimize) a specific per‐ formance criterion, subject to the constraints im...