The engineering and control of devices at the quantum mechanical level—such as those consisting of small numbers of atoms and photons—is a delicate business. The fundamental uncertainty that is inherently present at this scale manifests itself in the unavoidable presence of noise, making this a novel field of application for stochastic estimation and control theory. In this expository paper we ...

For many years the trend in macroeconomics has been towards models which are both explicitly stochastic and explicitly dynamic. With these models, researchers seek to replicate and explain observable properties of the major economic time series. One manifestation of this trend towards stochastic dynamic modeling has been increasing use of the inherently dynamic models developed in the field of ...

To address effectively the urgent societal need for safe structures and infrastructure systems under limited resources, science-based management of assets is needed. The overall objective of this two part study is to highlight the advanced attributes, capabilities and use of stochastic control techniques, and especially Partially Observable Markov Decision Processes (POMDPs) that can address th...

Stochastic processes that involve the creation and modification of objects and relations over time are widespread, but relatively poorly studied. For example, accurate fault diagnosis in factory assembly processes requires inferring the probabilities of erroneous assembly operations, but doing this efficiently and accurately is difficult. Modeled as dynamic Bayesian networks, these processes ha...

This paper presents a general class of dynamic stochastic optimization problems we refer to as Stochastic Depletion Problems. A number of challenging dynamic optimization problems of practical interest are stochastic depletion problems. Optimal solutions for such problems are difficult to obtain, both from a pragmatic computational perspective as also from a theoretical perspective. As such, si...

In this paper we examine the effect of the income tax on charitable giving. An important challenge in this literature has been to estimate the long-run response of giving to a persistent change in tax-price, which can be difficult to distinguish from intertemporal substitution arising from differences between current and expected future tax prices, arising for example due to transitory fluctuat...

This tutorial covers various approaches for modeling and analyzing dynamic processes in networks. Modeling the dynamic performance as a stochastic process, we apply tools from discrete and continues timeMarkov processes theory, renewal theory and queueing theory to analyze the long term, steady state, performance of the processes. Nonstochastic approaches include adversarial queueing theory, an...

ABSTRACT. The non-uniform global spread of emergent infectious diseases of humans is closely interrelated with the large-scale structure of the human population, and the human mobility process in the population structure. The mobile population becomes the vector for the disease. We present an SIRS stochastic dynamic epidemic process in a two scale structured population. The variability caused b...

Markov decision processes (MDPs) have proven to be popular models for decision-theoretic planning, but standard dynamic programming algorithms for solving MDPs rely on explicit, state-based specifications and computations. To alleviate the combinatorial problems associated with such methods, we propose new representational and computational techniques for MDPs that exploit certain types of prob...

Global and dynamic optimization of engineering problems usually involves complex physico-chemical models as constraints. These models are in general highly non-linear, resulting in multimodal optimization problems. The model may have discontinuous behavior and/or include a very large set of variables. As the complexity of the systems increases, equation-free modeling is becoming more common (Ke...