Series Expansions for Continuous-Time Markov Processes
نویسندگان
چکیده
We present update formulas that allow us to express the stationary distribution of a continuous-time Markov process with denumerable state space having generator matrix Q∗ through a continuous-time Markov process with generator matrix Q. Under suitable stability conditions, numerical approximations can be derived from the update formulas, and we show that the algorithms converge at a geometric rate. Applications to sensitivity analysis and bounds on perturbations are discussed as well. Numerical examples are presented to illustrate the efficiency of the proposed algorithm.
منابع مشابه
On $L_1$-weak ergodicity of nonhomogeneous continuous-time Markov processes
In the present paper we investigate the $L_1$-weak ergodicity of nonhomogeneous continuous-time Markov processes with general state spaces. We provide a necessary and sufficient condition for such processes to satisfy the $L_1$-weak ergodicity. Moreover, we apply the obtained results to establish $L_1$-weak ergodicity of quadratic stochastic processes.
متن کاملSeries Expansion Optimization
In this chapter we present an innovative approach to combine common results of Markov decision theory and series expansions for optimization. We call our algorithm the series expansion optimization algorithm. Our algorithm efficiently determines the optimal policy of continuous-time Markov decision processes exactly or alternatively up to an arbitrarily small precision. We prove its convergence...
متن کاملInformation content of signals using correlation function expansions of the entropy
Formally exact series expressions are derived for the entropy ~information content! of a time series or signal by making systematic expansions for the higher-order correlation functions using generalized Kirkwood and Markov superpositions. Termination of the series after two or three terms provides tractable and accurate approximations for calculating the entropy. Signals generated by a Gaussia...
متن کاملMarkov Processes
1 General Properties of Markov Processes 2 1.1 Discrete Time Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1.1 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1.2 Steady State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2 Continuous Time Processes . . . . ...
متن کاملThe Inventory System Management under Uncertain Conditions and Time Value of Money
This study develops a inventory model to determine ordering policy for deteriorating items with shortages under markovian inflationary conditions. Markov processes include process whose future behavior cannot be accurately predicted from its past behavior (except the current or present behavior) and which involves random chance or probability. Behavior of business or economy, flow of traffic, p...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Operations Research
دوره 58 شماره
صفحات -
تاریخ انتشار 2010