1 Basic aspects of continuous time Markov chains 3 1.1 Markov property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Regular jump chain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Holding times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.4 Poisson process . . . . . . . . . . . . . . ....

In this paper, we develop approximation methods for solving stochastic control problems. The systems under consideration involve regime switching, modulated by a continuous-time Markov chain. Using Markov chain approximation techniques, we construct discrete-time Markov chains having two components. In addition to convergence of the procedure, numerical experiments and remarks on controlled var...

We consider a Hidden Markov Model (HMM) where the integrated continuous-time Markov chain can be observed at discrete time points perturbed by a Brownian motion. The aim is to derive a filter for the underlying continuous-time Markov chain. The recursion formula for the discrete-time filter is easy to derive, however involves densities which are very hard to obtain. In this paper we derive exac...

We prove that under certain mild assumptions, the entropy rate of a hidden Markov chain, observed when passing a finite-state stationary Markov chain through a discretetime continuous-output channel, is jointly analytic as a function of the input Markov chain parameters and the channel parameters. In particular, as consequences of the main theorems, we obtain analyticity for the entropy rate as...

Investors use different approaches to select optimal portfolio. so, Optimal investment choices according to return can be interpreted in different models. The traditional approach to allocate portfolio selection called a mean - variance explains. Another approach is Markov chain. Markov chain is a random process without memory. This means that the conditional probability distribution of the nex...

We consider stochastic differential equations with Markovian switching (SDEwMS). An SDEwMS is an ordinary stochastic differential equation with drift and diffusion coefficients depending not only on the current state of the solution but also on the current state of a right-continuous Markov chain taking values in a finite state space. Let W be a one-dimensional Brownian motion on the unit inter...

'Pae multiple time scale decomposition of discrete time, finite state Markov chains is addressed. In [1, 2], the behavior of a continuous time Markov chain is approximated using a fast time scale, e-independent, continuous time process, and a reduced order perturbed process. The procedure can then be iterated to obtain a complete multiple time scale decomposition. In the discrete time case pres...

in a finite stationary markov chain, transition probabilities may depend on some explanatoryvariables. a similar problem has been considered here. the corresponding posteriors are derived andinferences are done using these posteriors. finally, the procedure is illustrated with a real example.

Aldous' spectral gap conjecture asserts that on any graph the random walk process and the random transposition (or interchange) process have the same spectral gap. We prove the conjecture using a recursive strategy. The approach is a natural extension of the method already used to prove the validity of the conjecture on trees. The novelty is an idea based on electric network reduction, which re...