Jump Markov linear systems (JMLS) are linear systems whose parameters evolve with time according to a finite state Markov chain. We present three original deterministic and stochastic iterative algorithms for optimal state estimation of JMLS whose computational complexity at each iteration is linear in the data length. The first algorithm yields conditional mean estimates. The second algorithm ...

Markov population models (MPMs) are a widely used modelling formalism in the area of computational biology and related areas. The semantics of a MPM is an infinitestate continuous-time Markov chain. In this paper, we use the established continuous stochastic logic (CSL) to express properties of Markov population models. This allows us to express important measures of biological systems, such as...

In this paper, we formulate six different resolutions of a continuous-time approximation of the Wright-Fisher sample genealogical process. We derive Markov chains for the six different approximations in the spirit of J.F.C. Kingman. These Markov chains are essential for inference methods. One of the resolutions is the well-known n-coalescent due to Kingman. The second resolution was mentioned b...

Aldous’ spectral gap conjecture asserts that on any graph the random walk process and the random transposition (or interchange) process have the same spectral gap. We prove the conjecture using a recursive strategy. The approach is a natural extension of the method already used to prove the validity of the conjecture on trees. The novelty is an idea based on electric network reduction which red...

In this paper we present an algorithm for pricing barrier options in one-dimensional Markov models. The approach rests on the construction of an approximating continuous-time Markov chain that closely follows the dynamics of the given Markov model. We illustrate the method by implementing it for a range of models, including a local Lévy process and a local volatility jump-diffusion. We also pro...

investors use different approaches to select optimal portfolio. so, optimal investment choices according to return can be interpreted in different models. the traditional approach to allocate portfolio selection called a mean - variance explains. another approach is markov chain. markov chain is a random process without memory. this means that the conditional probability distribution of the nex...

In this paper we present an algorithm for pricing barrier options in one-dimensional Markov models. The approach rests on the construction of an approximating continuous-time Markov chain that closely follows the dynamics of the given Markov model. We illustrate the method by implementing it for a range of models, including a local Lévy process and a local volatility jump-diffusion. We also pro...

In this paper, we consider denumerable state continuous time Markov decision processes with (possibly unbounded) transition and cost rates under average criterion. We present a set of conditions and prove the existence of both average cost optimal stationary policies and a solution of the average optimality equation under the conditions. The results in this paper are applied to an admission con...

We present a logical formalism for expressing properties of continuous time Markov chains. The semantics for such properties arise as a natural extension of previous work on discrete time Markov chains to continuous time. The major result is that the veriication problem is decidable; this is shown using results in algebraic and transcendental number theory.