Considering difference equations in discrete space instead of differential equations in Euclidean space, we investigate a probabilistic formula for the solution of the Dirichlet problem for biharmonic functions. This formula involves the expectation of a weighted sum of the pay-offs at the successive times at which the Markov chain is in the complement of the domain. To make the infinite sum co...

The AMP Markov property is a recently proposed alternative Markov property for chain graphs. In the case of continuous variables with a joint multivariate Gaussian distribution, it is the AMP rather than the earlier introduced LWF Markov property that is coherent with data-generation by natural block-recursive regressions. In this paper, we show that maximum likelihood estimates in Gaussian AMP...

Jump Markov linear systems (JMLS) are linear systems whose parameters evolve with time according to a finite state Markov chain. We present three original deterministic and stochastic iterative algorithms for optimal state estimation of JMLS whose computational complexity at each iteration is linear in the data length. The first algorithm yields conditional mean estimates. The second algorithm ...

Markov population models (MPMs) are a widely used modelling formalism in the area of computational biology and related areas. The semantics of a MPM is an infinitestate continuous-time Markov chain. In this paper, we use the established continuous stochastic logic (CSL) to express properties of Markov population models. This allows us to express important measures of biological systems, such as...

Continuous time modified Cox-Ingersoll-Ross (1985) stochastic model is employed, combining with Hamilton (1989) type Markov regime switching framework, to study daily foreign exchange rates, where all parameter values depend on the value of a continuous time Markov chain. The Expectation-Maximization algorithm is extended, generalized, applied to a more general class of regime switching models ...

Canonical forms of Markovian distributions and processes provide an efficient way of describing these structures by eliminating the redundancy of general representations. Canonical forms of order 2 stationary Markov arrival processes (MAPs) have already been established for both continuous and discrete time. In this paper we present canonical form of continuous time non-stationary MAPs of order...

Let t be a continuous Markov chain on N states. Consider adjoining a Brownian motion with this Markov chain so that the drift and the variance take diierent values when t is in diierent states. This new process Z t is a hidden Markov process. We study the probability distribution of the rst passage time for Z t. Our result, when applied to the stock market, provides an explicit mathematical int...

Aiming at reduction of complexity, this work is concerned with two-time-scale Markov chains and applications to quasi-birth-death queues. Asymptotic expansions of probability vectors are constructed and justified. Lumping all states of the Markov chain in each subspace into a single state, an aggregated process is shown to converge to a continuous-time Markov chain whose generator is an average...

Markov chains are widely used in the context of performance and reliability evaluation of systems of various nature. Model checking of such chains with respect to a given (branching) temporal logic formula has been proposed for both the discrete [8] and the continuous time setting [1], [3]. In this short paper, we describe the prototype model checker E T MC for discrete and continuous-time Mark...