This paper is concerned with asymptotic properties of consensus-type algorithms for networked systems whose topologies switch randomly. The regime-switching process is modeled as a discrete-time Markov chain with a finite state space. The consensus control is achieved by using stochastic approximation methods. In the setup, the regime-switching process (the Markov chain) contains a rate paramet...

In the following, we will outline how to obtain invariants for hidden Markov and related models, based on an approach which, in its most prevalent application, served to solve the identifiability problem for hidden Markov processes (HMPs) in 1992 [13]. Some of its foundations had been layed in the late 50’s and early 60’s in order to get a grasp of problems related to that of identifying HMPs [...

This paper focuses on the Bayesian posterior mean estimates (or Bayes’ estimate) of the parameter set of Poisson hidden Markov models in which the observation sequence is generated by a Poisson distribution whose parameter depends on the underlining discrete-time time-homogeneous Markov chain. Although the most commonly used procedures for obtaining parameter estimates for hidden Markov models ...

We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this conditional signal is weakly ergodic when the signal is ergodic and the observations are nondegenerate. This permits a delicate exchange of the intersection a...

This article concerns maximum likelihood estimation for discrete time homogeneous nonparametric semi-Markov models with finite state space. In particular, we present the exact maximum likelihood estimator of the semi-Markov kernel which governs the evolution of the semi-Markov chain. We study its asymptotic properties in the following cases: (i) for one observed trajectory, when the length of t...

Among the class of discrete time Markovian processes, two models are widely used, the Markov chain and the Hidden Markov Model. A major di erence between these two models lies in the relation between successive outputs of the observed variable. In a visible Markov chain, these are directly correlated while in hidden models they are not. However, in some situations it is possible to observe both...

We investigate how and when to diversify capital over assets, i.e., the portfolio selection problem, from a signal processing perspective. To this end, we first construct portfolios that achieve the optimal expected growth in i.i.d. discrete-time two-asset markets under proportional transaction costs. We then extend our analysis to cover markets having more than two stocks. The market is modele...