Finite Generalized Markov Programming.

Lecture 1: Generalized Ray Knight Theorem for Finite Markov Chains

Finite Markov Chains

This paper provides basic information and theorems about finite Markov chains. The inspiration for this paper came from Professor Laci Babai’s Discrete Mathematics lecture notes from the REU program of 2003. After reading through his chapter on Markov chains, I decided to proceed by answering as many exercises from the notes as possible. Below is what I have finished.

Finite Markov Chain Analysis

Finite, discrete-time Markov chain models of genetic algorithms have been used successfully in the past to understand the complex dynamics of a simple GA. Markov chains can exactly model the GA by accounting for all of the stochasticity introduced by various GA operators, such as initialization, selection, crossover, and mutation. Although such models quickly become unwieldy with increasing pop...

Markov-switching generalized additive models

We consider Markov-switching regression models, i.e. models for time series regression analyses where the functional relationship between covariates and response is subject to regime switching controlled by an unobservable Markov chain. Building on the powerful hidden Markov model machinery and the methods for penalized B-splines routinely used in regression analyses, we develop a framework for...

Interactive Generalized Semi-Markov Processes

We have previously introduced Generalized Semi-Markovian Process Algebra (GSMPA), a process algebra based on ST semantics which is capable of expressing durational actions, where durations are expressed by general probability distributions. In this paper we introduce the calculus of Interactive Generalized Semi-Markov Processes (IGSMP), a variant of GSMPA where synchronizable actions are restri...