3 Discrete stochastic models are widely used to describe current engineer4 ing and logistics problems. The stochastic simulation of such models can get 5 very expensive if the models are stiff or rare system events are of interest. 6 Proxels are a state space-based simulation technique that does not have these 7 drawbacks. They implicitly use a discrete-time Markov chain to determin8 istically ...

This work is devoted to analyzing adaptive filtering algorithms with the use of sign-regressor for randomly time-varying parameters (a discrete-time Markov chain). In accordance with different adaption and transition rates, we analyze the corresponding asymptotic properties of the algorithms. When the adaptation rate is in line with the transition rate, we obtain a limit of a Markov switched di...

The coefficients of the Markov binomial distribution are solved in terms of the underlying state-contingent probabilities of the Markov chain. This will be useful for researchers concerned with the analysis of data generated by a discrete Markov chain. The paper exploits the regenerative nature of the problem and solves the difference equations known to define the distribution. The LLN, CLT and...

Time reversibility plays an important role in the analysis of continuous and discrete time Markov chains (DTMCs). Specifically, the computation of the stationary distribution of a reversible Markov chain has been proved to be very efficient and does not require the solution of the system of global balance equations. A DTMC is reversible when the processes at forward and reversed time are probab...

We develop the connection between large deviation theory and more applied approaches to stochastic hybrid systems by highlighting a common underlying Hamiltonian structure. A stochastic hybrid system involves the coupling between a piecewise deterministic dynamical system in Rd and a time-homogeneous Markov chain on some discrete space Γ. We assume that the Markov chain on Γ is ergodic, and tha...

We will be closely following the book ”Essentials of Stochastic Processes”, 2nd Edition, by Richard Durrett, for the topic ‘Finite Discrete time Markov Chains’ (FDTM). These preliminary notes are for giving an overview, and for providing some guidance on what to aim at while studying this topic. An FDTM has a finite number of states on which it can rest at discrete times 1, 2, · · · . Usually t...

Models that aim to optimize the design of supply chain networks have gained more interest in the supply chain literature. Mixed-integer linear programming and discrete-event simulation are widely used for such an optimization problem. We present a hybrid approach to support decisions for supply chain network design using a combination of analytical and discrete-event simulation models. The prop...

We study the problem of finite-horizon probabilistic invariance for discrete-time Markov processes over general (uncountable) state spaces. We compute discrete-time, finite-state Markov chains as formal abstractions of general Markov processes. Our abstraction differs from existing approaches in two ways. First, we exploit the structure of the underlying Markov process to compute the abstractio...

This paper deals with a complete bipartite matching problem with the objective of finding an optimal matching that maximizes a certain generic predefined utility function on the set of all matchings. After proving the NP-hardness of the problem using reduction from the 3-SAT problem, we propose a randomized algorithm based on Markov Chain Monte Carlo (MCMC) technique for solving this. We sample...