We propose a novel model for semisupervised classification by bringing in reward in Markov random walks. Both angle and distance metrics for vectors are combined in this model. Taking advantage of absorbing states, transient analysis of Markov chain can be performed more easily, based on Markov random walks. Diffusion of unlabeled data points makes our approach suffer less from error propagatio...

The distribution theory for reward functions on semi-Markov processes has been of interest since the early 1960s. The relevant asymptotic distribution theory has been satisfactorily developed. On the other hand, it has been noticed that it is difficult to find exact distribution results which lead to the effective computation of such distributions. Note that there is no satisfactory exact distr...

Markov decision processes (MDP) have become one of the standard models for decision-theoretic planning problems under uncertainty. In its standard form, rewards are assumed to be numerical additive scalars. In this paper, we propose a generalization of this model allowing rewards to be functional. The value of a history is recursively computed by composing the reward functions. We show that sev...

The Stochastic Petri Net Package (SPNP) [2] is a versatile modeling tool for solution of Stochastic Petri Net (SPN) models. The SPN models are described in the input language for SPNP called CSPL (C-based SPN Language) which is an extension of the C programming language [8] with additional constructs which facilitate easy description of SPN models. Moreover, if the user does not want to describ...

In this paper we study two basic optimality criteria used in the theory of zero-sum semi-Markov games. According to the first one, the average reward for player 1 is the lim sup of the expected total rewards over a finite number of jumps divided by the expected cumulative time of these jumps. According to the second definition, the average reward (for player 1) is the lim sup of the expected to...

Growing amount of information on biological sequences has made application of statistical approaches necessary for modeling and estimation of their functions. In this paper, sensitivity and specificity of the first and second Markov chains for prediction of genes was evaluated using the complete double stranded  DNA virus. There were two approaches for prediction of each Markov Model parameter,...

The need for the combined performance and reliability analysis of fault tolerant systems is increasing. The common approach to formulating and solving such problems is to use (semi-)Markov reward models. However, the large size of size of state spaces is a problem that plagues Markovian models. Combinatorial models have been used for modeling reliability and availability of complex systems with...

This paper investigates the asymmetric effects of monetary policy on economic growth over business cycles in Iran. Estimating the models using the Hamilton (1989) Markov-switching model and by employing the data for 1960-2012, the results well identify two regimes characterized as expansion and recession. Moreover, the results show that an expansionary monetary policy has a positive and statist...

Costs and rewards are important ingredients for many types of systems, modelling critical aspects like energy consumption, task completion, repair costs, and memory usage. This paper introduces Markov reward automata, an extension of Markov automata that allows the modelling of systems incorporating rewards (or costs) in addition to nondeterminism, discrete probabilistic choice and continuous s...