A Non-Markovian Process with Unbounded $p$-Variation

A Partially Observable Markovian Maintenance Process with Continuous Cost Functions

In this paper a two-state Markovian maintenance process where the true state is unknown will be considered. The operating cost per period is a continuous random variable which depends on the state of the process. If investigation cost is incurred at the beginning of any period, the system wit I be returned to the "in-control" state instantaneously. This problem is solved using the average crite...

Non-markovian process modelling with Echo State Networks

Reservoir Computing (RC) is a relatively recent architecture for using recurrent neural networks. It has shown interesting performance in a wide range of tasks despite the simple training rules. We use it here in a logistic regression (LogR) framework. Considering non-Markovian time series with a hidden variable, we show that RC can be used to estimate the transition probabilities at each time ...

Markovian quadratic and superquadratic BSDEs with an unbounded terminal condition

On a Markovian Process AlgebraPeter

Process algebras extended by a concept to present timing behaviour have been very recently proposed as a good modelling tool for the combined analysis of qualitative and quantitative system behaviour. We introduce a a process algebra including exponentially distributed time delays and a systematic approach to compute the underlying labelled transition system and continuous time Markov process. ...

Event-Learning with a Non-Markovian Controller

Recently a novel reinforcement learning algorithm called event-learning or E-learning was introduced. The algorithm based on events, which are defined as ordered pairs of states. In this setting, the agent optimizes the selection of desired sub-goals by a traditional value-policy function iteration, and utilizes a separated algorithm called the controller to achieve these goals. The advantage o...