We consider the problem of finding an optimal policy in a Markov decision process that maximises the expected discounted sum of rewards over an infinite time horizon. Since the explicit iterative dynamical programming scheme does not scale when increasing the dimension of the state space, a number of approximate methods have been developed. These are typically based on value or policy iteration...

The distributed parameters systems can be described by linear two-dimensional (dependent on two spatial directions) parabolic partial differential equations. Using the finite difference method a distributed parameters system can be transformed to a linear discrete state space model. The controller design based on this description is complicated because of the large dimension of the model. There...

a memory control for t-s fuzzy discrete-time systems with sto- chastic input delay is proposed in this paper. dierent from the common assumptions on the time delay in the existing literatures, it is assumed in this paper that the delays vary randomly and satisfy some probabilistic dis- tribution. a new state space model of the discrete-time t-s fuzzy system is derived by introducing some stocha...

directional simulation, among techniques for reliability analysis of engineering structures, is well known. in this context, directional simulation in load space, because of the considerable advantages of lower dimension of working space, is of considerable interest. as is expected due to structural resistance variability, the location of the relevant limit state(s) is no longer constant and mu...

An iterative procedure for constructing subsolutions of deterministic or stochastic optimal control problems in discrete time with continuous state space is introduced. The procedure generates a non-decreasing sequence of subsolutions, giving true lower bounds on the minimal costs. Convergence of the values at any fixed initial state is shown. 2000 AMS subject classifications: 60J05, 90-08, 90C...

A method based on the algorithmization of the geometric approach, using the equivalence between the maximal unobservable subspaces and the maximal (A, B)-invariant subspaces is proposed in order to solve the problem of group decoupling for linear multivariable systems. Necessary and sufficient conditions for compatibility of the maximal controllability subspaces are transformed to the necessary...

The problem of deadbeat control is now to find a feedback law F that will drive an arbitrary initial state Xo to Xk = 0 after a minimum number of steps k. All subsequent xi (for i > k) are then also zero, which explains the term 'deadbeat'. For standard state-space systems (i.e. E In) this is known to be equivalent to finding a feedback F such that all eigenvalues of A + BF lie at h = O, and A ...

In this paper we consider the following problem: viewing both a nonlinear system model and its linearization as mappings from input-tostate, we define the error between the state of the original nonlinear system and that of the linearization and find the region of the state space where this error is norm-bounded, in the integral-square (or L2-norm) sense. Using the Hamilton–Jacobi inequality we...

This paper considers the problem of synthesizing the finite-word-length (FWL) two-dimensional (2-D) state-space filter structures with minimum weighted sensitivity. Two kinds of frequency-weighted sensitivity measures, one based on a mixture of L1=L2 norms and the other a pure L2 norm, are defined in place of the usual sensitivity measure and an upper bound expressed in terms of 2-D weighted Gr...