The paper considers a discrete-time linear quadratic Gaussian model with constrained control. It is formulated with Markov systems. With the derivative equation, a performance gradient with respect to control parameters is estimated from a sample path. Then a learning algorithm is proposed to obtain a suboptimal feedback policy in affine linear form. The learning algorithm can be implemented on...

In this paper we study the solution to optimal control problems for constrained discrete-time linear hybrid systems based on quadratic or linear performance criteria. The aim of the paper it twofold. First we give basic theoretical results on the structure of the optimal state feedback solution and of the value function. Second we describe how the state feedback optimal control law can be effic...

In this paper we study the solution to optimal control problems for discrete time linear hybrid systems. First, we prove that the closed form of the state-feedback solution to finite time optimal control based on quadratic or linear norms performance criteria is a time-varying piecewise affine feedback control law. Then, we give an insight into the structure of the optimal state-feedback soluti...

In this paper, the finite-horizon near optimal adaptive regulation of linear discrete-time systems with unknown system dynamics is presented in a forward-in-time manner by using adaptive dynamic programming and Q-learning. An adaptive estimator (AE) is introduced to relax the requirement of system dynamics, and it is tuned by using Q-learning. The time-varying solution to the Bellman equation i...

This paper deals with the discrete-time infinite-horizon linear quadratic problem with indefinite cost criterion. Given a discrete-time linear system, an indefinite costfunctional and a linear subspace of the state space, we consider the problem of minimizing the costfunctional over all inputs that force the state trajectory to converge to the given subspace. We give a geometric characterizatio...

This paper studies the discrete-time linear quadratic regulation problem for switched linear systems (DLQRS) based on dynamic programming approach. The unique contribution of this paper is the analytical characterizations of both the value function and the optimal control strategies for the DLQRS problem. Based on the particular structures of these analytical expressions, an efficient algorithm...

We present a receding horizon control scheme that utilizes a discrete time version of a nonlinear, iterative, projection-based optimization routine. The routine utilizes a projection operator to ensure feasible system trajectories at each iteration of the optimization resulting in valid control signals even when constraints on computation time prevent full convergence. An additional feature of ...

A particular class of uncertain linear discrete-time periodic systems is considered. The problem of robust stabilization of real polytopic linear discrete-time periodic systems via a periodic state-feedback control law is tackled here, along with H2 performance optimization. Using additional slack variables and the periodic Lyapunov lemma, an extended sufficient condition of robust H2 stabiliza...

A problem of robust guaranteed cost control of stochastic discrete-time systems with parametric uncertainties under Markovian switching is considered. The jump Markovian switching is modelled by a discrete-time Markov chain and the noise or stochastic environmental disturbance is modelled by a sequence of identically independently normally distributed random variables. Using linear matrix inequ...

The paper establishes a non-variational procedure in order to obtain the solution to an optimal control problem. The optimal control refers to a quadratic criterion with finite final time, regarding a timevariant linear system, for continuous and discrete time cases. The proposed solution is more convenient for implementation by comparison with the classical ones. The indicated optimal controll...