It is well known that stochastic control systems can be viewed as Markov decision processes (MDPs) with continuous state spaces. In this paper, we propose to apply the policy iteration approach in MDPs to the optimal control problem of stochastic systems. We first provide an optimality equation based on performance potentials and develop a policy iteration procedure. Then we apply policy iterat...

This paper proposes a novel adaptive control law for nonlinear systems using Takagi-Sugeno fuzzy system. Takagi-Sugeno fuzzy system is used to identify nonlinear system components theta alpha and theta beta. Stable Indirect Adaptive control law is such that it has two control components one is certainty equivalence control and other is sliding mode control. Sliding mode controller is used to en...

We consider a base stock inventory control system serving two customer classes whose demands are generated by two independent compound renewal processes. We show how to derive order and volume fill rates of each class. Based on assumptions about first order stochastic dominance we prove when one customer class will get the best service. That theoretical result is validated through a series of n...

It is shown that any stabilizing, certainty equivalence control used within an adaptive control system, causes the familiar interconnection of a controlled process and associated output estimator to be detectable through the estimator's output error e p , for every frozen value of the index or parameter vector p upon which both the estimator and controller dynamics depend. The fact that certain...

We consider a team problem, with two decision makers for simplicity, where the uncertainties are dealt with in a minimax fashion rather than in a stochastic framework. We do not assume that the players exchange information at any time. Thus new ideas are necessary to investigate that situation. In contrast with the classical literature we do not use necessary conditions, but investigate to what...

This paper studies the equational theory of bisimulation equivalence over the process algebra BCCSP extended with the priority operator of Baeten, Bergstra and Klop. It is proven that, in the presence of an infinite set of actions, bisimulation equivalence has no finite, sound, ground-complete equational axiomatization over that language. This negative result applies even if the syntax is exten...

The standard approach in feedback control systems is to sample and control periodically. For some applications, such as networked control systems or severely energy-constrained systems, it could be advantageous to instead use event-based control schemes. Aperiodic control (that is, event-based control with no specified minimum inter-event time) of first-order stochastic systems has been investi...

We consider a stochastic control problem where the set of strict (classical) controls is not necessarily convex, and the system is governed by a nonlinear backward stochastic differential equation. By introducing a new approach, we establish necessary as well as sufficient conditions of optimality for two models. The first concerns the relaxed controls, who are measure-valued processes. The sec...

The general task of optimal adaptive control with recursive identification (self-tuning control) is very complicated problem. This problem is solved usually by the separation of identification and control – the Certainty Equivalency (CE) Principle. The aim of this paper is to present the solution of this problem using the Dual Adaptive Control (Bicriterial Approach). The main idea of this appro...

In this paper we study the stochastic stability properties of certain networked control system. Specifically we study the stability of the Model-Based Network Control System introduced in [9] under time-varying communication. The Model-Based Network Control System uses knowledge of the plant to reduce the number of packet exchanges and thus the network traffic. Stability conditions for constant...