Nonlinear Sequential State Estimation for Solving Pattern-Classification Problems

Sequential state estimation has established itself as one of the essential elements of signal processing and control theory. Typically, we think of its use being confined to dynamic systems, where we are given a set of observables and the requirement is to estimate the hidden state of the system on which the observables are dependant. However, when the issue of interest is that of pattern-classification (recognition), we usually do not think of sequential estimation as a relevant tool for solving such problems. Perhaps, this oversight may be attributed to the fact that pattern-classification machines are usually viewed as static rather than dynamic systems. In this chapter, we take a different view: Specifically, we look to nonlinear sequential state estimation as a tool for solving pattern-classification problems, where the problem is solved through an iterative supervised learning process. In so doing, we demonstrate that this approach to solving pattern-classification problems offers several computational advantages compared to traditional methods, particularly when the problem of interest is a difficult one. The chapter is structured as follows: Section II briefly discusses the back-propagation (BP) algorithm and support-vector machine (SVM) learning as two commonly used procedures for pastern classification; these two approaches are used later in this chapter as a framework for experimental comparison with the sequential state-estimation approach for pattern-classification. Section III describes how the idea of nonlinear sequential state estimation can be used as supervised training of a multilayer perceptron. With the material of Section III at hand, the stage is set for specializing the well-known extended Kalman filter (EKF) for the supervised training of a multilayer perceptron in Section IV. Section V describes a difficult pattern-classification task, which is used as a