Non-differentiable Optimization of Fuzzy Logic Systems

In the present use of fuzzy logic systems tuning of their parameters has become an important issue. Many techniques, mainly based on the application of gradient descent, have been applied to this task generally in order to minimize a quadratic error function. The class of fuzzy logic systems using piecewise linear membership functions (e.g., triangular or trapezoidal) and/or minimum or maximum operators generates an objective function that is not differentiable (i.e., at all points in the search space) with respect to some of its parameters. This in turn makes the techniques applied to this task no longer a gradient descent algorithm, and thus any convergence proof (even if only at a zero gradient point) no longer holds. This paper discusses the problem and shows some of the issues it raises using a very simple example. INTRODUCTION Quite some attention of the early nineties research on fuzzy logic systems (FLSs) was focused on parametric design. Triangular or trapezoidal membership functions, along with minimum t-norm, are common choices in the use of FLSs. One of the first approaches to tuning antecedent and consequent parameters of a fuzzy logic system through gradient descent is the one of Nomura et al. (1992). In this approach symmetric triangular membership functions and constant consequents are used. The authors tune consequent constants and antecedent membership functions parameters (center and width). The non-differentiability of the error function with respect to the triangle center at a finite number of points is briefly noted (it is also non-differentiable with respect to the widths). The problem is faced pragmatically by zeroing the corresponding parameter update whenever non-differentiability is encountered. Many other authors neglected the non-differentiability problem encountered with piecewise linear membership functions and applied “gradient descent” approaches to the supervised learning problem. This paper discusses the application of gradient descent to supervised learning of fuzzy logic systems using piecewise linear membership functions and/or max—min operators. The next section shows how this generates an optimization problem that is not only nonlinear, but also non-differentiable. In the third section a simple function approximation problem will be introduced, and in the fourth section some of the issues involved with this type of problem will be showed through some numerical results. The final section offers some concluding remarks.