Improving the Robustness of Data-Driven Fuzzy Systems with Regularization

Regularization is an important aspect whenever a matrix inversion during the training phase is required, as this inversion may lead to an unstable (ill-posed) problem, usually simply because of a matrix with a high condition or even a singular matrix, guiding the learning algorithm to wrong solutions. In this paper we present regularization issues applied to off-line and on-line training of Takagi-Sugeno fuzzy systems for increasing the robustness of the learning procedure and the accuracies of the models. After defining the problem of ill-posedness for the learning of linear consequent parameters (when applying least squares optimization measure), we describe several methods for finding an optimal parameter setting in the Tichonov regularization. We also describe the way how to apply regularization to evolving fuzzy models. The paper is concluded with a comparison of conventional (not regularized) FLEXFIS resp. FLEXFIS-Class method with their regularized extensions and with (not-regularized) genfis2. This comparison will be based on high-dimensional real-world data sets from engine test benches and from an image classification framework and will underline the impact of the regularized methods on prediction and classification accuracy.