Distilling Complex Knowledge Into Explainable T-S Fuzzy Systems

Abstract

This article introduces a novel method for distilling knowledge from complex models using fuzzy systems. The complex knowledge comes from a proposed hybrid NFN-LSTM model (teacher) composed of a long shor-term memory (LSTM) coupled to a neo-fuzzy neuron (NFN) structure. The proposed student model, the NFN-MOD, is an explainable Takagi–Sugeno fuzzy model that resembles modular characteristics to mimic the temporal memory of the LSTM part in the teacher model. The NFN-MOD is adaptable across many scenarios, including solo learning (without a teacher), with the estimation of a previously trained teacher, or training in parallel with the teacher. The complexity reduction of the student model is achieved through the pruning of its consequent parameters with the lowest L1-norm. Application of NFN-MOD in industrial case studies (sulfur recovery unit and cement manufacturing process) demonstrates the efficiency of NFN-MOD in distilling complex knowledge from the teacher model NFN-LSTM, with emphasis on parallel training and parameter pruning. In addition, a novel explainability analysis is introduced, which evaluates the influence of antecedent parameters of the student model in relation to the expected real system output.