" Review of Artificial Intelligence and Neural Network Techniques for Mapping Features to Symbols "
Since the early revival of neural networks, the aim of rereading in symbolical terms a function learnt by a neural network has been pursued by many researchers. Indeed, although no one can refuse that subsymbolic attitudes, such as intuition or experience, lie at the basis of practical useful actions and are indispensable ingredients of common sense reasoning, we must at least be able to describe through symbols the core of these actions and their rationale. This delineates an inherent hierarchy where at low levels we locate actions and at higher levels we locate their formal explanation. Intelligent systems, viewed as tools for modeling the brain or as machines, have been accordingly categorized on the basis of the abstraction level of the processing involved: symbolic or subsymbolic. Historically, these two main approaches are represented by Artificial Intelligence (AI) and Neural Networks (NN), respectively. Although each category can specify intelligent systems by itself, the limits are not clear and become increasingly vague. The emergent need for intelligent systems that can work in both frameworks, symbolic and subsymbolic, leads to hybrid approaches. In the review, a taxonomy of hybrid intelligent systems is proposed, which builds upon previous work by other researchers (Medsker, Hilario). According to the taxonomy, which uses a different terminology with respect to previous ones, hybrid intelligent systems are classified into three categories.
In a first part of the review, conventional AI methods for mapping features to symbols are examined, including probabilistic reasoning, decision trees, pure symbolic learning techniques and fuzzy knowledge-based systems. A next part of the review covers conventional NN techniques for solving the problem of feature to symbol mapping. In this context, biological mapping paradigms are considered and various artificial neural network models are discussed, including approaches such as tree neural networks, mixture of experts, probabilistic neural networks and active decision processes, as well as fuzzy neural networks.
The major part of the review is organized following the proposed three-category taxonomy for the classification of systems arising from neurosymbolic integration.
The first category, Combined Intelligent Systems, comprises systems that use neural networks as tools for symbolic processing and NN models that use AI concepts in order to support symbolic processing. The former systems represent a top-down design approach (start with a high-level function and proceed with the design of an appropriate connectionist infrastructure), whereas the latter represent a bottom-up approach (start from neurons and try to model high-level functions). Nevertheless, in many cases the above distinction is not obvious. This category includes Neural Expert Systems, Knowledge-Based Artificial Neural Networks (KBANN), the Knowledgetron model, neural network methods for the classification of strings and structures, and several neural network techniques for fuzzy inference.
The second category, Transformational Intelligent Systems, use NN and AI techniques to transform symbolic representations to subsymbolic and vice versa. The main operations provided are rule insertion, rule extraction and rule refinement. The most important point is that the NN and AI methods used for this operation are stand-alone and based on conventional approaches, in the corresponding framework. Several techniques are reviewed under this category, including RULEX, TREPAN, Validity Interval Analysis, Neurorule, and other rule extraction and explanatory mechanisms.
The third category, Coupled Intelligent Systems, integrate AI and NN modules to produce a system with the ability to work at both symbolic and subsymbolic level, achieving effective functional interaction between conventional neural networks and symbolic processors. Coupled Intelligent Systems can be further distinguished according to the role played by the neural and symbolic components in relation to each other and to the overall system.
The above taxonomy and characterization of algorithms and representations is the basis for the development of a theory and methodology in the PHYSTA Project. According to the PHYSTA perspective, the rule extractor is not an external agent but a piece of neural network by itself. Stated in other words, we look at physiological models in order to develop realistic models for passing from features to symbols. A direct approach tries to "open" a trained neural network in order to recognize symbolic structures, an efficient way only in cases where the information content of connection weights is very low. Otherwise, we can shorten this subsymbolic process through a symbolical high-level training that agrees with the neurophysiological learning process hypothesized to occur in the brain.