Principled Hybrid Systems: 
Theory and Applications

Systematic principles for integrating symbolic and subsymbolic processing will be developed in the project. Key aims are to ensure that the resulting total hybrid system retains desirable properties of both processing levels. On the one side the signal processing abilities, robustness and learning capability of neural networks should be preserved. On the other side advantage should be taken of the ability of rule-based systems to exploit high level knowledge and existing algorithms and to explain (to a user) why conclusions were reached in particular cases. The methodologies to be developed in the project will be tested in a challenging application related to human computer interaction, which is recognition of emotion based on both voice and visual cues. Low level features will be extracted from signals using neural networks and subsequent formulation of rules will provide a conceptual framework, substantial for emotion analysis.
See DEMO of the facial features extraction procedure.


PHYSTA Participants

Image, Video and Multimedia Systems Laboratory (NTUA)
King's College (KCL)
Katholieke Universiteit Nijmegen (KUN)
University of Milan (UM)
Queen's University of Belfast (QUB)


Review of Artificial Intelligence and Neural Network Techniques for Mapping Features to Symbols (summary or complete report)
Review of Existing Techniques for Human Emotion Understanding and Applications in Human-Computer Interaction (summary or complete report)
Development of Feature Representations from Emotionally coded Facial Signals and Speech (summary or complete report)
Test Material Format and Availability (summary or complete report)
Confidence estimation and on line retraining of neural networks (complete, .ps)
Neural networks for mapping features to symbols (complete, .ps)
A theory for combining subsymbolic and symbolic representations (summary or complete report: zipped .doc)
Generation of subsymbolic representations in the emotion understanding problem (summary or complete report: zipped .doc)

Conference papers


From Synapses To Rules: The Self-Referential Perspective (Abstract or complete .pdf file) B. Apolloni (Italy), G. Biella (Italy), A. Stafylopatis (Greece)
What a neural net needs to know about emotion words (Abstract or complete .pdf file) R. Cowie, E. Douglas-Cowie, B. Apolloni, J. Taylor, A. Romano, W Fellenz (UK)
Bridging the gap between subsymbolic and symbolic techniques: A pragmatic approach (Abstract or complete .pdf file) G. Stamou, D.Vogiatzis (Greece) and S. Strove (The Netherlands)
Comparing Template-based, Feature-based and Supervised Classification of Facial Expressions from Static Images (Abstract or complete .pdf file) W. Fellenz (United Kingdom), J.G. Taylor (United Kingdom), N. Tsapatsoulis (Greece), S.Kollias (Greece)
Towards a Neural-Based Theory of Emotional Dispositions (Abstract or complete .pdf file) J.G. Taylor, W.A. Fellenz, R. Cowie, E. Douglas-Cowie (UK)
Hydrid Hierarchical Steps and Learning by Abstraction for an Emotion Recognition System (Abstract or complete .pdf file) B.Apolloni, C.Orovas, G.Palmas (Italy)
Simulation and bootstrap on a pRAM architecture (Abstract or complete .pdf file) B. Apolloni (Italy), A. Brenna (Italy), D. de Falco (Italy), J. G. Taylor (UK)
Emotion Recognition Using Feature Extraction and 3-D Models (Abstract or complete .pdf file) K. Karpouzis, G. Votsis, G. Moschovitis, S. Kollias (Greece)
Refining Rules of Emotion Recognition in Hybrid Systems.(.doc) (Abstract or complete .pdf file coming up soon:) Piat, F. and Stamou, G. (Greece)

ICME 2000

Exploring the time course of facial expressions with a fuzzy system. (.html) Piat, F. and Tsapatsoulis, N. (Greece)

BPSC 2000

FEELTRACE: Validating a tool for continuous measurement of perceived emotional content. (Abstract or .doc) S. Savvidou, R. Cowie and E. Douglas-Cowie (UK)


A Fuzzy System for Emotion Classification based on the MPEG-4 Facial Definition Parameter. (Abstract or .doc) N. Tsapatsoulis, K. Karpouzis, G. Stamou, F. Piat and S. Kollias (Greece)


The PHYSTA project (Abstract or .doc) Kollias S. and Piat, F. (Greece)

Journal papers

Project Coordinator
Prof. Stefanos Kollias
Institute of Communications and Computer Systems
National Technical University of Athens
Electrical and Computer Engineering
9 Heroon Polytechniou Str.
Zographou 157 73 Athens Greece