Combining multiple modes of information using unsupervised neural classifiers

Abstract

A modular neural network-based system is presented where the component networks learn together to classify a set of complex input patterns. Each pattern comprises two vectors: a primary vector and a collateral vector. Examples of such patterns include annotated images and magnitudes with articulated numerical labels. Our modular system is trained using an unsupervised learning algorithm. One component learns to classify the patterns using the primary vectors and another classifies the same patterns using the collateral vectors. The third combiner network correlates the primary with the collateral. The primary and collateral vectors are mapped on a Kohonen self-organising feature map (SOM), with the combiner based on a variant of Hebbian networks. The classification results appear encouraging in our attempts to classify a set of scene-of-crime images and in our attempts to investigate how pre-school infants relate magnitude to articulated numerical quantities. Certain features of SOM's, namely the topological neighbourhoods of specific nodes, allow for one to many mappings between the primary and collateral maps, hence establishing a broader association between the two vectors when compared with the association due to synchrony in a conventional Hebbian association. © Springer-Verlag Berlin Heidelberg 2003.