Self-Organizing Feature Extraction Using Associative Memory

Abstract

This paper proposes a new self-organizing system for feature extraction. Human-beings can easily and unconsciously extract the features to distinguish objects in ambiguous and noisy environment. The extracted features are self-organized through learning, and the neurons which represent similar features are said to be assembled in a columnar network in a human brain. To realize these characteristics, the proposed system consists of three hierarchical neural networks. They are an associative memory layer, a middle layer and a symbol layer. The units in each layer connect to the units in other layers. The associative memory finds out the features of the input pattern. In this paper, the feature is defined as a combination of parts from which the system recalls the memorized patterns. The units in the middle layer represent the roughly extracted features. In the middle layer, a new method of weight duplication is proposed when a new unit is required. The duplication is to copy all the connections of the most frequently activated unit to those of the new unit. The weight duplication is effective to avoid the dead unit problem and to extract variable features of the input patterns without degradation of the system performance. The weights between the symbol layer and the associative memories express the detailed features, which are useful to distinguish the similar input patterns. The rough features and complementalry detailed features are extracted simultaneously to distinguish the input patterns without designers' prior knowledge. Simulations are carried out to show the feasibility of the proposed system.