A combinatorial neural network exhibiting episodic and semantic memory properties for spatio-temporal patterns

A model is described in which three types of memory--episodic memory, complex sequence memory and semantic memory--coexist within a single distributed associative memory. Episodic memory stores traces of specific events. Its basic properties are: high capacity, single-trial learning, memory trace permanence, and ability to store non-orthogonal patterns. Complex sequence memory is the storage of sequences in which states can recur multiple times: e.g. (A B B A C B A). Semantic memory is general knowledge of the degree of featural overlap between the various objects and events in the world. The model's initial version, TEMECOR-I, exhibits episodic and complex sequence memory properties for both uncorrelated and correlated spatio-temporal patterns. Simulations show that its capacity increases approximately quadratically with the size of the model. An enhanced version of the model, TEMECOR-II, adds semantic memory properties.; The TEMECOR-I model is a two-layer network that uses a sparse, distributed internal representation (IR) scheme in its layer two (L2). Noise and competition allow the IRs of each input state to be chosen in a random fashion. This randomness effects an orthogonalization in the input-to-IR mapping, thereby increasing capacity. Successively activated IRs are linked via Hebbian learning in a matrix of horizontal synapses. Each L2 cell participates in numerous episodic traces. A variable threshold prevents interference between traces during recall.; The random choice of IRs in TEMECOR-I precludes the continuity property of semantic memory: that there be a relationship between the similarity (degree of overlap) of two IRs and the similarity of the corresponding inputs. To create continuity in TEMECOR-II, the choice of the IR is a function of both noise ({dollar}Lambda{dollar}) and signals propagating in the L2 horizontal matrix and input-to-IR map. These signals are deterministic and shaped by prior experience. On each time slice, TEMECOR-II computes an expected input based on the history-dependent influences, then computes the difference between the expected and actual inputs. When the current situation is completely familiar, {dollar}Lambda{dollar} = 0 and the choice of IRs is determined by the history-dependent influences. The resulting IR has large overlap with previously-used IRs. As perceived novelty increases, so does {dollar}Lambda{dollar}, with the result that the overlap between the chosen IR and any previously-used IRs decreases.