The fact that local reverberations are almost a logical necessity does not remove the need to test their existence empirically. This has been done in the last few years in a particularly convincing way by Miyashita et al., in a culmination of a program started some twenty years ago by Fuster and Niki. In these experiments monkeys are trained to perform delayed image matching (delayed match to sample DMS) of visual images. The images are supposed meaningless for the monkey and mutually uncorrelated in their geometrical structure. For this purpose the images are generated by a computer using a graphic procedure with several stochastic components. Typical images are given in Fig. miya.image. Following training, testing proceeds according to the following protocol: 1. an image appears on the screen for a short period (200$ms$); 2. The screen remains blank for a prolonged period (as long as 16 seconds); 3. a second image appears briefly; 4. the monkey should react selectively upon whether the first and second image have been the same or different.

Training is performed, prior to the insertion of recording electrodes, by presenting the monkeys long sequences of pairs of images as described above, and rewarding them for correct responses. In different experiments the sequence of first images is varied. For example, the first images can be drawn at random from a store of generated images; or, it can be a fixed sequence shown in a fixed order; the set of images can be divided into pairs, the members of each pair follow each other in a fixed order as first stimulus during training, the pairs are selected at random. The second stimulus is always randomly selected with about 50% chance of being equal to the first. Note that the existence and the structure of the attractor deps only on the first stimulus. The second stimulus, the one used for matching serves to maintain the attention to the task. The ordering of first stimuli applies exclusively to the training phase. During testing, first stimuli are drawn at random.

FIGURES AVAILABLE ONLY IN HARD COPY

Figure 1. Several visual images used in the experiment.

By impressive use of circumstantial evidence Miyashita et al. succeed in identifying a small part (about 1mm$^2$) of anterior ventral temporal (AVT) cortex where stimulus selective persistent activity is manifested during the delay period, i.e. in the interval between the presentations of the first and second images, in which the stimulus is absent. The fact that the selective activity distribution can persist for as long as 16 seconds, in a rather noisy environment, is convincing evidence for the local maintenance of a reverberation by the feed-back in the synaptic structure.

FIGURES AVAILABLE ONLY IN HARD COPY

Figure 2. Reverberation dynamics. Four types of neuronal behavior observed in single units. At the top of each window 12 spike rasters demonstrate the reproducibility of the delay activity on the single unit level, despite intervening presentations of other stimuli. Each row of dots is the representation of spike times recorded from a given neuron in a single trial, with the same image for first stimulus. The similar density of spikes, in the delay period (the wide central interval ) in all 12 rasters, despite the fact that other images intervened as first stimulus between them, is the reproducibility underpinning the representation concept. Bottom: spike rate histograms. a) Neuron active in presence of stimulus and persisting in its absence; b) Same neuron unaffected by stimulus, active in delay period (a different stimulus leading to the same attractor); c) Same neuron, third stimulus, neuron active in presence of stimulus, weakly active in reverberation, a different attractor; d) Same neuron inhibited in presence of a fourth stimulus and weakly active in delay period. Same attractor as in (c). Under each window is the time course of the trial's protocol: pre-stimulus; warning; first stimulus; delay; second stimulus.

The main findings of these experiments, as seen by a theorist, are:

1. Self-sustained, long lived, stimulus selective spike rate distributions (reverberations) exist in the cortex;
2. The locus of the reverberations is locally modular, i.e. within a given cortical region they appear to concentrate in one restricted part. Namely, the module could be considered as a spatially cohesive group of neurons, in contrast to a sparsely distributed collection. This fact concords well with the anatomical observations , which indicate that the probability of synaptic connectivity between neurons in cortex falls off with separation. But within a range of 1mm that probability is still of the order of several percent. Thus, despite the fact that single synaptic contacts in cortex are typically weak (on the order of 100 pre-synaptic inputs are required to elicit a spike), this level of connectivity allows for the maintenance of robust attractors (Some studies find a small fraction of strong synapses. Those are ascribed to multiple contacts between certain pairs of neurons. If that fraction is significant, it may lead to some interesting internal dynamics inside the attractor.;)
3. The reverberations are reproducible on the single unit level, i.e. the rates recorded on the same electrode from the same cell dep only on the stimulus leading to the reverberation. In other words, during testing the first stimuli of the trials are presented in random order, hence between two presentations of the same image as first stimulus many other images intervene as first stimuli. Yet the delay activity is indepent of this history. The reverberations can be considered as internal representations of the stimulus, in the sense discussed in the section above;
4. The reverberations (representations) are not single neuron properties. Single neurons cannot maintain selective activity rates. They are most likely collective emergent properties of the local module (the assembly). (See also discussion of apparent paradox concerning single units, below). In fact, these experiments demonstrate explicitly that neurons driven by the stimulus may not be active during the reverberation and vice versa: neurons unaffected during the presentation of the stimulus may be driven in the delay period.
5. The internal representations are distributed: many neurons participate in the representation of each memory and different representations share neurons;
6. The local internal representations are attractors (as was mentioned in item the above), a whole class of similar stimuli leads to the same reverberation;
7. The representations in this particular module of cortex are of prototypes, e.g. they are blind to color, size, angular orientation. This seems rather typical to this part of cortex, see [tanaka];
8. The representations code in their spike rate distributions, among other things, also for temporal correlations in the training phase. In other words, what is represented deps not only on what is learned but also on how it is learned. This is an embryo of context sensitivity on the neuro-physiological level (see discussion of the implications for cognitive psychology Section {emp.pred}). To be more specific, one may suppose, as is the case in the theoretical model (see e.g. [abt]), that each of the correlated attractors contains information (expressed in neural activities) about the image activating it as well as about the images activating the attractors correlated with it. Hence, each attractor `knows' about its neighbors in the temporal sequence;
9. In a different learning protocol, in which the sequence of first stimuli is presented as a set of permanently ordered pairs, the internal representation becomes identical for both members of each pair. So does the behavior of the monkey in the behavioral paradigm. This does not imply that the monkey cannot detect the difference between the two pictures, but only that in the module used for generating the response for this specific task the two representations merge.

FIGURES AVAILABLE ONLY IN HARD COPY

Figure 3. Correlated reverberations. Correlation coefficients (Kal rank coefficients) of spike activities in a neural population in the delay period as a function of the positional separation of the stimuli exciting the reverberations in the training sequence. (From ref. [miya2] Fig. 3c.) Full circles represent correlations of delay activity distribution for learned images (used in training). Empty circles refer to activity distributions provoked by `new' images (not used in training). The different curves represent different samplings of neurons in the module selected for the computation of the correlations. The stars are irrelevant to the present discussion.

The above presentation of the empirical situation requires several notes of clarification. On the one hand, we have emphasized the distributive and collective nature of the internal local representations and on the other, the reproducibility of recordings of single units. This apparent contradiction disappears if one keeps in mind that the enhanced, persistent activity of any individual neuron can only be sustained due to the support of its fellow neurons participating in the same reverberation, via the synaptic local feed-back. The activity of the single neuron, therefore, carries information about the stimulus only if it is accidentally selective between the different distributed reverberations. Among the active neurons there may be several that have the same activity in different representations. On the other hand, the fact that the reverberations are dominated in their detail by the synaptic structure, and not by fine details of the stimulus, implies that every time the same stimulus is presented, the same reverberation is aroused. Consequently, the internal representations, for whatever they represent, may be perused and catalogued recording one neuron at a time, as is in fact done in the experiments of Miyashita et al.

This precludes, of course, observing phenomena related to correlations in spike emission times if those manifest themselves in different neurons. To observe those, multi-unit probes are required. Yet, I feel that the information, representational, computational and cognitive, contained in rate distributions is far from exhausted. On the other hand, the relative facility of access to this type of information as well as of its analysis and modeling, compared to multi-electrode data, makes it a very attractive subject of investigation.

6. Empirical and cognitive implications

A significant aspect of the generation of random images in the Miyashita experiments is that it is quite likely that the correlations between internal representations of stimuli are due only to their context depence, i.e. their frequent contiguous appearance in the training phase. The effort invested in the generation of the images is recompensed by the fact that in some sense the registered correlations between the attractors are the minimal correlations among internal representations: those due purely to the constraints imposed on the learning process by the sequential training and the existence of attractors.

The attractor picture and the observed correlations it creates among internal representations have a rather universal feature. The dynamical tagging of a given memory may produce persistent activities in several modules in the cortex. The representation of one stimulus in different modules may represent different features of the stimulus, such as color, shape etc... Different modules may be involved in the generation of different types of reactions. One gets the impression, from the experiments of the Miyashita group, that the observed module in AVT is directly related to the pair association [miya3], while it is not as directly related to the matching task. The universality inted here is that this does not matter, in the sense that wherever the attractor related to the reaction is, it must represent similar correlations of the internal representations, because those dep only on the fact that one is learning one attractor while reverberating in a previous one.

This observation suggests that one may use the lessons of these experiments to speculate directly about human cognitive psychology. The the fact that correlations form between the attractors representing semantically meaningless, uncorrelated stimuli implies that priming phenomena should be observed among stimuli of this kind. (Priming is the experimental observation that the time for recognition of an incomplete pattern is shortened if the presentation of the stimulus to be recognized is preceded by the presentation of a cognitvely related stimulus.) The only condition is that they be presented in an ordered sequence during training, indepently of whether we can observe the attractor cortical network involved in the cognitive task or not. That priming should take place in a situation of correlated attractors can be concluded intuitively, and is confirmed in model networks. It is the mere observation that if the assembly is in a given reverberation, due to the priming stimulus, the test stimulus to be recognized will find it easier (and hence faster) to provoke a transition to another reverberation attractor, the more similar the activity distribution in the latter attractor is to the priming attractor.

Thus, purely on the phenomenological, pre-theoretical level, given the observation of correlated attractor representations, one is lead to consider for example:

1. Testing priming effects in humans with semantically meaningless stimuli, essentially imitating the Miyashita experiments, but measuring reaction time changes upon priming. One would expect a decrease in the priming effect with the distance in the training sequence, i.e. with decrease of the correlation between the internal representations.
2. Investigating the assumption that false alarms are also caused by attractor correlations. (False alarms is an experiment in which a subject is required to identify whether a given stimulus belongs to a subset of stimuli or not. The effect is that when the test stimulus is correlated (cognitively) with one of the items in the subset, the number of wrong `yes' answers increases.) This hypothesis can be tested most clearly upon sequentially memorized semantic-free stimuli.
3. Testing the effects of the intensity of temporal correlations on the internal attractor correlations. In other words, one can prepare a set of stimulus sequences with increasing proportion of sequences of fixed order, the others being random. One should expect a threshold behavior in the priming effect as a function of this proportion.

Similarly, the attractor interpretation of the experiments suggests informative extensions of the experiments in primate `cognitive-neuro-physiology':

1. An immediate consequence of the framework proposed is that simple attractors (internal representations) must be formed prior to the correlated ones, as an intermediate stage in forming correlated attractors. Hence, one should find a point in the training process at which internal representations exist, but express no correlations;
2. A related experiment to the one above is to train at length with patterns presented in a random sequence. Internal representations should form, but they should be uncorrelated. Their correlations with the activity distributions driven by the stimuli themselves, would be invaluble to the solidification of the modeling effort;
3. One can perform on monkeys the priming experiments, suggested for humans above, while observing attractor transitions in the cortical module under investigation;
4. One can ext the experiments on pair association [miya3], which have been interpreted as attractor fusing. This can be done by measuring the correlations between pair representations in situations of partial ordering of the pairs in the training phase. Specifically, in the experiment first stimulus images belonging to a pair were always shown, during training, contiguous and in the same order, while pairs were selected at random. One can conceive of keeping the strict ordering within each pair and having each pair followed by a given pair with some probability p and choosing the subsequent pair at random with probability 1-p;
5. It is of importance to establish the structure of the distribution of activity in the module following the response;
6. But most important would be the identification of the additional representation modules in the cortex. These would be in different parts of the cortex, or even outside it, as different dimensions of a stimulus seem to be stored at different stations along the cortical elaboration path. See e.g. Damasio and Damasio.

7. Toy models and realistic modeling

Modern physics has had it as a very powerful methodology that very structured phenomena in a complicated system are investigated in toy models. In other words, phenomena like super-conductivity, magnetism, liquid crystals etc., are not searched for by starting from the well established dynamical laws of systems of nuclei and electrons. Instead, some essential features of the elements, deemed relevant to the structured, emergent phenomenon under investigation, are represented in a simple tractable model. If the dynamics of the toy model actually produces the expected structure, the robustness of the phenomenon to the reintroduction of the omitted complexity of the underlying elements is investigated. This iteration serves both to justify the toy as well as to study further details of the emerging structures.

The discussion in the sections above has left us with a double task:

1. The demonstration of a theoretical framework in which a plurality of stimulus selective persistent activities can be embedded in a single assembly of neuron-like elements;
2. The demonstration that in such a framework correlations (context sensitivity) emerge between reverberations (internal representations) corresponding to uncorrelated learned stimuli.

The first task has been solved by the Hopfield model which has served as the basic toy model in describing the emergence of a diversity of structured attractors, robust to many types of random damage and noise. This was done by employing an explicit form for a synaptic matrix, that has a learning flavor, in the sense that the set of synaptic efficacies is constructed, in an additive fashion, from the correlations of activities of neural pairs in afferent patterns that are to be the attractors of the network. This is in the grain of the Hebbian paradigm. Namely, external stimuli to be learned impose activity distributions on an assembly, which in the learning process develops a set of synaptic efficacies that can maintain, autonomously such activity distributions as reverberations. It is after all the synaptic matrix, and only the synaptic matrix that can maintain the delay activity in the absence of the stimulus. The possibility of generating a synaptic matrix which ows a network with a large variety of different robust attractors in a single module, was the main achievement of this model. Moreover, the formulation of the model allowed for the detailed computation of many properties of the dynamical response of the assembly to external afferent stimuli.

Furthermore, this toy model has also served to generate metaphors for several psychiatric and neurological pathologies. Hoffmann has used it to describe a distinction between mania and schizophrenia. Virasoro has used the properties of this model under a random destruction of synapses (a lesion) to capture phenomena such as prosopagnosia.

Yet this model has left unanswered a whole set of questions of detail. A representative list includes:

1. The model predicts high spike rates in attractors, while recordings produce rates much below saturation. In fact, the attractors in the model are activity distributions in which about half the neurons in the assembly are quiescent while the other half emits spikes at saturation rates. This allowed modeling in terms of binary discrete variables for the neurons;
2. The recorded coding levels (The proportion of neurons in the assembly with elevated spike rates) in a given reverberation, are much lower than the 50% implied by the toy model;
3. The model produced auto-associative networks, i.e. with attractors as close as possible to the memorized stimuli, while experiment did not;
4. It predicted a bi-modal distribution of rates in an attractor, unlike the empirical observation.

Early criticisms of the Hopfield model have concentrated on the type of synaptic matrices used. Symmetric, fully connected matrices, with random distribution of excitatory and inhibitory synapses on the axons of each neuron, and infinite analog depth (The ability to maintain with high precision a large number of different, closely spaced values.) for each synapse, have made the analysis by theoretical physicists easier. Those are unrealistic impositions, since it is unlikely that the cortex will generate symmetric synaptic structures; cortex is connected at most at a level of 10% ; excitation and inhibition find their places on different neurons -- Dale's rule. Yet these criticisms have been shown to be relatively innocuous. The synaptic dilution and the limited analog depth of the synapses have been treated by Sompolinsky and shown to affect the performance of the network only mildly. If fact, in some cases the performance per remaining material resource (such as storage capacity per surviving synapse) was found even to improve in the less ideal system.

Also the question of the low coding levels has been found to be relatively simple. Though its resolution has brought to light the fact that if one looked for a network with uniform thresholds for the neurons, the behavior of the network is strongly depent on the choice of the representation for the neural states. If one insists on representing neurons by two state variables, there is a clear advantage, in representing these states by (0,1) over (--1,1).

A more elaborate modification of the dynamics of the original toy model was required in order to account for the relatively low spike rates in the attractors observed in experiment. It required a more detailed treatment of the single neuron dynamics, arriving at a description of neurons in terms of coupled systems of afferent currents and efferent spike rates. This description has produced networks with attractors operating far below the saturation of the neurons composing the network. The modified networks, with low (arbitrary) coding levels and low spike rates preserved the main features of the original toy model: robust diversity of attractors, classifying stimuli auto-associatively. Two outstanding issues remained: auto-associativity and bi-modality.

The difficult problem of modifying a network to form attractors with correlations of the Miyashita type from uncorrelated stimuli learned in a fixed order, found a solution with a flavor of the Miyashita training scenario, at the level of the toy model. Indicating once again the usefulness of such models as drawing boards for new ideas. Auto-associative ANN's are based on the idea that the synaptic matrix codes for the correlations of activities of pairs of neurons as induced by a given afferent stimulus. The neural pair correlations in different stimuli are coded indepently of each other. It was shown that when synaptic modifications, induced by training on a sequence of stimuli presented in a fixed order, record { also} the correlations of the activities of pairs of neurons induced by one stimulus with that of its immediate predecessor in the sequence, then the resulting attractors display correlations of the Miyashita type. The resulting attractors, each classified by the uncorrelated stimulus that had been learned and that excites it, are correlated for as far as five apart in the training sequence. This theoretic result has manifested itself in a dramatic way, in that each persistent delay activity (attractor) has a finite similarity index with exactly five of the nearest stimuli in the sequence. It is just the approximate range of significant correlations observed in the experiment. But this surprising { five} was deduced in the rather artificial context of the toy model.

Note that two types of correlations enter this discussion, and they should not be confused: One is the correlations of activities of pairs of neurons, during the presentation of stimuli for learning. They drive the Hebbian learning. Then, when learning had generated the synaptic matrix, the network's dynamics is controlled by that matrix. In particular, this synaptic matrix determines the structure of the attractors (delay activity distributions). The correlations between these attractors are the second type. They are the ones measured by Miyashita.

The promise of the result was in that

1. Synaptic information about single neighbor contiguity in the training sequence, i.e. the inclusion in the synaptic efficacies (learning) of activity correlations of pairs of neurons, one active as driven by a given stimulus and the other by the preceding one, was sufficient to induce correlations of the corresponding internal representations to a distance compatible with experiment;
2. The attractor picture underlying the model makes the information about a preceding stimulus in a training sequence naturally available in the persistent attractor, at the time of presentation of the current stimulus. Recall that the network is supposed to code for activity correlations in successive { first} stimuli. Any two such stimuli are presented with a separation of many seconds. But, the existence of attractors allows the information of one, first, stimulus to be around for as long as is needed for the subsequent stimulus to be presented.;
3. The form of the correlations (magnitude of coefficients and its rate of decay as function of separation in the training sequence) was found to be indepent (in a certain range) of the value of the contiguity amplitude parameter, which is the relative strength of the contribution to the synaptic efficacy due to neighboring images in the training sequence to the contribution due to each image separately.

[FIGURES AVAILABLE ONLY IN HARD COPY]

Figure 4. Confrontation with theoretical model (4000 neurons): Kal rank coefficients of attractors as a function of the separation of the stimuli in the training sequence. To each stimulus corresponds an attractor (reverberation), the one excited by it. The attractors are labeled by the serial position number (SPN) of the corresponding stimuli in the training sequence. The learned stimuli forming the synaptic matrix are uncorrelated, since the images presented for learning were uncorrelated. The activity distributions in the attractors are. The error bars are standard errors in the sample of neurons. a. Correlations for regular sample second curve from top, b. Correlations in enhanced sample, top curve. Full curve, model results; dotted curve, experiment.

In fact, the phenomenon persists, when the formal neurons are replaced by quasi-realistic, integrate-and-fire, neurons. An ANN operating with a synaptic matrix containing information on temporal contiguity in the training process, preserves the main features of the attractor correlations in the systems of discrete neurons. The correlation coefficients (Kal rank coefficients, as used in the experiments) of the network of realistic neurons , which also includes reactive, separated inhibition, agree quantitatively quite well with the measured values. Moreover, the more realistic model presents some additional features which brings the model even closer to the biological experience:

The attractors expressing the Miyashita correlations do not exhibit a simple bi-modal distribution of spike activity among the neurons in the assembly. All previous ANN models produced sharp bi-modality, either because the neurons were discrete, i.e. quiet or at saturation frequencies, or because the models implemented auto-associativity. Experiments manifest attractors, but not simple bi-modality. The model predicts a large, stimulus selective, peak at very low spike frequencies, and a wide distribution of rates among the active neurons. Consequently, the combination of realistic neurons and attractor correlations (i.e. the departure from auto-associativity) gives a potential response to the problem of the nature of the rate distribution.

To conclude this discussion we show one more measurement carried out both on the model and on the performing monkey. We show the distribution of activities produced in a given neuron in the reverberations provoked by the presentation of the complete set of learned stimuli. One sees the rate of spike emission by this neuron, in the delay period, for each of the stimuli plotted in the order in which they had been learned.

This is not the right context in which to develop a more detailed discussion of the confrontation of theory with experiment. The discussion has been opened here only to indicate that the insights gained by the interpretation of the Miyashita experiments in the language of attractor dynamics is accompanied by a candidate model which captures the experimental results to a very impressive degree of detail. Such a model can consequently serve as a drawing board for the development of future paradigms in cognitive psychology.

[FIGURES AVAILABLE ONLY IN HARD COPY]

Figure 5. Average delay discharge rate vs serial position separation on a given cell. (a) Fig. 3a (b) Model. This displays the level of activation of the particular neuron in the reverberation stimulated by each of the hundred stimuli in the learned sequence. The existence of two peaks indicates that this neuron participates in the representation of two uncorrelated stimuli. The side wings of each peak are due to the correlations of the attractors, developed in learning a fixed sequence.

8. Predictive theories

The attractor description as a language and as a set of models is a strongly predictive framework. It produces several detailed experimental predictions even prior to the elaboration of detailed models. The models enlarge the predictive set of commitments of the approach. The detailed studies lead to the following predictions:

1. The appearance of the uncorrelated reverberations, either due to short training periods, or to training with random sequences of stimuli, should be accompanied by narrower distributions of spike rates.
2. The relation between the coding rate of the stimuli, the fraction of the module's neurons driven into high rates when the stimulus is present, to the coding rate in the corresponding attractor;
3. Statistics of the distribution of spike rates expressed on a given neuron by the entire set of stimuli memorized in the assembly (ibid);
4. If pair associations are formed by the training process, then when the pair attractors become correlated, by partially ordering the pairs the correlations of the pair attractors will be very high and will ext far down the sequence.

   
   

   The list can be continued.

9. Some provisos and defensive outlook

The above picture sounds too good. It may still have to undergo modifications. The main exposed flanks we perceive at this stage are:

1. It may be the case that the Miyashita correlated reverberations are not autonomous, in the sense that they are not really coded in the observed area, but reflect attractor activity in one or several other areas. Such modules could then drive afferently the module observed. This is a question which should be given serious attention;
2. The specific predictions follow from models with a prescribed synaptic matrix. This may be too restrictive. The models represent in a plausible way the learning process hypothesized. What gives it some credence is the relative robustness of the results to variations in the structure of the synaptic matrix, provided the logic of learning a sequence of stimuli with fixed order is maintained. In particular, there is the robustness under variation of the contiguity amplitude parameter.
3. The neural elements in the model networks may still be somewhat too idealized. This may be at the origin of some systematic differences in the details of the attractor correlation coefficients and rate distributions.

Even if some modifications are required, the reverberation picture is too fertile to be rapidly discarded. We have received it from Hebb aged some fifty years. It has only recently been given a direct empirical (neuro-physiological) dimension and has been owed with a precise mathematical model. One is tempted to start drawing a host of speculative conclusions from the new framework exposed by Miyashita's monkeys. The context provides fertile ground for adventures in cognitive psychology and even for some aspects of linguistics ranging from binding, which may be related to syntax, to priming, which may be extrapolated to semantics. It may even suggest a substrate for psychology itself.

Yet the lessons learned from these experiments include the one which advises restraint. It is just these experiments which indicate that our imagination concerning brain computation is still too much constrained by formal mathematics, by computer languages and by artificial intelligence. In this connection it is well worth recalling the wisdom of John Von Neumann, writing 40 years ago: {quote}

...Thus the outward forms of our mathematics are not absolutely relevant from the point of view of evaluating what the mathematical or logical language truly used by the central nervous system is... the above remarks about reliability and logical and mathematical depth prove that whatever the system is, it cannot fail to differ considerably from what we consciously and explicitly consider as mathematics. [The Computer and the Brain, Yale 1954 p. 82, emphases in the original]

It is most likely that atting for a while longer to the details of the contact between modeling and experiment would keep options open which a premature harvest of speculation would close.

Acknowledgments

I am indebted to Prof. Peter Hillman for a critical reading of an earlier version of this manuscript and to an anonymous referee who has helped me improve the paper very significantly.

References

Miyashita Y and Chang HS 1988 Neuronal correlate of pictorial short-term memory in the primate temporal cortex, Nature, { 331} 68

Miyashita Y 1988 Neuronal correlate of visual associative long-term memory in the primate temporal cortex, {\em Nature}, { 335} 817

Sakai K and Miyashita Y 1991 Neural organization for the long-term memory of paired associates, {\em Nature}, { 354} 152.

Amit DJ 1993 In defense of single electrode recordings, {\em NETWORK}, { 3} 385

McNaughton BL, Barnes CA and Andersen P 1981 Synaptic efficacy and EPSP summation in granule cells of rat fascia dentata in vitro, {\em J Neurophysiol} { 46} 952

Sayer RJ, Redman SJ, Andersen P 1989 Amplitude fluctuations in small EPSPs recorded from CA1 pyramidal cells in the guinea pig hippocampal slice, {\em J Neurosci} { 9} 840

Mason A, Nicoll A and Stratford K 1991 Synaptic transmission between individual pyramidal neurons of the rat visual cortex in vitro, {\em J Neurosci} { 11} 72

Tanaka K, 1992 Inferotemporal cortex and higher visual function, {\em Current Biology} {\em 2} 502

O'Keefe J and Speakman A Single unit activity in the rat hippocampus during a spatial memory task, {\em Exp. Brain Res.} { 68} 1

Amit DJ Brunel N and Tsodyks MV 1993 Correlations of Hebbian reverberations, {\em Jour. of Neurosci.} in press.

Hebb DO 1949 {\em The Organization of Behavior} (Wiley, NY)

Hebb DO and Donderi DC 1987 {\em Textbook of Psychology} fourth edition, (Laurence Erlbaum Ass, Hisdale NJ)

{deNo}Lorente de {\'No} 1949 in Fulton JF editor, {\em Physiology of the Nervous System} (Oxford University press, NY)

Fuster JM 1973 Behavioral electrophysiology of the prefrontal cortex, {\em J. Neuropysiol.}, { 36} 61

Niki H 1974 Prefrontal unit activity during delay alternation in the Monkey, {\em Brain Res.} { 68 } 185.

Braitenberg V and Schutz A 1991 {\em Anatomy of Cortex} (Berlin: Springer-Verlag)

Anisfeld M and Knapp M 1968 Association, synonymity, and directionality in false recognition. {\em Journal of Experimental Psychology} { 77}, 171

Damasio AR and Damasio H 1991 Cortical systems underlying knowledge retrieval: evidence from human lesion studies, Background manuscript for the Dahlem Conference on Exploring Brain Function: Models in Neuroscience, Berlin

Hopfield JJ 1982 Neural networks and physical systems with emergent selective computational abilities, {\em Proc. Natl. Acad. Sci. USA} { 79}, 2554

Amit DJ 1989 {\em Modeling Brain Function} (Cambridge University Press, NY)

Hoffman RE 1987 Computer simulations of neural information processing and the schizophrenia-mania dichotomy, {\em Archives of General Psychiatry} { 44} 178

Abeles M, Vaadia E and Bergman H 1990 Firing patterns of single units in the prefrontal cortex and neural-network models, {\em NETWORK}, { 1} 13

Virasoro AM 1988 Categorization in neural networks and prosopagnosia, {\em Physics Reports} { 184} 99

Tsodyks MV and Feigel'man MV 1988, The enhanced storage capacity in neural networks with low activity level, {\em Europhys. Lett.}, { 46} 101

Buhmann J, Divko R and Schulten K 1989, Associative memory with high information content, {\em Phys. Rev.} { A39} 2689

Sompolinsky H 1986 Neural networks with nonlinear synapses and static noise, {\em Phys. Rev.}, { A34}, 2571 and The theory of neural networks: The Hebb rule and beyond, in L. van Hemmen and I. Morgenstern eds. {\em Heidelberg Colloquium on Glassy Dynamics}, (Springer-Verlag, Heidelberg)

Amit DJ and Tsodyks MV 1991 Quantitative study of attractor neural network retrieving at low spike rates I: Substrate -- spikes, rates and neuronal gain {\em NETWORK} { 2} 259

Amit DJ and Tsodyks MV 1991 Quantitative study of attractor neural network retrieving at low spike rates II: Low-rate retrieval in symmetric networks {\em NETWORK} { 2} 275

Griniasty M, Tsodyks MV and Amit DJ 1993 Conversion off temporal correlations between stimuli to Spatial correlations between attractors, {\em Neural Computation} { 5} 1

Cugliandolo L 1994 Correlated attractors from uncorrelated stimuli, {\em Neural Computation} { 6} 220

On leave from Racah Institute of Physics, Hebrew University, Jerusalem