THE NEURAL BASIS OF COGNITIVE DEVELOPMENT: A CONSTRUCTIVIST MANIFESTO

Keywords

Abstract

1 Introduction

Although these questions are at the heart of cognitive science, development still resists our attempts to understand it. To develop is to change, and therein lies the challenge. For the structures underlying development are not fixed; they undergo large-scale changes during cognitive skill acquisition. What is more, these changes may not be intrinsically determined; they may depend on interacting with the environment. If so, then the distinction between biological maturation---the brain unfolding according to its intrinsic schedule---and learning breaks down. Descriptions at both levels of explanation, the biological and the cognitive, would then be required in an integrated level of explanation.

Figure 1: The logic of the problem of development. At its most abstract, the problem is defined as one of characterizing the mapping from some set of input data into the target state (the adult competence; see Wexler & Culicover (1980) for such a treatment). This problem is complicated by two elements that are often dismissed from such a characterization: changing resource limitations in terms of working memory and structural/representational change.

Figure 2: Candidate measures of structural/representational complexity. There are three possible (non-exclusive) measures: synaptic numbers, dendritic arborization, and axonal arborization. The figure also summarizes the basics of neural information processing elements: synaptic input travels along the dendrites to the cell body, where it is integrated and an action potential fires down the axon if some threshold is exceeded.

Figure 3: Summary of the relations among major cognitive and neural theories. Instructivism is often identified with Hebb's work, since on his view the environment guides or instructs brain growth. We refer to such theories as constructivist since the latter is a more widely recognized class of theories.

Figure 4: Synaptic density in layer III, human middle frontal gyrus from birth to 8 years. Data from Huttenlocher (1979). Note the absence of data points between 1 and 5 years of age.

Figure 5: The generation of specificity according to the standard model. A, schematic representation of the connections between retinal ganglion cells and the lateral geniculate nucleus of the thalamus and connections to primary visual cortex. B, the two phased process of development and perturbations to that development. An initial projection shows a wide overlap in axonal arbors from both eyes, followed by activity-dependent segregation in normal development or a failure to segregate in the case of blockade of activity. In the case of unequal activity, as in monocular deprivation, the eye with the most activity is stabilized preferentially. (modified from Goodman & Shatz 1993).

Figure 6: The development of horizontal connections in the ferret. A, At postnatal day 22 (P22) little clustering can be seen. As development proceeds, however, axon cluster formation become evident, particularly by P28. B, Subsequent development, seen here under higher magnification, reveals the refinement and elaboration of axon clusters. The development of these horizontal connections supports the hypothesis that axon development involves both selective retraction of inappropriately placed arbors and progressive elaboration of correctly placed arbors. Reproduced by permission from Durack & Katz (1996).

Many of the studies summarized in Table 2 also point to an important activity-dependent component in axonal development. Among these, the Friedlander et al. (1991) study is an important one, as it examined the effects of monocular deprivation on Y-type geniculocortical axons found in cats. According to selectionism, monocular deprivation causes an expansion of columns of the non deprived eye. This expansion is caused by relatively more correlated activity from the non deprived eye, and so its arbors compete favorably for territory that is initially shared by overlapping arbors. The deprived eye columns, in contrast, shrink (see Figure 5). Friedlander et al. (1991), however, found that the deprived arbors did not shrink due to a lack of stabilizing input, but instead failed to grow in the first place. The nondeprived arbors did not simply stabilize over more territory in the absence of competing fibers. Rather, their growth was accelerated and extended.[1]

These studies, then, point to the central role of activity in the progressive growth of these systems. We next examine the third candidate measure, dendritic arborization.

2.3 Dendritic development

Dendritic development has typically been overlooked in theoretical models of development, which focus instead on axonal development. We suggest that dendrites play an important role in determining the representational properties of cortex and that their development supports neural constructivism. First, we will consider some general reasons why dendritic development is important for the representational properties of cortex.

As we noted in section 1, nonstationarity, while perhaps increasing the learning capacities of a developing system, introduces a number of complications. The main reason for this is that changes to the underlying architecture can be unwieldy. Even small architectural changes can have severe consequences for the system's overall performance. One way this may happen is if changes to an architecture undo previous learning by reconfiguring structural elements that represented already acquired knowledge (a phenomenon sometimes referred to as "catastrophic interference"). Such a process could also have negative consequences for learning if it introduced large-scale changes rather than incremental ones. For example, large-scale changes could make learning too sensitive to small details of training (resulting in overfitting) and would undo the dependency on previous states that makes learning incremental, and thereby stable.

These concerns lead to the following two related conditions that a nonstationary mechanism must satisfy:

1. The Locality Condition

The addition of structure must be at the appropriately local scale and must not result in wholesale changes in representation with each new elemental change;

2. The Stability Condition

Under normal circumstances, local changes must not undo previous learning.

2.3.1 Dendrites as learning structures

Dendritic growth satisfies the locality condition in that, unlike the retraction of an axon which might alter the connectivity of thousands of cells, dendritic segments are local processing elements, whose elimination will not cause large-scale changes in the overall pattern of connectivity. Hence, the constructivist learning scheme operates at the level of individual dendritic segments, rather than the whole cell, thereby building in these conditions.

It is also important that dendrites grow much more slowly than do axons. Axon's grow at approximately 500 microns/day compared to 15-35 microns/day for dendrites (see Uylings et al. 1990; Katz et al. 1984). This suggests that the two are sensitive to statistical structure at different timescales and that dendrites are extracting this structure on a much longer temporal scale.

A more important and general reason for examining the growth of dendrites in relation to the construction of representations is that dendrites are the primary receptive surface of a neuron. Moreover, since dendrites do not just conduct passively, but amplify the incoming signal in nonlinear ways, their processing properties make them central to how information is processed by neural systems. It is essential, then, to assess the developmental processes that shape dendritic form and the role of these process in determining the representational properties of neural circuits.[2]

2.3.2 Dendritic arborization

Figure 7: Camera lucida drawings of basal dendrites of layer V human pyramidal cells; a, newborn; b, 3 months; c, 6 months; d, 15 months; e, 24 months; f, adult (from Schade & van Groenigen 1961).

Table 3 summarizes some further studies of progressive dendritic expansion. Although there is little doubt that regressive events also occur in dendritic development (e.g., Koester & O'Leary 1992; Vercelli et al. 1992), the above examples motivate the search for the processes regulating dendritic development.[3] In the following sections, we accordingly examine the mode of dendritic development in some detail---the extent of progressive processes at the level of dendritic structure and their malleability by changes in activity. From this, we go on to formulate some features of dendritic development, considering their cellular basis, and relating these to the learning and representational properties of cortex.

2.3.3 Environmental conditions and dendritic development

Study	System	Remarks
Valverde, 1968	mouse visual cortex, stellate cells	enucleation resulted in dendrites redirected toward remaining afferents
Ruiz-Marcos and Valverde, 1970	mouse visual cortex	enucleation resulted in decrease in dendritic complexity, most pronounced in layer III pyramidal cells
Valverde, 1971	mouse visual cortex	dark-reared subjects placed back into normal environment; new growth in apical dendrites seen by 2 days
Volkmar and Greenough, 1972	occipital cortex	enriched environmental rearing resulted in increased higher order branches
Greenough and Volkmar, 1973	occipital cortex	follow up of Volkmar and Greenough (1972); found most increase in basal dendrites of pyramidal cells
Borges and Berry, 1976, 1978	rat visual cortex, layer IV stellate cell	dark rearing reoriented dendrites toward residual input
Ulyings et al. 1978	adult rat, visual cortex	enriched environments increased dendritic complexity and length of layer II, III pyramidal cells
Fiala et al., 1978	dentate granule cells	significant differences between complex and impoverished environment dendritic branches, length and width of dendritic fields
Juraska et al. 1980	adult rat, visual cortex	enriched environments increased dendritic complexity and length
Camel et al., 1986	rat visual cortex	dendritic alterations induced by exposure to a complex environment persisted even after return to individual caging for 30 days
Harris and Woolsey, 1981	mouse somatosensory cortex	vibrissal removal results in reduced representation in corresponding barrel cortex with increase in spared vibrissae
Conlee and Parks, 1983	avian cochlear nucleus	monaural acoustic deprivation led to 38% reduction in dendritic length
Schilling et al., 1991	in vitro study of Purkinje cell development	branching of Purkinje cell dendrites depends on functional synaptic contacts
Wallace et al., 1992	rat visual cortex, layer III pyramidal cells	+6 % increased total dendritic length in basal dendrites within four days of exposure to a complex environment
Mooney et al., 1992	hamster superior colliculus	enucleation results in superior colliculur neurons to redirect their dendrites and develop response properties appropriate for the spared modality

Table 4: Representative studies demonstrate the effects of differential rearing and manipulations to activity on dendritic development. See text for details.

Unlike axons, which in many cases begin to grow during migration (Shoukimas & Hinds, 1978), dendrites typically do not begin to differentiate until they complete their migration and their final placement within a cortical layer (Noback & Purpura 1961). This suggests that the cellular environment may be a particularly important factor in determining dendritic form, as studies of genetically altered animals have demonstrated (Rakic & Sidman 1973; Mariani et al. 1977; Caviness & Rakic 1978; Pinto-Lord & Caviness 1979).

Table 4 summarizes some studies on the effects of manipulating input pathways on dendritic development. One of the earliest is Valverde (1968), which examined the effects of enucleation on stellate cell dendrites in mouse visual cortex. As Figure 8a illustrates, in normal development cells outside layer IV, the location of the primary thalamic projection, extend dendrites throughout layers III, IV, and V. In contrast, as Figure 8b illustrates, in enucleated animals, cells outside layer IV did not project their dendrites into that layer. Instead, they directed their dendrites toward layers III and V, as though they were looking for afferent input outside layer IV. Valverde (1968 p.290) concludes, "dendrites are not passive structures but actively growing neuronal formations which must accommodate to changing functional demands.''

Figure 8: Dendritic organization in visual cortex of normal mice (A) and enucleated mice (B). The degenerative afferent termination is evident in B, where layer IV is sparsely covered with dendrites, whereas adjacent layers are more heavily covered, suggesting that these dendrites have reorganized according to remaining patterns of afferents (from Valverde 1968).

The dependence of dendritic form on patterned activity is supported by analogous experiments in the vertebrate optic tectum (Katz and Constantine-Paton 1988). Although the optic tectum normally receives input from only one eye, it can be induced to receive input from two eyes by experimentally adding a third eye primordium during embryonic development (Constantine-Paton & Law 1978). In these cases, afferents from the two eyes segregate into alternating stripes reminiscent of ocular dominance columns, which are not present in the normal frog. A striking result of the Katz and Constantine-Paton (1988) study was that tectal cell dendrites not normally receiving input from more than one eye nonetheless become organized so as to respect the experimentally induced stripes. As in the Katz et al. (1989) study, it is the degree of correlation in the afferent activity rather than simply the presence of activity that underlies dendritic organization.[4]

An interpretation of these results is that dendritic segments function as detectors of correlated activity and grow preferentially in such regions. Support comes from Tieman and Hirsch's (1982) finding that exposure to lines of only one orientation during development has specific effects on dendritic development. The dendritic field orientations of cells from cats raised with exposure to lines of a single orientation were significantly elongated in conformity with this shift in the visual environment.

An insight from this study is that a dendritic tree samples its input space actively in response to the environmental structure. A similar result has been obtained for layer IV stellate cells by Coleman et al. (1981), who suggest (p.19): "[I]f an alteration of the spatio-temporal pattern of the afferent activity is sufficient to lead to dendritic alterations during development, this implies that dendritic trees may develop in a form that will optimize spatio-temporal summation for the postsynaptic neuron."

Recently, Kossel et al. (1995) used many of the experimental manipulations that led to activity-dependent rules for axonal growth to examine dendritic growth. They used double labeling techniques to visualize OD columns and dendritic morphology simultaneously under conditions of monocular deprivation and divergent squint (strabismus). Strabismus results in a decrease in between-eye correlations and should therefore enhance ocular segregation, as has been seen in the case of presynaptic arborizations (Shatz et al. 1977). Kossel et al. (1995) found this to be the case for the dendritic fields of layer IV stellate cells, the primary cell type that seems to reflect the afferent arborization. In the case of monocular deprivation, however, dendrites in the non deprived column were only weakly influenced by borders, reflecting the decrease in uncorrelated activity across that border.

Kossel et al. (1995) also found that cells in the deprived column extended their dendrites into the nondeprived activity. This agrees with other evidence we have reviewed that dendrites are not merely passive structures but actively seek out regions of correlated activity. As Kossel et al. (1995) conclude, their results suggest that dendrites develop according to the same sorts of rules that have been suggested for axonal arbors and that both structures develop according to patterns of correlated input activity.

2.3.5 The cellular basis of directed dendritic growth

The activity-dependent stabilization of co-active synapses has come to be a central developmental theme in both experimental work (rev. in Shatz 1990; Cline 1991) and computational models of development (reviewed in Goodhill, 1992). The discovery of the NMDA type of glutamate receptor made an important contribution to this research because the kinetic properties of the NMDA-receptor fit with the cooperative model of synaptic plasticity in both developmental (reviewed in Shatz 1990; Cline 1991) and mature forms of plasticity (Kelso et al. 1986).

Neurobiologists refer to the cooperative model of synaptic plasticity as "Hebbian learning," after Donald Hebb's (1949) proposal for a neurally plausible associative learning rule. In development, however, Hebbian learning is generally given a selectionist interpretation as a rule governing the stabilization of existing synapses. Hebb, though, made his original proposal in the context of neural development and the activity-dependent construction of new synapses in collections of neurons he called "cell assemblies." Hebb even discusses Kapper's neurobiotaxis theory, an early, extreme constructivism, and defends a limited version of it. Ironically, Hebb was reluctant to embrace a stronger version of constructivism because of Sperry's (1943) influential work. Sperry's elegant work on the regeneration of the retinotectal pathway led to his "Chemoaffinity hypothesis," that neurons bear unique molecular addresses making their connections precise, a hypothesis that would dominate neurobiological thinking for three decades.

There is suggestive evidence that neural constructivism is the most appropriate one for the NMDA-receptor's properties and that the Hebbian model should include directed growth. For example, from their experimental observations, Katz and Constantine-Paton et al. (1988) suggest that such a broader action of the NMDA-receptor's associative principles may underlie the organization of dendritic structures; they state (p.3178):

Our observations that single tectal dendrites can function as autonomous postsynaptic detectors of correlated afferents are consistent with the proposed role for the NMDA conductance. Depolarization of a single dendrite by activity in a subset of converging synapses would allow glutamate to activate the conductance within a restricted domain of the postsynaptic cell. This could, in turn, provide cues for stabilizing and enlarging a small portion of the dendritic arbor, independent of the behavior of other dendrites.

Cell culture studies further support the role of NMDA-mediated constructive processes in dendritic development. For example, Brewer and Cotman (1987) found that NMDA-receptor mediated activity in hippocampal dentate granule cell cultures results in significant branching and outgrowth whereas NMDA blockade leads to a significant decrease in these measures. Similar results have been reported in a variety of other systems (e.g., Pearce et al. 1987; Balazs et al. 1989; Bulloch & Hauser 1990).[5]

Recently, Williams et al. (1995) have shown that local stimulation along developing neuronal processes results in branching. These new branches are stabilized if the appropriate targets or signals are present. This branching is highly regulated and is calcium-dependent, as are the mechanisms involved in Hebbian learning. This again suggests that dendritic structure is added to those areas of activity to support more input from sources localized to that region.

3 Directed dendritic development and representational change

What sort of representations does the brain use? One of the most important principles of cortical representation involves "geometric principles of information processing design" (Mitchison & Durbin 1986; Mead 1989; Durbin & Mitchison 1990; reviewed in Churchland & Sejnowski 1992). Regarding this principle, Mead (1989, p.277) states:

Computation is always done in the context of neighboring information. For a neighborhood to be meaningful, nearby areas in the neural structure must represent information that is more closely related than is that represented by areas further away. Visual areas in the cortex that begin the processing sequence are mapped retinotopically. Higher-level areas represent more abstract information, but areas that are close together still represent similar information. It is this map property that organizes the cortex such that most wires can be short and highly shared; it is perhaps the single most important architectural principle in the brain.

From this principle, the physical structure of a neural area corresponds to a representational space. In this representational space, nearby things are more closely related semantically than things that are far apart. This map property is extremely powerful as a representational system. When brain areas can communicate, increasingly rich representations can be successively built up. Each area is a layer in an increasingly abstract feature space. Just as information in a map is held by such spatial properties as physical distance, the physical structure of cortex encodes information. With geometric principles of information processing the information is held in the three-dimensional pattern of neural connectivity. As constructive factors play a central role in building this physical structure, they also shape the representational properties of cortex. Building neural circuits with directed growth thereby builds the brain's representational properties.

These spatial properties of representation are largely lost in the traditional connectionist network because of the way the connectionist neuron integrates information, typically summing its input and sending a (perhaps graded) output if some threshold is exceeded. This makes the entire cell the basic computational unit. In contrast, biological neurons are thought to segregate into sub-regions that function as autonomous processors. Local dendritic segments might be the brain's basic computational units (see also Koch et al. 1982, 1983; Shepherd & Brayton 1987; Mel 1992a; 1992b; 1994; Jaslove 1992; Segev & Davis 1995). Dendrites are not simple signal integrators with passive conductance properties, as in classical cable models (Rall 1964). Imaging studies have found that some dendritic systems (e.g., CA1 hippocampal neurons) have a heterogeneous distribution of voltage-gated Ca2+ channels, suggesting nonlinear membrane properties (Jones et al. 1989; Regehr et al. 1989). Intradendritic recordings in these cells likewise reveal strong nonlinearities in their electrical properties (Wong et al. 1979; Bernardo et al. 1982;). In some instances, these properties make a dendritic segment act more like an axon, sending an amplified signal to the cell body (Stuart & Sakmann 1994).

Nonlinear properties give a cell more computational power than traditionally thought (Feldman & Ballard 1982; Durbin & Rumelhart 1989; Mel & Koch 1990; Koch & Poggio 1992). A cell with active dendritic segments can perform the nonlinear discrimination that requires a hidden-layer network of connectionist neurons. The spatial properties of a cell may also determine many of its functional properties. To connecting this back with our earlier discussion of geometric principles of information processing, when such a cell is embedded in a representational space, its spatial structure takes on additional significance. A phenomenon referred to as the "clustering" of related inputs onto dendritic segments results in a pattern of termination mirroring the informational structure of input: electrotonically close synapses encode common features of the input space and effectively fire the cell (Mel 1992a; 1992b; 1994). The presence of cluster-encoded features significantly alters both the representational properties of cortex and its computational power.

3.1 Developmental mechanisms underlying clustering

For clustering to serve an interesting information processing role, there must be an appropriate developmental scheme underlying its formation. For this, temporal patterns of activity must somehow be translated into appropriate spatial patterns of termination. As Mel states (1992b, p.43):

the ordering of afferent connections onto an excitable dendritic arbor is a crucial determinant of the cell's responses to different patterns of synaptic input: It is this ordering, or permutation, that determines which input patterns will activate synapses that are spatially grouped into clusters, and which will not. If the nervous system is to take advantage of this capacity for pattern discrimination based on spatial ordering, then a learning mechanism capable of manipulating synaptic ordering must be available to these neurons.

A number of Hebbian schemes have been proposed to subserve the formation of these clusters, with a cell able to tune itself to its input space (Mel 1992a; 1992b). Many of these schemes are biologically implausible, however, because of what is known as "the problem of sampling."

The sampling problem is the needle in a haystack problem: clusters depend on forming contacts from axons carrying similar information onto a single dendritic segment. Rearranging contacts involves the problem of finding the right dendritic segment. The sampling problem has been considered in a more general context by Montague at al. (1991) and Gally et al. (1990). In view of the developing nervous system's sparse connectivity, Gally et al. suggested that a spatially diffusible substance was acting (see Figure 9). Not confined to the anatomically defined synapse, a spatial signal is free to diffuse into a local volume, thereby potentially affecting all cells synapsing in that volume, whether or not a given cell shares a synaptic contact with it. In particular, Gally et al. proposed that nitric oxide, a membrane permeable gas, has a number of characteristics that make it a leading candidate for such a role. Subsequent research has confirmed that nitric oxide plays a key role in synaptic plasticity (Bohme et al. 1991; Haley et al. 1992; O'Dell et al. 1991; Schuman & Madison 1991) and transmission (O'Dell et al. 1991; Manzoni et al. 1992; Montague et al. 1994).

Figure 9: A diffusible substance allows synapses in a local volume of tissue to communicate whether or not they share a connection. Using such a signal, it is possible for synapse X1 and X4 to modify their weights according to an associative learning rule. (From Montague & Sejnowski 1994). One such rule is: , where is the change in the `weight' or synaptic efficacy of a connection, is a constant controlling rate of change of synaptic efficacy, is a measure of presynaptic activity and is a threshold that determines whether a terminal is active at time t. is a threshold, dependent on the activity of the presynaptic terminal, which determines the direction of synaptic change. The postsynaptic factor of typical Hebbian rules has been replaced by a term for substance concentration, at time t located at position r.

In large-scale computer simulations in collaboration with P.R. Montague we are exploring how this scheme may be readily modified to include activity-dependent branching. The probability of branching/retraction at a terminal segment can be made proportional to the weight of nearby synapses over time. Making the probability of branching depend on synaptic weight automatically transfers the associative conditions necessary for weight changes to those for branching/retraction. The value of directed growth into these volumes is that it augments the processes leading to what we refer to as spatial clustering, that is, functional clustering of statistically correlated afferent axons into spatial domains defining higher-order features of the input space. This, then, corresponds to locally regulated growth, allowing differential sampling as a function of the correlational structure of input patterns to form spatial clusters. In addition, since the production of the diffusible substance is postsynaptic, the postsynaptic structures play an important role in determining the properties of this feature space. Other mechanisms, such as the distribution of membrane channels and localized inhibitory input, will also participate in defining these clusters. We suggest that the establishment of spatial domains as regions of higher-order features will be central to the information-processing properties of neuronal populations.

3.2 Hierarchical Representation Construction

A particularly striking feature of the human brain is the protracted development of some of its parts. It is sometimes suggested that the severe constraints placed by bipedalism on the dimensions of the birth canal forced the human brain to be particularly immature at birth. There is nothing unique about the human brain's degree of immaturity at birth, however. As Gibson (1990) points out, at birth the human brain is about 25% of its mature weight, making it about average among mammals. In contrast, the kitten's brain at birth is only 10% of its mature weight. Depending on cortical layer, dendrites of human primary visual cortex cells are between 30 and 50% of total adult lengths at birth. For these layer V pyramidal cells, adult lengths are reached by around four months compared to 18 months for layer III cells.

Much of non visual cortical development, in contrast, displays an extensive and more protracted development. Cells in frontal cortex are far slower to develop and undergo the majority of their growth after two years of age (Schade and van Groenigen, 1961). In addition, the extent of their postnatal development is dramatic---they grow to over thirty times their dendritic length at birth. Scheibel (1993) likewise reports a long period of dendritic development in Broca's area in which mature forms emerge only after 6-8 years. Why, then, is human non-visual cortical development so slow to develop and so extensive?

Our view is that the human brain's development is a prolonged period in which environmental structure shapes the brain activity that in turn builds the circuits underlying thought. In place of pre-wired modules, patterned activity builds up increasingly complex circuits, with areas staging their development. Cortical areas further away from the sensory periphery wait in anticipation of increasingly complex patterns of activity resulting from development in lower areas. As this development proceeds, areas of the brain become increasingly specialized for particular functions, reflecting a cascade of environmental shaping. Some brain circuits close to the sensory periphery, such as in our early visual system, are in place by six months of age; but those in language areas, further away from the sensory periphery, do not begin to complete their development until the eighth year of life.

3.3 What is the role of regressive events in development?

The evidence we have examined demonstrates that the popular view of development as largely a regressive event must be reconsidered. We suggest that regressive events are simply the consequence of reduced neural specificity, as indicated by the counterevidence to Sperry's chemoaffinity hypothesis. Any theory, whether selectionist or constructivist, that rejects a strong view of neural specificity will thus need to posit regressive events. If cells do not bear nearly unique molecular addresses, then stochastic sampling mechanisms must be posited. These will by their very nature introduce some structure into a system that will later be eliminated. Neural constructivism allows these sampling mechanisms to be directed, but they are still stochastic. Structural elimination, or error-correction, are likewise required, but this does not mean that error-correcting processes are the only developmental mechanisms, or that developmental selection occurs only among intrinsically generated structures. [7] Rather, selection is only one kind of process in a dynamic interaction between environmentally derived activity and the neural growth mechanisms that activity regulates. This changes the emphasis from synaptic elimination to synaptic turnover. New techniques that allow the same structures to be examined successively over time (O'Rourke et al. 1994; O'Rourke & Fraser 1986; 1990; Purves & Hadley 1985; Purves et al. 1986; 1987; Dailey & Smith 1996) reveal that constructive and selective events co-occur as required for directed sampling mechanisms.

Where does this leave the selectionism? We see no way for its strong interpretation to include mechanisms for directed growth without trivializing its driving analogy from population biology. Development that is directed is not selectionist---if environmental structure builds neural circuits, instead of simply selecting among variation created by intrinsic mechanisms, then the strict selectionist position is untenable.

4 A learning-theoretic approach to development

4.1 Development and learnability

The best known characterization of a developing system's learning properties comes from language acquisition---what syntactic properties a child could learn, what in the environment could serve as evidence for that learning, and ultimately, what must be prespecified by the child's genetic endowment. From these questions, 30 years of research have provided mainly negative results: the environment is impoverished and could provide only limited information, so few syntactic properties could be learned. In the end, theorists concluded that the child must bring most of its syntactic knowledge, in the form of a universal grammar, to the problem in advance. Learning could only serve to set a few free parameters in that universal grammar so as to particularize it to the language confronting the child.

Although it is also based on empirical studies of linguistic input (e.g., Brown 1973), the perception that this striking view of syntax acquisition is based primarily on rigorous results in formal learning theory makes it especially compelling. Indeed, above all it is this formal feature that has prompted its generalization from syntax to the view of the entire mind as a collection of innately specified, specialized modules (e.g., Fodor 1983; Barkow et al. 1992; Gazzaniga 1992; Hirschfeld & Gelman 1994). Although Piaget's legacy remains undeniable in developmental psychology (e.g., Fischer 1980; Bates and MacWhinney 1987; Karmiloff-Smith 1992) it is probably no overstatement to suggest that much of cognitive science is still dominated by Chomsky's nativist view of the mind.

According to formal learning theory, development is a learning problem and so is constrained by the learning-theoretic pressures confronting any learner (Gold 1967; Pinker 1979; Wexler & Culicover 1980; Osherson et al. 1986). This assumption allows for a very general characterization of the learner. The classic formulation derives from Mark Gold's work on language identification (Gold 1967). Gold established upper bounds or worse-case scenario results by asking what a general learner could learn when presented with example sentences of some language. Gold supposed that the learner's task was to conjecture a hypothesis regarding the grammar that might generate that language. The learner was said to identify the language in the limit if it eventually chose a grammar that was consistent with every string.

A good question to ask is, where does Gold's learner get the grammars that it conjectures? Gold's learner adopts a maximally general strategy and first simply enumerates every grammar belonging to some class of grammars. Starting with the first grammar, the learner then rejects each one in turn if it is inconsistent with what it has seen so far and tries out the next grammar in the enumeration.

Such a learner will eventually find the right grammar if it has some finite position in the enumeration. The formal definition of a language from mathematical logic lends itself to formulating the languages that can be learned in this scenario. Primitive recursive languages emerge from a ranking of grammars known as The Chomsky hierarchy as the most powerful that can be learned by Gold's learner. They are the most powerful decidable language, which means that the right grammar will indeed have a finite place in the enumeration.

Some immediate troubles arise from Gold's model. As Pinker (1979) notes, this learner may have to test on the order of 10100 possible grammars even in an extremely simplified case---a computation that could never actually be performed. The learner is so slow because of the general strategy it adopts. Although this guarantees convergence, learning becomes in general impossible because of the vast search it requires. These prohibitive results may seem to indicate that language learning is impossible, but their consequences are ambiguous because of some major limitations. Even ignoring their dubious assumptions regarding the psychology of learning, there are two internal limitations: their concern only for convergence in the limit and their requirement that the learner precisely identify the target concept (no mistakes allowed).

In 1984, Les Valiant introduced a probabilistic model of learning that remedied these two limitations; his accordingly becoming the standard model of inductive inference in the field (see Dietterich 1990 and Natarajan, 1991 in the case of machine learning). Rather than disallowing any mistakes, Valiant's learner could make a hypothesis that was only a good approximation with high probability. This framework was dubbed the "probably approximately correct" (PAC) model of learning. It also addressed the question of convergence time, as it distinguished between feasible and infeasible learning by classifying problems according to whether or not they were learnable in polynomial time. Valiant's model thus shifted the main emphasis of the learning problem from what is in principle learnable to what is learnable from some representation class in feasible time.

As we mentioned, the key result that came out of the Gold paradigm was that the child must come equipped with a highly restricted set of hypotheses regarding the target grammar---in the case of language, a universal grammar. This conclusion falls out of the view of learning as essentially a search problem in a hypothesis space (e.g., searching through the grammars) to the target concept. To make this a feasible search, the space must be restricted by building in an inductive bias, roughly the system's background knowledge. One of the Valiant model's key virtues was that it quantified the relation between inductive bias and learning performance from within a complexity-based account (e.g., Haussler 1989). Results with Valiant's model thus showed how difficult some problem was to learn with various inductive biases or background knowledge.

The Valiant model thus demonstrated what could not be fully characterized in the earlier limit-based formal learning theory: learning systems face severe learning-theoretic pressures and can be successful in some domain only if they have solved this difficult prior problem involving representation. That is, from the perspective of the PAC model of learning, the fundamental problems of learning are not those involving statistical inference; they instead center around how to find appropriate representations to underlie efficient learning (reviewed in Geman et al. 1992). This problem precedes the treatment of learning as statistical inference, as a learner's choice of representation class (background knowledge) largely determines the success of learning as statistical inference.

What makes learning so difficult? In statistical studies, estimation error is decomposed into two components: bias and variance. Very roughly, bias is a measure of how close the learner's best concept in its representation space approximates the target function (the thing to be learned). Variance refers to the actual distance between what the learner has learned so far and the target function. To make this a bit more concrete, a small neural network will be highly biased in that the class of functions allowed by weight adjustments is very small. If the target function is poorly approximated by this class of functions, then the bias will contribute to error. By making a network large, hence flexible in terms of what it can represent (by decreasing bias) variance's contribution to error typically increases. That is, the network has many more possible states, and so is likely to be far away from the function of interest. This means that very large training sets will be required to learn because many examples will be required to rule out all the possible functions.

As Geman et al. (1992) state it, this results in a dilemma: highly biased learners will work only if they have been carefully chosen for the particular problem at hand whereas flexible learners seem to place too high a demand on training time and resources. This is essentially the same impasse that leads to nativism. Learning is too hard without severely restricting what can be learned. Indeed, from an entirely different perspective, Geman et al. (1992) likewise suggest that deliberately introduced bias (the nativist route) may be the best way out of this dilemma.

What makes these results interesting for the present discussion is that this basic problem of representation choice is a developmental one for natural systems. This, then, implies that the fundamental problem facing natural cognitive systems is a developmental one. How have natural systems chosen a developmental strategy to get out of this impasse?

Once we are talking about natural systems, it is worthwhile to raise a neurobiological constraint. So far, this discussion has proceeded as though the only significant factors were learning-theoretic pressures, but it is particularly important to consider whether the views coming out of learning theory are consistent with neurobiological constraints on development. For natural systems, the constraint that a learning theory should be consistent with underlying neural mechanisms has been severely underestimated. Indeed, in our opinion this biological constraint has equal footing with the learning-theoretic one and both must be viewed as complementary constraints that developmental systems must satisfy.

As suggested by Quartz and Sejnowski (1994), the view that strong, domain-specific knowledge is built into cortical structures runs into severe difficulties from developmental neurobiological evidence. Although we will not review that material in detail here, recent experiments on heterotopic transplants (Stanfield & O'Leary 1985; Schlaggar & O'Leary 1991; reviewed in O'Leary et al. 1992), cross modal rewiring (Frost 1982; Sur et al. 1988; Roe et al. 1990, 1992; Pallas et al. 1990; reviewed in Sur et al. 1990; Shatz 1992) and clonal analysis of cell migration (Walsh & Cepko 1988; 1992, 1993) all establish that the regional characteristics of mature cortex depend fundamentally on interaction with afferent input. While the cortex is not a tabula rasa, as there may be a common basic circuitry and repetitive arrays of cell types (see O'Leary et al. 1992), it is largely equipotential at early stages (in agreement with studies on cortical plasticity and early brain damage (e.g., Neville 1991).

Consistent with this view, O'Leary (1990) refers to the immature cortex as protocortex, which shares a common laminated structure, cell types, and basic circuitry but which diminishes the need for prespecification. It is the differing pattern of afferent activity, reflective of different sensory modalities, that confers area-specific properties onto the cortex---not predispositions that are somehow embedded in the recipient cortical structure. In addition, the fact that many of these processes operate before birth, as in the case of spontaneous visual activity (Maffei & Galli-Resta 1990; Meister et al. 1991), suggests that cortical specification could begin by the very mechanisms that will be used postnatally through interaction with an environment. Hence, the fact that various regions of cortex receive different patterns of afferent termination and activity seems to be the prime determinant of specialized cortical functions. A system in which the cortex is "enslaved by the periphery" has a number of clear advantages in terms of responding flexibly to varying environmental pressures, plasticity, and changing body size (see Walsh & Cepko 1992; 1993). In the previous section, we tried to suggest how this interaction between developing cortex and environmentally derived activity builds up the neural circuits underlying cognition.

Adding the neurobiological constraint to the learning-theoretic one results in yet another impasse. From the perspective of learning theory, it appeared that the only response to the learnability problem was to build in much of the problem domain a priori in the form of highly specialized structures. Yet, from the perspective of biological constraints it appeared that cortical structures do not build in this knowledge, but rather allow both pre- and post-natal activity to determine features of the cortex. In the following section, we suggest that the neural constructivism offers a powerful means of escaping this impasse.

4.2 Constructive learning

The learning paradigm that is used in both Gold's and typically in Valiant's model is known as selective induction, with learning amounting to the search through some representation or hypothesis space. Most studies of learning assume that the learning mechanism's resources are stationary, or time-invariant, with learning a process of selective induction. Simple counting arguments show that the probability of a learner searching through a fixed hypothesis space to successfully learn a concept chosen at random is exponentially small (reviewed in Dietterich 1990). For this reason, the hypothesis space must be an exponentially small subset of possible concepts (see Blumer et al. 1988). This restriction in a learner's expressive power has far-reaching consequences, particularly in terms of limiting its flexibility.

To see these consequences, the first question to ask is, what does failure signify on such an account? Since the hypothesis space must be a very restricted subset of possible concepts, failure to learn may simply indicate that the learner chose the wrong hypothesis space; this may say nothing about the learnability of some class of concepts. As Baum (1989, p.203) states, "a pragmatic learner should be willing to use any class of representations necessary to solve his problem. He should not be limited by a priori prejudices." Is there a way for a learner to be more flexible, to avoid having to make such a restrictive initial choice of representations?

The constructivist learner shows its advantages here. It does not involve a search through an pre-defined hypothesis space, and so it is not one of selective induction (also known as model-based estimation, or parametric regression). Instead, the constructivist learner builds its hypothesis space as it learns. This has shifted the problem from one of parameter estimation to a nonparametric regime. We must accordingly ask, what is the effect of allowing a system to add structure---to build representations---as it learns?

Here again nonstationarity poses a challenge since we are asking about the effects of building representations according to the features of the learning problem. Neural network research has been particularly helpful in characterizing this sort of nonstationarity because the close relation between a network's architecture and its representational properties provides a constrained framework for looking at representational change during learning.

An increasingly sophisticated literature on the formal properties of neural networks has emerged. For example, a number of general results on the approximation properties of neural networks have been established (e.g., Cybenko 1989; Hornik et al. 1989; Girosi & Poggio 1990). From a nonparametric framework, White (1990) has demonstrated that a network that adds units at an appropriate rate relative to its experience is what statisticians call a consistent nonparametric estimator. This asymptotic property means that it can learn essentially any arbitrary mapping. The intuition behind this result, which will play a central role in characterizing constructive learning, follows a general nonparametric strategy: slowly increase representational capacity by reducing bias at a rate that also reduces variance. Since network bias depends on the number of units, as a network grows, its approximation capacities increase. The secret is regulating the rate of growth so that variance's contribution to error does not increase. Encouraging bounds on the rate of convergence have recently been obtained (Barron 1994).

White's demonstration of the power of neural networks depends on allowing the network to grow as it learns. In fact, many of the limitations encountered by neural networks are due to a fixed architecture. Judd (1988) demonstrated that learning the weights in a neural network is an NP-complete problem, and therefore computationally intractable, a result that extended to architectures of just three nodes (Blum & Rivest 1988). These results suggest that severe problems may be lurking behind the early success of network learning. As Blum and Rivest (1988) note, however, these results stem from the fixed architecture property of the networks under consideration. In contrast, the loading problem becomes polynomial (feasible) if the network is allowed to add hidden units. This suggests fundamentally different learning properties for networks that can add structure during learning. This has been confirmed by studies such as that of Redding et al. (1993), who presented a constructivist neural network algorithm that can learn very general problems in polynomial time by building its architecture to suit the demands of the specific problem.

Underlying this sort of result is Baum's (1988; 1989), demonstration that networks with the power to add structure as a function of learning are complete representations, capable of learning in polynomial time any learning problem that can be solved in polynomial time by any algorithm whatsoever. As Baum notes (1989, p.201), this makes the learner a sort of general or universal one. This is in contrast to systems which utilize incomplete representations, as in a fixed hypothesis space. Most negative learnability results, such as those for syntax, depend on a system using incomplete representations (see below). If a network is allowed to build its representations as it learns in response to the informational structure of its environment, the question of learnability shifts from the question of what is learnable from some particular representation class to the question of what is learnable from any representation class.

The general strategy of constructivist learning is this. Rather than start with a large network as a guess about the class of target concepts, avoid the difficulties associated with overparameterized networks by starting with a small network. The learning algorithm then adds appropriate structure according to some performance criterion and where it is required until a desired error rate is achieved. Since the construction of the learner's hypothesis space is sensitive to the problem domain facing the learner, this is a way of tailor making the hypothesis space to suit the demands of the problem at hand. This allows the particular structure of the problem domain to determine the connectivity and complexity of the network. Since the network has the capacity to respond to the structure of the environment in this way, the original high bias is reduced through increases in network complexity, which allows the network to represent more complex functions. Hence, the need to find a good representation beforehand is replaced by the flexibility of a system that can respond to the structure of some task by building its representation class as it samples that structure to learn any polynomial learnable class of concepts. Research on constructive algorithms has become increasingly sophisticated, and the results with constructive learners are impressive (e.g., Fahlman & Lebiere 1990; Frean 1990; Hirose et al. 1991; Platt 1991; Azimi-Sadjadi et al. 1993; Wynne-Jones 1993; Kadirkamanathan & Niranjan 1993; Shultz et al 1994; Shin & Ghosh 1995).

The research we have just examined indicates a fundamental distinction between the constructivist strategy and models of selective induction. For the latter to have any chance of learning, the network must build in much of the problem domain a priori. Besides the neurobiological implausibility of this strategy, there are more general reasons why using highly biased networks is not a sound strategy in the biological case. Primary among these is that the highly biased network will only work for the specified application, but if the nature of the application is not properly predicted, the network will be a poor performer. Hence, tailor-making network architectures to suit the particular demands of some problem domain may be a useful heuristic strategy for artificial networks whose problem space is defined, or at least delimited, in advance by the designer. Biological learners, however, face an additional problem: not only is the problem space not defined beforehand, it is changing---the environment is highly nonstationary. Systems that are highly specialized for the anticipation of a particular problem domain will fail in the event of significant changes to that domain. The upshot is that specialization may bring efficiency, but it comes at the expense of flexibility. Although most natural systems are only confronted with ecological change, human cognition requires highly flexible and adaptive representations to accommodate both cultural and technological innovations. We doubt that the pace of this change can be met by a representational scheme requiring a major intrinsic specification.

4.3 Neural constructivism and language acquisition

Because constructive learning shifts the learning problem from what is learnable from a fixed hypothesis space to that of what is learnable from any representation class, it suggest that the class of learnable concepts might be larger than traditionally supposed. It is, therefore, worthwhile to ask whether constructive learning has any bearing on the results coming from Gold's work.

We can approach this question by first asking what the results from Gold's work really show. Do they demonstrate that syntax is unlearnable? The shift in the meaning of learnability we just mentioned suggests that the unlearnability of syntax has two possible senses. It may mean that syntax is not learnable from some fixed hypothesis space H. Two possible causes underlie this sort of unlearnability: either the target function (encoding syntax) is too large or H is too restricted (see Baum 1988; 1989; Valiant 1991). Most negative results are of the second sort. As we showed, the constructivist learner escapes these sorts of negative results by constructing more powerful representations than those contained by the fixed architecture. So, in this case a negative result just indicates that a poor hypothesis space was chosen---it is only a negative result for this specific hypothesis space and says nothing about the learnability of syntax itself.

Most cognitive scientists, however, do not view the unlearnability of syntax as this sort of result. Instead, they see it as a representation-independent result. This is a much stronger sort of result, claiming that syntax is unlearnable relative to any hypothesis space. In this case, there would be no reason to look for more effective representations or systems that can build representations as they learn because no representation at all could possibly suffice. Is this justified? The answer is no---the only representation-independent results are for complicated cryptographic functions, such as those known as "polyrandom functions" (functions that cannot be distinguished from purely random ones in polynomial time; see Goldreich et al. 1984). This type of representation-independent result, however, is of little relevance to the learnability of syntax, or for the sorts of concepts natural systems must learn. Learning syntax is nothing like having to solve the general decryption problem. Hence, although the general perception is that Gold's work proved syntax to be representation-independent unlearnable, there is no justification for this strong claim.

The negative results regarding syntax are of the weaker sort: unlearnability relative to some fixed hypothesis space. It is also important to point out that there are some idiosyncratic features of Gold's learner that make learning appear to be hard: learning as selective induction, a stationary learner, extremely dubious assumptions regarding the psychology of learning, an impoverished account of linguistic input, a worst-case analysis, and extremely rigid performance conditions. Above all, because Gold's learner uses such a general strategy, simply enumerating an entire class of grammars, and then evaluating learning in the worst case, its results are limited to its own framework and have little applicability to the problem of learning in general. Indeed, to us the main lesson of learnability arguments in Gold's paradigm demonstrate is the insufficiency of its own model---the baby may have been thrown out with the mathematical bathwater.

Since syntax appears to belong to the class of concepts that are learnable by natural systems, as indicated by it not being a representation-independent unlearnable class, the appropriate response to results from Gold's framework is to reject this model of learning and begin to explore alternatives. In particular, nonstationary learners, long dismissed by Chomsky and others (e.g., Pinker, 1984), offer a more powerful response to the problem of learning. In particular, constructive learning is a maximally powerful approach, in that it forms complete representations, capable of learning any learnable concept.

The powerful learning properties of constructive learning are not its only advantages. We suggested that all candidate learners must satisfy both learning theoretic and neurobiological constraints. Constructive learning points to the dynamic interaction between a structured environment and the neural mechanisms that are responsive to that structure. As such, it minimizes the amount of built-in structure required, making it the only learner consistent with a largely equipotential cortex. Constructive learning is, therefore, the only learner consistent with both learning and neurobiological constraints.

The themes we have presented in this target article are very simple steps toward characterizing the complex interactions between developmental mechanisms and a structured environment. Already, however, we think they force extreme caution in formulating theories of acquisition in their absence. Although this interaction will be no doubt far richer than what we have captured, it raises some intriguing possibilities that have been discounted under the influence of nativist approaches, which we consider next.

No learner can be completely assumption free since pure tabula rasa learning is impossible---there must be some built-in assumptions. A future research direction will be to characterize the sorts of biases that are consistent with a largely equipotential cortex: those deriving from such features as generic initial cortical circuitry, conduction velocities, subcortical organization, learning rates, and hierarchical development. The way these constraints provide grounding for constructive learning to build the mature representational structures of cortex will likely be a very rich account, although the tools required to understand this complex interplay are still rudimentary.

We also think it is important to turn attention back to examining environmental structure, as in earlier traditions of developmental psychology. Both nativism in psychology (e.g., Chomsky 1965; 1980) and selectionism in neurobiology (e.g., Edelman, 1987) have made much of poverty of the stimulus arguments. The upshot of these arguments has been a depreciation of the environmental structure's role in guiding acquisition. As neural network and neurological research are finding, however, there appears to be far more structure latent in the environment than the poverty of the stimulus arguments suppose. In addition, we think the details of human cortical development provides another clue to the richness of environmental structure. Because human cortical development is much more prolonged and extensive than what purely physical limits predict, we think this suggests that the human brain's evolution has maximized its capacity to interact and be shaped by environmental structure through progressively building the circuits underlying thinking, as we explore in more detail next.

4.4 Neural constructivism and evolution

Neural constructivism suggests an evolutionary view that contrasts with the view evolutionary psychologists propose. Evolutionary psychologists (e.g., Barkow et al. 1992; Pinker 1994) champion Chomsky's nativism. Viewing development as an intrinsic process, they sought a phylogenetic source for the innately specified skills comprising the human cognitive repertoire. Although Chomsky himself remained uncommitted to the claim that complex mental structures evolved on the basis of selective pressures, this has become the banner of evolutionary psychology. Currently, it is popular to regard not only cognitive skills but also social and political attitudes as the expression of genetically coded responses to phylogenetic pressures acting over the course of evolutionary history.

Sometimes this view is inserted into a selectionist framework (e.g. Gazzaniga 1992). Selectionism, however, is strictly incompatible with the claim that evolutionary pressures have picked out specialized circuits. According to selectionism (e.g., Edelman 1987), selective pressures operate on epigenetic variation during the ontogeny of the individual (in "somatic" time), not on a phylogenetic timescale. Pinker (1994) is more consistent when he restates Roger Sperry's chemoaffinity hypothesis that address-encoding surface markers on individual cells underlie the connectivity of specialized circuits (see Figure 3). Unfortunately, neurobiologists abandoned this extreme view of neural specificity twenty-five years ago (see Easter et al. 1985 for a review). The recent comparative analysis of Finlay & Darlington (1995) lend further support to the view that the brain is not a hodgepodge of specialized circuits, each chosen by evolutionary pressures. A major challenge for evolutionary psychologists, then, is to show how their view satisfies neurobiological constraints.

According to neural constructivism, to see human cognitive evolution as the progressive increase in specialized structures is to misinterpret cortical evolution. The hallmark of cortical evolution is not the ever-increasing sophistication of dedicated or specialized cortical circuitry (Gazzaniga 1995) but an increasing representational flexibility that allows environmental factors to shape the human brain's structure and function. At the expense of increased vulnerability during a protracted developmental period, constructive learning allows alterations in the learner's environment either through natural or human innovation to be accommodated by flexible representations without such changes being mediated by intrinsic mechanisms of specification. Human development accordingly consists of two processes, first a prolonged period of representation construction in which neural structures respond to the informational structure of the environment, and, second, rapid learning, made possible by the first.

5 Conclusions

The extent and duration of large-scale brain changes during development has also been underappreciated. Whereas many researchers believe that the major events in brain development end by two years of age, the evidence we have reviewed illustrates these continue well past the first decade of life. Rather than being strictly reductive, neural constructivism points to the interaction between cognitive and neural processes in development, suggesting that cognitive and neural levels of description will need to be integrated into a single explanatory framework to explain this prolonged growth. Neural constructivism thus provides a meeting ground for cognitive scientists and neuroscientists. Although we are only beginning to understand how the world and brain interact to build the mind, the story that is unfolding already makes nativist theories appear implausible. What lies ahead promises to be an exciting---and far richer---account in which the mind emerges from a prolonged interaction with a structured world.

Acknowledgments

Antonini, A. & Stryker, M.P. (1993) Development of individual geniculocortical arbors in cat striate cortex and effects of binocular impulse blockade. Journal of Neuroscience 13:3549-73.

Azimi-Sadjadi, M.R., Sheedvash, S. & Trujillo, F.O. (1993) Recursive dynamic node creation in multilayer neural networks. IEEE Transactions on Neural Networks 4:242-56.

Bailey, C.H. & Chen, M. (1988a) Long-term sensitization in Aplysia increases the number of presynaptic contacts onto the identified gill motor neuron L7. Proceedings of the National Academy of Sciences of the United States of America 85:9356-9.

Bailey C.H. & Chen, M. (1988b) Morphological basis of short-term habituation in Aplysia. Journal of Neuroscience 8:2452-9.

Balazs, R., Hack, N., Jorgensen, O. S. & Cotman, C. W. (1989) N-methyl-D-aspartate promotes the survival of cerebellar granule cells: pharmacological characterization. Neuroscience Letters 101:241-6.

Barkow, J.H., Cosmides, L. & Tooby, J. (eds). (1992) The adapted mind: Evolutionary psychology and the generation of culture. Oxford University Press.

Barron, A.R. (1994) Approximation and estimation bounds for artificial neural networks. Machine Learning 14:115-33.

Bates, E.A. & Elman, J.L.(1993) Connectionism and the study of change. In: Brain development and cognition: A reader, ed. M.H. Johnson. Blackwell.

Bates, E.A. & MacWhinney, B. (1987). Competition, variation, and language learning. In: Mechanisms of language aquisition, ed. Brian MacWhinney. Lawrence Erlbaum Associates.

Baum, E.B. (1988) Complete representations for learning from examples. In: Complexity in Information Theory, ed. Abu-Mostafa. Springer-Verlag.

Baum, E.B. (1989) A proposal for more powerful learning algorithms. Neural Computation 1:201-207.

Becker, L.E., Armstrong, D.L. Chan, F. & Wood, M.M. (1984) Dendritic development in human occipital cortical neurons. Developmental Brain Research 13:117-124.

Bennett, M.R. & Pettigrew, A.G. (1974) The formation of synapses in striated muscle during development. Journal of Physiology (London) 241:515-545.

Bernardo, L.S., Masukawa, L.M. & Prince, D. A. (1982) Electrophysiology of isolated hippocampal pyramidal dendrites. Journal of Neuroscience 2:1614-1622.

Black, J. E., Isaacs, K.R., Anderson B.J., Alcantara, A.A., & Greenough, W.T. (1990) Learning causes synaptogenesis, whereas motor activity causes angiogenesis, in cerebellar cortex of adult rats. Proceedings of the National Academy of Sciences of the United States of America 87:5568-72.

Blum, A. & Rivest, R.L. (1988) Training a 3-node neural network is NP-complete. In: Advances in Neural Information Processing Systems, ed. D.S. Touretzky. Morgan Kaufmann.

Blumer, A., Ehrenfeucht, A., Haussler, D. & Warmuth, M. (1988) Learnability and the Vapnik-Chervonenkis dimension. UCSC-CRL-87-20.

Bohme, G.A., Bon, C., Stutzmann, J.M., Doble, A. & Blanchard, J.C. (1991) Possible involvement of nitric oxide in long-term potentiation. European Journal of Pharmacology 199:379-81.

Boothe, R. G., Greenough, W. T., Lund, J. S., & Wrege, K. (1979) A quantitative investigation of spine and dendrite development of neurons in visual cortex (area 17) of Macaca nemestrina monkeys. Journal of Comparative Neurology 186:473-89.

Borges, S. & Berry, M. (1976) Preferential orientation of stellate cell dendrites in the visual cortex of the dark-reared rat. Brain Research 112:141-7.

Borges, S. & Berry, M. (1978) The effects of dark-rearing on the development of the visual cortex of the cat. Brain Research 180:277-300.

Bourgeois, J.P., Goldman-Rakic, P.S. & Rakic, P. (1994) Synaptogenesis in the prefrontal cortex of rhesus monkeys. Cerebral Cortex 4:78-96.

Bourgeois, J.P., Jastreboff, P.J. & Rakic, P. (1989) Synaptogenesis in visual cortex of normal and preterm monkeys: evidence for intrinsic regulation of synaptic overproduction. Proceedings of the National Academy of Sciences of the United States of America 86:4297-301.

Brewer, G.J. & Cotman, C.W. (1989) NMDA receptor regulation of neuronal morphology in cultured hippocampal neurons. Neuroscience Letters 99:268-73.

Brown, R. (1973) A first language: the early stages. Harvard Univeristy Press.

Buell, S.J. & Coleman, P.D. (1981) Quantitative evidence for selective dendritic growth in normal human aging but not in senile dementia. Brain Research 214:23-41.

Bulloch, A.G. & Hauser, G.C. (1990) Sprouting by isolated Helisoma neurons: enhancement by glutamate. International Journal of Developmental Neuroscience 8:391-8.

Callaway, E.M. & Katz, L.C. (1990) Emergence and refinement of clustered horizontal connections in cat striate cortex. Journal of Neuroscience 10:1134-1153.

Callaway, E.M. & Katz, L.C. (1991) Effects of binocular deprivation on the development of clustered horizontal connections in cat striate cortex. Proceedings of the National Academy of Sciences of the United States of America 88:745-749.

Callaway, E.M. & Katz, L.C. (1992) Development of axonal arbors of layer 4 spiny neurons in cat striate cortex. Journal of Neuroscience 12:570-82.

Camel, J.E., Withers, G.S. & Greenough, W.T. (1986) Persistence of visual cortex dendritic alterations induced by postweaning exposure to a "superenriched" environment in rats. Behavioral Neuroscience 100:810-3.

Caviness, V.S. & Rakic, P. (1978) Mechanisms of cortical development: a view from mutations in mice. In: Annual Review of Neuroscience, eds. W.M. Cowan, Z.W. Hall, & E.R. Kandel. Annual Reviews.

Chang, F.L. & Greenough, W.T. (1984) Transient and enduring morphological correlates of synaptic efficacy change in the rat hippocampal slice. Brain Research 309:35-46.

Chang P.L., Isaacs, K.R. & Greenough, W.T. (1991) Synapse formation occurs in association with the induction of long-term potentiation in two-year-old rat hippocampus in vitro. Neurobiology of Aging 12:517-22.

Changeux, J.P. & Danchin, A. (1976) Selective stabilisation of developing synapses as a mechanism for the specification of neuronal networks. Nature 264:705-712.

Changeux. J.P. & Dehaene, S. (1989) Neuronal models of cognitive functions. Cognition 33:63-109.

Chomsky, N. (1965) Aspects of the theory of syntax. MIT Press.

Chomsky, N. (1980) Rules and representations. Behavioral and Brain Sciences 3:1-61.

Churchland, P.S. & Sejnowski, T.J. (1988) Perspectives on Cognitive Neuroscience. Science 242:741-745.

Churchland, P. S. & Sejnowski, T. J. (1992) The Computational Brain. MIT Press.

Cline, H.T. (1991) Activity-dependent plasticity in the visual systems of frogs and fish. Trends in Neurosciences 14:104-111.

Coggeshall, R.E. (1992) A consideration of neural counting methods. Trends in Neurosciences 15:9-13.

Coggeshall, R.E., & Lekan, H.A. (1996) Methods for determining numbers of cells and synapses: A case for more uniform standards of reviews. Journal of Comparative Neurology 364:6-15.

Coleman, P.D., Flood, D.G., Whitehead, M.C. & Emerson, R.C. (1981) Spatial sampling by dendritic trees in visual cortex. Brain Research 214:1-21.

Conlee, J. W. & Parks, T.N. (1983) Late appearance and deprivation-sensitive growth of permanent dendrites in the avian cochlear nucleus (Nuc. Magnocellularis). Journal of Comparative Neurology 217:216-226.

Constantine-Paton, M. & Law, M.I. (1978) Eye-specific termination bands in tecta of three-eyed frogs. Science 202:639-41.

Cragg, B.G. (1975) The development of synapses in kitten visual cortex during visual deprivation. Experimental Neurology 46:445-451.

Cybenko, G. (1989) Approximation by superpositions of a sigmoid function. Mathematics of Control, Signals, and Systems 2:303-314.

Dailey, M.E., & Smith, S.J. (1996) The dynamics of dendritic structure in developing hippocampal slices. Journal of Neuroscience 16:2983-2994.

Dekaban, A.S., & Sadowsky, D. (1978) Changes in brain weights during the span of human life: relation of brain weights to body heights and body weights. Annals of Neurology 4:345-356.

Dietterich, T.G. (1990) Machine learning. Annual Review of Computer Science 4:255-306.

Durbin, R. & Mitchison, G.J. (1990) A dimension reduction framework for understanding cortical maps. Nature 343:644-7.

Durbin, R. & Rumelhart, D.E. (1989) Product units: A computationally powerful and biologically powerful extension to backpropagation networks. Neural Computation 1:133-142.

Easter, S.S., Jr., Purves, D., Rakic, P. & Spitzer, N.C. (1985) The changing view of neural specificity. Science 230:507-11.

Edelman, G. (1987) Neural Darwinism: The Theory of neuronal group selection. Basic Books.

Elman, J.L., Bates, E.A., Johnson, M.H., Karmiloff-Smith, A., Parisi, D. & Plunkett, K. (in press, 1996) Rethinking innateness: A connectionist perspective on development. MIT Press.

Fahlman, S.E. & Lebiere, C. (1990) The cascade-correlation architecture. In: Advances in Neural Information Processing Systems, ed. D.S. Touretzky. Morgan Kaufmann.

Feldman, J.A. & Ballard, D.H. (1982) Connectionist models and their properties. Cognitive Science 6:205-254.

Fiala, B.A., Joyce, J.N. & Greenough, W.T. (1978) Environmental complexity modulates growth of granule cell dendrites in developing but not adult hippocampus of rats. Experimental Neurology 59:372-83.

Finlay, B.L., & Darlington, R.B. (1995) Linked regularities in the development and evolution of mammalian brains. Science 268:1578-84.

Fischer, K.W. (1980) A theory of cognitive development: The control and construction of hierarchies of skills. Psychological Review 87:477- 531.

Flood, D.G. & Coleman, P.D. (1979) Demonstration of orientation columns with [14C] 2-deoxyglucose in a cat reared in a striped environment. Brain Research 173:538-542.

Fodor, J. (1983) The Modularity of mind. Bradford Books.

Frean, M. (1990) The upstart algorithm: a method for constructing and training feedforward neural networks. Neural Computation 2:198-209.

Friedlander, M.J. & Martin, K.A.C. (1989) Development of Y axon innervation of cortical area 18 in the cat. Journal of Physiology (London) 416:183-213.

Friedlander, M.J., Martin, K.A. & Vahle, H.C. (1985) The structure of the terminal arborizations of physiologically identified retinal ganglion cell Y axons in the kitten. Journal of Physiology (London) 359:293-313.

Friedlander, M.J., Martin, K.A.C. & Wassenhove-McCarthy, D. (1991) Effects of monocular visual deprivation on geniculocortical innervaton of area 18 in cat. Journal of Neuroscience 11:3268-3288.

Frost, D.O.(1982) Anomalous visual connections to somatosensory and auditory systems following brain lesions in early life. Brain Research 255:627-35.

Gally J.A., Montague, P.R., Reeke, G.N. Jr & Edelman, G.M. (1990) The NO hypothesis: possible effects of a short-lived, rapidly diffusible signal in the development and function of the nervous system. Proceedings of the National Academy of Sciences of the United States of America 87:3547-51.

Gazzaniga, M.S. (1995) On neural circuits and cognition. Neural Computation 7:1-13.

Gazzaniga, M.S. (1992) Nature's mind. Basic Books.

Geman, S., Bienenstock, E. & Doursat, R. (1992) Neural networks and the bias/variance dilemma. Neural Computation 4:1-58.

Gibson, K.R. (1990) New perspectives on instincts and intelligence: Brain size and the emergence of hierarchical mental constructional skills. In: "Language" and intelligence in monkeys and apes: Comparative developmental perspectives, eds. S.T. Parker & K.R. Gibson. Cambridge University Press.

Girosi, F. & Poggio, T. (1990) Networks and the best approximation property. Biological Cybernetics 63:169-76.

Globus, A., Rosenzweig, M.R., Bennett, E.L. & Diamond, M.C. (1973) Effects of differential experience on dendritic spine counts in rat cerebral cortex. Journal of Comparative and Physiological Psychology 82:175-81.

Globus, A., & Scheibel, A.B. (1967) The effect of visual deprivation on cortical neurons: a golgi study. Experimental Neurology 19:331-245.

Gold, E.M. (1967) Language identification in the limit. Information and Control 10:447-474.

Goldreich, O., Goldwasser, S. & Micali, S. (1984) How to construct random functions. Journal for the Association of Computing Machinery 33:792-807.

Goodhill, G.J. (1992) Correlations, Competition and Optimality: Modelling the Development of Topography and Ocular Dominance. Cognitive Science Research Paper 226, University of Sussex.

Gottlieb, G. (1976) Conceptions of prenatal development: behavioral embryology. Psychological Review 83: 215-34.

Gottlieb, G. (1991) Experiential canalization of behavioral development: Theory. Developmental Psychology 27:4-13.

Greenough, W.T., Black, J.E. & Wallace, C.S. (1987) Experience and brain development. Child Development 58:539-559.

Greenough, W.T. & Chang, F.L. (1988) Dendritic pattern formation involves both oriented regression and oriented growth in the barrels of mouse somatosensory cortex. Brain Research 471:148-152.

Greenough, W.T. & Volkmar, F.R. (1973) Pattern of dendritic branching in occipital cortex of rats reared in complex environments. Experimental Neurology 40:491-504.

Haley, J.E., Wilcox, G.L. & Chapman, P.F. (1992) The role of nitric oxide in hippocampal long-term potentiation. Neuron 8:211-6.

Harris, R.M & Woolsey, T.A. (1981) Dendritic plasticity in mouse barrel cortex following postnatal vibrissa follicle damage. Journal of Comparative Neurology 196:357-37.

Haussler, D. (1989) Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence 36:177-222.

Hebb, D.O. (1949) The organization of behavior: A neuropsychological theory. John Wiley and Sons.

Herrmann, K. & Shatz, C.J. (1995) Blockade of action potential activity alters initial arborization of thalamic axons within cortical layer 4. Proceedings of the National Academy of Sciences 92:11244-8.

Hirose, Y., Yamashita, K. & Hijiya, S. (1991) Back-propagation algorithm which varies the number of hidden units. Neural Networks 4:61-6.

Hirschfeld, L.A. & Gelman, S.A. (eds). (1994) Mapping the mind: Domain specificity in cognition and culture. Cambridge University Press.

Hornik, K., Stinchcombe, M. & White, H. (1989) Multilayer feedforward networks are universal approximations. Neural Networks 2:359-366.

Hubel, D. & Wiesel, T. (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. Journal of Physiology (London) 160:106-154 .

Hubel, D. & Wiesel, T. (1963) Receptive fields of cells in striate cortex of very young, visually inexperienced kittens. Journal of Neurophysiology 26:994-1002.

Hubel, D. & Wiesel, T. (1965) Binocular Interactions in striate cortex of kittens reared with artificial squint. Journal of Neurophysiology 28:1041-1059.

Hubel D. & Wiesel, T. (1972) Laminar and columnar distribution of geniculo-cortical fibers in the macaque monkey. Journal of Comparative Neurology 146:421-50.

Humphrey, A.L., Sur, M., Uhlrich, D.J. & Sherman, S.M. (1985) Projection patterns of individual X- and Y-cell axons from the lateral geniculate nucleus to cortical area 17 in the cat. Journal of Comparative Neurology 233:159-89.

Huttenlocher, P.R. (1979) Synaptic density in human frontal cortex-developmental changes and effects of aging. Brain Research 163:195-205.

Huttenlocher, P.R. (1990) Morphometric Study of human cerebral cortex development. Neuropsychologia 28:517-527.

Huttenlocher, P.R. & de Courten, C. (1987) The development of synapses in striate cortex of man. Human Neurobiology 6:1-9.

Huttenlocher P.R., de Courten, C., Garey, L.J. & Van der Loos, H. (1982) Synaptogenesis in human visual cortex: evidence for synapse elimination during normal development. Neuroscience Letters 13:247-52.

Jacobs, B., Schall, M. & Scheibel, A.B. (1993) A quantitative dendritic analysis of Wernicke's area in humans. II. Gender, hemispheric, and environmental factors. Journal of Comparative Neurology 327:97-111.

Jaslove, S.W. (1992) The integrative properties of spiny distal dendrites. Neuroscience 47:495-519

Jerne, N. (1967) Antibodies and learning: Selection versus instruction. In: The Neurosciences: A Study Program, ed. G.C. Quarton, T. Melnechuk, F.O. Schmitt. Rockefeller University Press.

Jernigan, T.L., Archibald, S.L., Berhow, M.T., Sowell, E.R., Foster, D.S. & Hesselink, J.R. (1991) Cerebral structure on MRI, Part I: Localization of age-related changes. Biological Psychiatry 29:55-67.

Johnson, M.H. (1990) Cortical maturation and the development of visual attention in early infancy. Journal of Cognitive Neuroscience 2:81-95.

Jones, O.T., Kunze, D.L. & Angelides, K.J. (1989) Localization and mobility of omega-conotoxin-sensitive Ca2+ channels in hippocampal CA1 neurons. Science 244:1189-93.

Judd, S. (1988) On the complexity of loading shallow neural networks. Journal of Complexity 4:177-192.

Juraska, J.M. & Fifkova, E. (1979) A Golgi study of the early postnatal development of the visual cortex of the hooded rat. Journal of Comparative Neurology 183:247-56.

Juraska, J.M., Greenough, W.T., Elliott, C., Mack, K.J. & Berkowitz, R. (1980) Plasticity in adult rat visual cortex: an examination of several cell populations after differential rearing. Behavioral and Neural Biology 29:157-67.

Kadirkamanathan, V. & Niranjan, M. (1993) A function estimation approach to sequential learning with neural networks. Neural Computation 5:954-75.

Kalil, R.E., Dubin, M.W., Scott, G. & Stark, L.A. (1986) Elimination of action potentials blocks the structural development of retinogeniculate synapses. Nature 323:156-8.

Karmiloff-Smith, A. (1992) Beyond modularity: A developmental perspective on cognitive science. MIT Press.

Kasper, E.M., Larkman, A.U., Lubke, J., & Blakemore, C. (1994) Pyramidal neurons in layer 5 of the rat visual cortex. II. Development of electrophysiological properties. Journal of Comparative Neurology 339:475-94.

Katz, L.C. & Constantine-Paton, M. (1988) Relationships between segregated afferents and postsynaptic neurones in the optic tectum of three-eyed frogs. Journal of Neuroscience 8:3160-80.

Katz, L.C., Gilbert, C. D. & Wiesel, T.N. (1989) Local circuits and ocular dominance columns in monkey striate cortex. Journal of Neuroscience 9:1389-1399.

Katz, M.J., George, E.B. & Gilbert, L.J. (1984) Axonal elongation as a stochastic walk. Cell Motility 4:351-370.

Kelso, S.R., Ganong, A.H. & Brown, T.H. (1986) Hebbian synapses in hippocampus. Proceedings of the National Academy of Sciences of the United States of America 83:5326-30.

Koch, C. & Poggio, T. (1992) Multiplying with synapses and neurons. In: Single Neuron Computation, eds. T. McKenna, J. Davis, S. Zornetzer. Academic Press.

Koch, C., Poggio, T. & Torre, V. (1982) Retinal ganglion cells: a functional interpretation of dendritic morphology. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences 298:227-63.

Koch, C., Poggio, T. & Torre, V. (1983) Nonlinear interactions in a dendritic tree: localization, timing, and role in information processing. Proceedings of the National Academy of Sciences of the United States of America 80:2799-802.

Koester, S.E. & O'Leary, D.D. (1992) Functional classes of cortical projection neurons develop dendritic distinctions by class-specific sculpting of an early common pattern. Journal of Neuroscience 12:1382-93.

Kossel, A., Lowel, S. & Bolz, J. (1995) Relationships between dendritic fields and functional architecture in striate cortex of normal and visually deprived cats. Journal of Neuroscience 15:3913-3926.

Lee, K.S., Schottler, F. Oliver, F. & Lynch, G. (1980) Brief bursts of high-frequency stimulation produce two types of structural change in rat hippocampus. Journal of Neurophysiology 44:247-258.

Lenneberg, E.H. (1967) Biological Foundations of Language. Wiley.

LeVay, S. & Stryker, M.P. (1979). The development of ocular dominance columns in the cat. In: Society for Neuroscience Symposium: Aspects of Developmental Neurobiology, ed. J.A. Ferrendelli. Society for Neuroscience.

LeVay, S., Wiesel, T.N. & Hubel, D.H. (1980) The development of ocular dominance columns in normal and visually deprived monkeys. Journal of Comparative Neurology 191:1-51.

Lichtman, J.W. (1977) The reorganization of synaptic connexions in the rat submandibular ganglion during post-natal development. Journal of Physiology (London) 320:121-130.

Lightfoot, D. (1989) The child's trigger experience: Degree-0 learnability. Behavioral & Brain Sciences 12:321-375.

Lightfoot, D. (1991) How to set parameters: arguments from language change. MIT Press.

Lubke, J. & Albus, K. (1989) The postnatal development of layer VI pyramidal neurons in the cat's striate cortex, as visualized by intracellular Lucifer yellow injections in aldehyde-fixed tissue. Brain Research. Developmental Brain Research 45:29-38.

Lund, J.S. & Holbach, S.M. (1990a) Postnatal development of thalamic recipient neurons in the monkey striate cortex: I. Comparison of spine acquisition and dendritic growth of layer 4C alpha and beta spiny stellate neurons. Journal of Comparative Neurology 309:115-28.

Lund, J.S., Holbach, S.M., & Chung, W.W. (1990b) Postnatal development of thalamic recipient neurons in the monkey striate cortex: II. Influence of afferent driving on spine acquisition and dendritic growth of layer 4C spiny stellate neurons. Journal of Comparative Neurology 309:129-40.

Macnamara, J. (1982) Names for things: a study of child language. MIT Press.

Maffei, L. & Galli-Resta, L. (1990) Correlation in the discharges of neighboring rat retial ganglion cells during prenatal life. Proceedings of the National Academy of Sciences 87:2861-2864.

Mainen, Z.F., Joerges, J., Huguenard, J.R. & Sejnowski, T.J. (1995) A model of spike initiation in neocortical pyramidal neurons. Neuron 15:1427-1439.

Manzoni, O., Prezeau, L., Marin, P., Deshager, S., Bockaert, J. & Fagni L. (1992) Nitric oxide-induced blockade of NMDA receptors. Neuron 8:653-62.

Mariani, J., Crepel, F., Mikoshiba, K., Changeux, J.P. & Sotelo, C. (1977) Anatomical, physiological and biochemical studies of the cerebellum from Reeler mutant mouse. Philosophical Transactions of the Royal Society of London, Series B 281:1-28.

Mathers, L.J. (1979) Postnatal dendritic development in the rabbit visual cortex. Brain Research 168:21-9.

McCasland, J.S., Bernardo, K.L., Probst, K. & Woolsey, T.A. (1992) Cortical local circuit axons do not mature after early deafferentation. Proceedings of the National Academy of Sciences of the United States of America 89:1832-1836.

Mead, C. (1989) Analog VLSI and neural systems. Addison-Wesley.

Mehler, J. (1985) Language related dispositions in early infancy. In: Neonate cognition: beyond the blooming buzzing confusion, ed. J. Mehler & R. Fox. L. Erlbaum Associates.

Meister, M., Wong, R., Baylor, D. & Shatz, C.J. (1991) Synchronous bursts of action potentials in ganglion cells of the developing mammalian retina. Science 252:939-943.

Mel, B.W. (1992a) NMDA-based pattern discrimination in a modeled cortical neuron. Neural Computation 4:502-17.

Mel, B.W. (1992b) Information processing in an excitable dendritic tree. CNS Memo 17, Computational and Neural Systems Program, California Institute of Technology.

Mel, B.W. (1994) Information processing in dendritic trees. Neural Computation 6:1031-85.

Mel, B.W. & Koch, C. (1990) Sigma-Pi learning: On radial basis functions and cortical associative learning. In: Advances in Neural Information Processing Systems, ed. D.S. Touretzky. Morgan Kaufmann.

Miller, K.D., Keller, J.B. & Stryker, M.P. (1989) Ocular dominance column development: Analysis and simulation. Science 245:605-615.

Mitchison, G.J. & Durbin, R. (1986) Optimal numberings of an N X N array. S.I.A.M. Journal on Algebraic and Discrete Methods 7:571-582.

Montague, P.R. (1996) The resource consumption principle-attention and memory in volumes of neural tissue. Proceedings of the National Academy of Sciences 93:3619-3623.

Montague, P.R., Gally, J.A. & Edelman, G.M. (1991) Spatial signaling in the development and function of neural connections. Cerebral Cortex 1:199-220.

Montague, P.R., Gancayco, C.D., Winn, M.J., Marchase, R.B. & Friedlander, M.J. (1994) Role of NO production in NMDA receptor-mediated neurotransmitter release in cerebral cortex. Science 263:973-7.

Montague, P.R. & Sejnowski, T.J. (1994) The predictive brain: temporal coincidence and temporal order in synaptic learning mechanisms. Learning and Memory 1:1-33.

Mooney, R.D., Nikoletseas, M.M., King, T.D., Savage, S.V., Weaver, M.T. & Rhoades, R. W. (1992) Structural and functional consequences of neonatal deafferentation in the superficial layers of the hamster's superior colliculus. Journal of Comparative Neurology 315:398-412.

Moore, R.Y., & Bernstein, M.E. (1989) Synaptogenesis in the rat suprachiasmatic nucleus demonstrated by electron microscopy and synapsin I immunoreactivity. Journal of Neuroscience 9:2151-2162.

Natarajan, B. (1991) Machine learning: A theoretical approach. Morgan Kaufmann.

Neville, H. (1991) Neurobiology of cognitive and language processing: effects of early experience. In: Brain maturation and cognitive development, eds. K.R. Gibson & A.C. Peterson. Aladine de Gruyter Press.

Noback, C.R., & Purpura, D.P. (1961) Postnatal ontogenesis of neurons in cat neocortex. Journal of Comparative Neurology 171:291-308.

O'Dell T.J., Hawkins, R.D., Kandel, E.R. & Arancio O. (1991) Tests of the roles of two diffusible substances in long-term potentiation: evidence for nitric oxide as a possible early retrograde messenger. Proceedings of the National Academy of Sciences of the United States of America 88:11285-9.

O'Kusky, J. & Colonnier, M. (1982a) Postnatal changes in the number of neurons and synapses in the visual cortex (area 17) of the macague monkey: a stereological analysis in normal and monocularly deprived animals. Journal of Comparative Neurology 210:307-315

O'Kusky, J. & Colonnier, M. (1982b) Postnatal changes in the number of astrocytes, oligodendrocytes, and microglia in the visual cortex (area 17) of the macague monkey: a stereological analysis in normal and monocularly deprived animals. Journal of Comparative Neurology 210:307-315.

O'Leary, D.D.M. (1990) Do cortical areas emerge from a protocortex? Trends in Neurosciences 12:400-406

O'Leary, D.D.M. (1992) Development of connectional diversity and specificity in the mammalian brain by the pruning of collateral projections. Current Opinion in Neurobiology 2:70-7.

O'Leary, D.D.M., Schlaggar, B.L. & Stanfield, B.B. (1992) The specification of sensory cortex: lessons from cortical transplantation. Expermental Neurology 115:121-126.

O'Rourke, N.A., Cline, H.T. & Fraser, S.E. (1994) Rapid remodeling of retinal arbors in the tectum with and without blockade of synaptic transmission. Neuron 12:921-34.

O'Rourke, N.A. & Fraser, S.E. (1986) Dynamic aspects of retinotectal map formation revealed by a vital-dye fiber-tracing technique. Developmental Biology 114:265-76.

O'Rourke, N.A. & Fraser, S.E. (1990) Dynamic changes in optic fiber terminal arbors lead to retinotopic map formation: an in vivo confocal microscopic study. Neuron 5:159-171.

Osherson, D.N., Stob, M. & Weinstein, S. (1986) Systems that Learn. MIT Press.

Pallas, S.L., Roe, A.W. & Sur, M. (1990) Visual projections induced into the auditory patheay of ferrets. . novel inputs to prmary audtory cortex (AI) from the LP/pulvinar complex and the topography of the MGN-AI projection. Journal of Comparative Neurology 298:50-68

Parnavelas, J.G. & Uylings, H.B. (1980) The growth of non-pyramidal neurons in the visual cortex of the rat: a morphometric study. Brain Research 193:373-82.

Pearce, I.A., Cambray-Deakin, M.A. & Burgoyne, R.D. (1987) Glutamate acting on NMDA receptors stimulates neurite outgrowth from cerebellar granule cells. Febs Letters 223:143-7.

Petit, T.L., LeBoutillier, J.C., Gregorio, A. & Libstug, H. (1988) The pattern of dendritic development in the cerebral cortex of the rat. Brain Research 469, 209-219.

Piattelli-Palmarini, M. (1989) Evolution, selection and cognition: from "learning" to parameter setting in biology and in the study of language. Cognition 31:1-44.

Pinker, S. (1979) Formal models of language learning. Cognition 1:217-283.

Pinker, S. (1994) The language instinct. W. Morrow and Co.

Pinker, S. (1984) Language learnability and language development. Harvard University Press.

Pinker, S. (1989) Language acquisition. In: Foundations of cognitive science, ed. M. Posner. MIT Press.

Pinto Lord, M.C. & Caviness, V.S. Jr. (1979) Determinants of cell shape and orientation: a comparative Golgi analysis of cell-axon interrelationships in the developing neocortex of normal and reeler mice. Journal of Comparative Neurology 187:49-69.

Platt, J.C. (1991) A resource-allocating network for function interpolation. Neural Computation 3:213-25.

Plunkett, K. & Sinha, C. (1992) Connectionism and developmental theory. British Journal of Developmental Psychology 10:209-254.

Pomeroy, S. L., LaMantia, A.S. & Purves, D. (1990) Postnatal construction of neural circuitry in the mouse olfactory bulb. Journal of Neuroscience 10:1952-66.

Purves, D. & Hadley, R.D. (1985) Changes in the dendritic branching of adult mammalian neurones revealed by repeated imging in situ. Nature 315:404-406.

Purves, D., Hadley, R.D. & Voyvodic, J.T. (1986) Dynamic changes in the dendritic geometry of individual neurons visualized over periods of up to three months in the superior cervical ganglion of living mice. Journal of Neuroscience 6:1051-1060.

Purves D. & Lichtman, J.W. (1985). Principles of Neural Development. Sinauer Associates.

Purves, D., Voyvodic, J., Magrassi, L. & Yawo, H. (1987) Nerve terminal remodeling visualized in living mice by repeated examination of the same neuron. Science 238:1122-1126.

Pylyshyn, Z. (1984) Computation and cognition: Toward a foundation for cognitive science. Bradford Books.

Quartz, S.R. (1993) Nativism, neural networks, and the plausibility of constructivism. Cognition 48:123-144.

Quartz, S.R. & Sejnowski, T.J. (1994) Beyond modularity: neural evidence for constructivist principles in development. Behavioral and Brain Sciences 17:725-726.

Rakic, P., Bourgeois, J. P., Eckenhoff, M.F., Zecevic, N., & Goldman-Rakic, P. S. (1986) Concurrent overproduction of Synapses in diverse regions of the primate cerebral cortex. Science 232:232-235.

Rakic, P., Bourgeois, J.P. & Goldman-Rakic, P.S. (1994) Synaptic development of the cerebral cortex: implications for learning, memory, and mental illness. Progress in Brain Research 102:227-43.

Rakic, P. & Sidman, R.L. (1973) Weaver mutant mouse cerebellum: defective neuronal migration secondary to abnormality of Bergmann glia. Proceedings of the National Academy of Sciences of the United States of America 70:240-4.

Rall, W. (1964) Theoretical significance of dendritic trees for neuronal input-output relations. In: Neural theory of modelling, ed. R.F. Reiss. Stanford University Press.

Redding, N.J., Kowalczyk, A. & Downs, T. (1993) Constructive higher-order network algorithm that is polynomial time. Neural Networks 6:997-1010.

Regehr, W.G., Connor, J.A. & Tank, D.W. (1989) Optical imaging of calcium accumulation in hippocampal pyramidal cells during synaptic activation. Nature 341:533-6.

Roe, A.W., Pallas, S.L., Hahm, J. & Sur, M. (1990) A map of visual space induced in primary auditory cortex. Science 250:818-820.

Roe, A. W., Pallas, S. L., Kwon, Y. H. & Sur, M. (1992) Visual projections routed to the auditory pathway in ferrets: receptive fields of visual neurons in primary auditory cortex. Journal of Neuroscience 12:3651-64.

Ruiz-Marcos, A. & Valverde, F. (1970) Dynamic architecture of the visual cortex. Brain Research 19:25-39.

Rumelhart, D., McClelland, J. & the PDP research group. (1986). Parallel distributed processing: Explorations in the microstructure of cognition. Cambridge, MA: Bradford Books.

Schade, J.P. & van Groenigen, W.B. (1961) Structural organization of the human cerebral cortex. I. Maturation of the middle frontal gyrus. Acta Anatomica 47:72-111.

Scheibel, A.B. (1993) Dendritic structure and language development. In: Developmental neurocogniton: Speech and face processing in the first year of life, ed. B. de Boysson-Bardies. Kluwer Academic Publishers.

Schilling, K., Dickinson, M.H., Connor, J.A. & Morgan, J.I. (1991) Electrical activity in cerebellar cultures determines Purkinje cell dendritic growth patterns. Neuron 7:891-902.

Schlaggar, B.L. & O'Leary, D.D.M. (1991) Potential of visual cortex to develop an array of functional units unque to somatosensory cortex. Science 252:1556-1560

Schuman, E.M. & Madison, DV. (1991) A requirement for the intercellular messenger nitric oxide in long-term potentiation. Science 254:1503-6.

Segev, I., Rinzel, J. & Shepherd, G.M. (1995). The theoretical foundations of dendritic function: Selected papers by Wilfrid Rall with commentaries. MIT Press.

Shatz, C.J. (1990) Impulse activity and the patterning of connections during CNS development. Neuron 5:745-756.

Shatz, C.J. (1992) How are specifc connectons formed between thalamus and cortex. Current Opinion in Neuorbiology 2:78-82.

Shatz C.J., Lindstrom, S. & Wiesel, T.N. (1977) The distribution of afferents representing the right and left eyes in the cat's visual cortex. Brain Research 131:103-16.

Shatz, C.J. & Stryker, M.P. (1978) Ocular dominance in layer IV of the cat's visual cortex and the effects of monocular deprivation. Journal of Physiology 281:267-83.

Shepherd, G.M. & Brayton, R.K. (1987) Logic operations are properties of computer-simulated interactions between excitable dendritic spines. Neuroscience 21:151-65.

Shin, Y. & Ghosh, J. (1995) Ridge polynomial networks. IEEE Transactions on Neural Networks 6:610-22.

Shoukimas, G.M. & Hinds, J.W. (1978) The development of the cerebral cortex in the embryonic mouse: an electron microscopic serial section analysis. Journal of Comparative Neurology 179:795-830.

Shultz, T.R., Mareschal, D. & Schmidt, W.C. (1994) Modeling cognitive development on balance scale phenomena. Machine Learning 16:57-86.

Siegler, R.S. (1989) Mechanisms of Cognitive Development. Annual Review of Psychology 40:353-379.

Simonds, R.J. & Scheibel, A.B. (1989) The postnatal development of the motor speech area: a preliminary study. Brain and Language 37:42-58.

Sperry, R.J. (1943) Effect of 180° rotation of the retinal fields on visuomotor coordination. Journal of Experimental Zoology 92:263-279.

Sperry, R. (1963) Chemoaffinity in the orderly growth of nerve fiber patterns and connections. Proceedings of the National Academy of Science 50:703-710.

Stanfield, B.B. & O'Leary, D.D. (1985) Fetal occipital cortical neurones transplanted to the rostral cortex can extend and maintain a pyramidal tract axon. Nature 313:135-7.

Stryker, M. (1991) Activty-dependent reorganization of afferents in the developing mammalian visual system. In: Development of the visual system, eds. D. Lam & C. Shatz. MIT Press.

Stuart, G.J. & Sakmann, B. (1994) Active propagation of somatic action potentials into neocortical pyramidal cell dendrites. Nature 367:69-72.

Sur, M., Humphrey, A.L. & Sherman, S.M. (1982) Monocular deprivation affects X- and Y-cell retinogeniculate terminations in cats. Nature 300:183-5.

Sur, M., Garraghty, P.E. & Roe, A.W. (1988) Expermentially induced visual projections into auditory thalamus and cortex. Science 242:1437-1441.

Sur, M., Pallas, S. L. & Roe, A. W. (1990) Cross-modal plasticity in cortical development: differentiation and specification of sensory neocortex. Trends in Neuroscience 13:227-33.

Swindale, N.V. (1980) A model for the formation of ocular dominance stripes. Proceedings of the Royal Society of London, B 208:243-264.

Tieman, S.B., & Hirsch, S. (1982) Exposure to lines of only one orientation modifies dendritic morphology of cells in the visual cortex of the cat. Journal of Comparative Neurology 211:353-362

Tooby, J. & Cosmides, L. (1992) The psychological foundations of culture. In: The adapted mind: Evolutionary psychology and the generation of culture, eds. J.H. Barkow, L. Cosmides & J. Tooby. Oxford University Press.

Turner, A.M. & Greenough, W.T. (1985) Differential rearing effects on rat visual cortex synapses. I. synaptic and neuronal density and synapses per neuron. Brain Research 329:195-203.

Uylings, H.B.M., Kuypers, K., Diamond, M.C. & Veltman, W.A.M. (1978) Effects of differential environments on plasticity of dendrites of cortical pyramidal neurons in adult rats. Experimental Neurology 62:658-677.

Uylings, H.B.M., Van Eden, C.G., Parnavelas, J.G. & Kalsbeek, A. (1990) The prenatal and postnatal development of the rat cerebral cortex. In: The cerebral cortex of the rat, ed. B. Kolb & R.C. Tees. MIT Press.

Valiant, L.G. (1984) A theory of the learnable. Communications of the ACM 27:1134-1142.

Valiant, L.G. (1991) A view of computational learning theory. In: Computation and cognition: Proceedings of the first NEC research symposium, ed. C.W. Gear. SIAM.

Valverde, F. (1967) Apical Dendritic Spines of the visual cortex and light deprivation in the mouse. Experimental Brain Research 3:337-352.

Valverde, F. (1968) Structural changes in the area striata of the mouse after enucleation. Experimental Brain Research 5:274-92.

Valverde, F. (1971) Rate and extent of recovery from dark rearing in the visual cortex of the mouse. Brain Research 33:1-11.

Vercelli, A., Assal, F. & Innocenti, G.M. (1992) Emergence of callosally projecting neurons with stellate morphology in the visual cortex of the kitten. Experimental Brain Research 90:346-358.

Volkmar, F.R. & Greenough, W.T. (1972) Rearing complexity affects branching of dendrites in the visual cortex of the rat. Science 176:1445-1447.

Wallace, C.S., Kilman, V.L., Withers, G.S. & Greenough, W.T. (1992) Increases in dendritic length in occipital cortex after 4 days of differential housing in weanling rats. Behavioral and Neural Biology 58:64-8.

Walsh, C. & Cepko, C.L. (1988) Clonally related cortical cells show several migration patterns. Science 241:1342-1345.

Walsh, C. & Cepko, C.L. (1992) Widespread dispersion of neuronal clones across functional regions of the cerebral cortex. Science 255:434-440.

Walsh, C. & Cepko, C.L. (1993) Clonal dispersion in proliferative layers of developing cerebral cortex. Nature 362:632-5

Wexler, K, & Culicover, P. (1980) Formal principles of language acquisition. MIT Press.

White, H. (1990) Connectionist nonparametric regression: multilayer feedforward networks can learn arbitrary mappings. Neural Networks 3:535-549.

Williams, C.V., Davenport, R.W., Dou, P., & Kater, S.B. (1995) Developmental regulation of plasticity along neurite shafts. Journal of Neurobiology 27:127-40.

Winfield, D.A. (1981) The postnatal development of synapses in the visual cortex of the cat and the effects of eyelid closure. Brain Research 206:166-171.

Wong, R.K., Prince, D.A. & Basbaum, A.I. (1979) Intradendritic recordings from hippocampal neurons. Proceedings of the National Academy of Sciences of the United States of America 76:986-90.

Wynne-Jones, M. (1993) Node splitting: a constructive algorithm for feed-forward neural networks. Neural Computing and Applications 1:17-22.

Zecevic, N., Bourgeois, J.P. & Rakic, P. (1989) Changes in synaptic density in motor cortex of rhesus monkey during fetal and postnatal life. Brain Research. Developmental Brain Research 50:11-32.

1. Friedlander et al. (1991) also found a number of cellular differences between the two groups of arbors suggesting that the observed shift in autoradiographic studies might be exaggerated. Depending on the pattern of incorporation of a radiolabelled tracer, it is hence possible that the non deprived arbor took up more tracer and, because of the relatively low resolution of autoradiography, obscured deprived arbors (see Friedlander et al. 1991, p.3285).

2. We should note that although we are emphasizing dendritic development, aspects of axonal development also satisfy these conditions. As it is from the interaction between dendrites and axons that the structure of the mature system emerges, this interaction must ultimately be characterized.

3. Koester & O'Leary (1992) report a significant retraction of layer V apical dendrites, but Kasper et al. (1994) report that these apical dendrites continue to grow and that the apparent retraction is due to the expansion of cortex.

4. Differences in the degree of rostral-caudal dendritic bias between normal and stripe-induced cells support the view that this development involves progressive growth rather than elimination of exuberant structure (see Katz & Constantine-Paton (1988, p. 3178)). The conclusion according to Katz et al. (1989, p.1393) is that, ``the pattern of afferent segregation has played a significant role in shaping the structure of the postsynaptic dendritic field of cortical neurons."

5. Axonal growth will fit into this account in the following sense. Local axonal growth may be sensitive to the development and stabilization of synapses, so that local axonal outgrowth may result from synapse formation. This would have the effect of putting more presynaptic structure into a local region in an activity-dependent manner, thereby increasing the probability of subsequent synapse formation in that region. The outgrowth of axonal projections, such as the development of horizontal connections (Callaway and Katz 1991), suggests that the elaboration of axon terminals at this fine level may proceed in this way.

6. Specificity is maintained by requiring that the presynaptic terminal should be coincidentally active (See Montague & Sejnowski (1994) for discussion).

7. A widely used metaphor to describe this process is that of error-correction. It should not be assumed, however, that the exuberant connections are strictly in error, since they may serve a useful purpose in instances in which a changes in connectivity is required (as in the case of blindness).