Arthur M. Glenberg
Most memory theories presuppose that memory is for memorizing. What would memory theory be like if this presupposition were discarded? Here, I approach memory theory guided by the question, "What is memory for?" The answer that I develop is influenced by three sources. The first is Lakoff and Johnson's (Johnson, 1987; Lakoff, 1987; Lakoff & Johnson, 1980) cognitive linguistic analysis of language, conceptualization, and meaning. They propose that cognitive structures are embodied; that they arise from bodily interactions with the world (cf. Harnad, 1990, 1993). After a brief review of the Lakoff and Johnson program, I examine the literature on memory (the second source) for evidence that cognitive structures are, indeed, embodied, and why that is so. I will propose that memory evolved in service of perception and action in a three-dimensional environment, and that memory is embodied to facilitate interaction with the environment. The third set of ideas comes from research on mental model theory of language comprehension. I relate mental model theory to the notion of embodied memory by proposing that because language acts as a surrogate for more direct interaction with the environment, language comprehension must also result in embodied representations, which are in fact mental models. In exploring these ideas, I develop an approach to memory and language comprehension that suggests ways of dealing with old problems (e.g., why recollection and comprehension are effortful), as well as new concepts to replace old ideas (e.g., an association).
Why should psychologists interested in language, learning, and memory care about issues such as embodiment of memory? Because by ignoring them, we have been making a big mistake. Most theories of memory treat internal representations as meaningless symbols such as a string of zeros and ones that "encode" features (e.g., Hintzman, 1986; McClelland and Rumelhart, 1986; Metcalfe, 1993), as point-like objects with no structure (Gillund and Shiffrin, 1984), or as propositions relating intrinsically meaningless symbols (Kintsch, 1988). Two problems arise from this treatment. The first is the symbol grounding problem (Harnad, 1990): How do those meaningless symbols come to take on meaning? The answer is not as simple as referring the symbol to a lexicon, because words in the lexicon must also be grounded. Also, not all of those meaningless symbols are meant to represent words or word-like concepts, but some are meant to represent complex non-verbal displays (Posner & Keele, 1968; Schacter, Cooper, & Delaney, 1990). The second problem is that we have not availed ourselves of a golden opportunity. By treating internal representation as meaningless symbols, we have not thought about the possibility of taking advantage of other forms of representation. Instead of meaningless symbols, suppose that representations have a structure that is lawfully related to the objects being represented. The structure of the representations might then play an important role in determining, for example, what concepts are easily associated because their structures literally fit together. For example, it seems easy to associate "horse" and "spotted" because horses have surfaces that can be spotted, whereas it is more difficult to associate "idea" and "spotted." Note that this sort of thinking trades on the analogical nature of the representations rather than on propositional listings of content (see Palmer, 1978). That is, we could just as easily assert "the idea was spotted" as "the horse was spotted." Nonetheless, one seems to make sense, the other does not.
In the next few sections I develop the case that internal representations are analogically structured (embodied), that this structure helps to explain memory phenomena, and that in conjunction these ideas suggest that the standard memory paradigms are ill- conceived and that standard memory phenomena may be revealing little that is important about memory. These sections are followed by a discussion the possibilities for analogical representation underlying language comprehension.
A central concern of the Lakoff and Johnson program is the concept of meaning. According to Lakoff (1987), the standard theory of meaning in cognitive science is based on the notion of truth values of propositions, and as it turns out, this theory will not work as a theory of human meaning. Explication of why this is so requires a bit of patience, in part because the way psychologists use the term "proposition" is different from the way philosophers and logicians use it. For the psychologist, propositions are relations among symbols, that is, an assertion that a relation exists. It is these assertions that are supposed to be meaningful. Importantly, although the propositions are supposed to capture meaning, the symbols used in the propositions are taken to be, by themselves, meaningless or arbitrary: There is no intrinsic relation between a particular symbol and its meaning Thus when illustrating propositions, a psychologist may use a word to stand for an element in the proposition, but that is just a convenience. Indeed, the meanings of the words need to be specified, presumably by other propositions. Thus, we should replace any words in a psychologist's proposition with things such as "symbol X19." This state of affairs is quite useful because it allows for reasoning (the derivation of new propositions) to be based on the manipulation of propositions by syntactic rules. These rules are thought to operate independent of the referents of the elements (nodes and symbols) in the propositions.
For example, suppose that proposition 1 (P1) asserts that element a is in relation R to element b. In shorthand, P1: aRb. Furthermore, suppose P2: bRc. Now, if R is a transitive relation (such as "larger than"), and both P1 and P2 are true, then by the syntactic rules of transitive inference, P3:aRc is also true. Thus, for the psychologist, we have created new knowledge, namely, that P3 is true. Note that these propositions have truth values, but they fail a common-sense test of what it means to have meaning. Namely, in order for a statement to be meaningful (to us), we must know what the statement is about. In contrast, although we know that P3 is true, we have little idea what it is about, because we have no idea what a and b stand for.
The problem of what a and b stand for is the symbol grounding problem (Harnad, 1990): How do we give meaning to the arbitrary symbols? To know what these propositions are about requires a mapping between the elements in the propositions (a, b, c, and R) and the world (or a possible world, or a model of the world). Without this mapping, the symbols can only refer to other symbols which in turn refer to yet other symbols. Just like trying to learn the meaning of a word in a completely foreign language by using a dictionary written solely in that language, such a system of symbols will never generate meaning (Searle, 1980). Most psychologists don't see a problem here, because they are happy to point to perception: The arbitrary symbols are grounded by the perceptual system. That is, what a symbol means is what it refers to in the "outside" world.
Lakoff (1987) presents (at least) three arguments against the plausibility of generating meaning by this sort of symbol grounding. First, this theory requires that categories be Aristotelian, that they have sharp boundaries. Aristotelian categories are needed so that we can successfully map between the arbitrary symbols in the propositions and the elements in the world. Thus, if a proposition is supposed to be about a horse, to give the proposition its proper meaning we must be able to map the symbol for horse (X19, perhaps) onto horses, and exclude zebras and antelopes and even perhaps ponies. In contrast to this requirement, there is a tremendous amount of empirical work in the psychology of human categorization implying that categories in the head are not Aristotelian. Instead many significant categories have fuzzy boundaries (Oden, 1984, 1987), graded membership (Kalish, 1995), complex structures (Lakoff, 1987), or are based on prototypes (Rosch,1973). Furthermore, the extensions of even basic biological categories are less than certain, and categories based on human culture are even more fuzzy. Thus, categories such as democracy, justice, and mother (Lakoff discusses biological mothers, birth mothers, adoptive mothers, step-mothers, etc.) seem to have structures quite different from the classical Aristotelian category.
A second argument against the standard theory as a theory of human meaning is based on an analysis of Putnam (1981). This analysis, however, is directed toward the philosopher's meaning of proposition, and so it requires a bit of new terminology. To the psychologist, a proposition (like aRb) is supposed to have meaning. To the philosopher, aRb is a sentence in a formal language. The meaning of the sentence (its propositional content) corresponds to the function that determines, for any possible situation, whether that sentence is true or false. In plain language, which is not an exact equivalent but close enough, the meaning of a sentence such as "the horse is spotted" is whatever allows one to determine if it applies to particular situations. Furthermore, two sentences have the same meaning if they have the same truth values for all possible situations.
Putnam discovered a serious problem with this truth-value notion of meaning: It is not difficult to construct pairs of formal sentences whose symbols are mapped to radically different things, but that have the same truth values in all situations. In other words, even though the sentences are about radically different things, on the truth-value account of meaning, because the sentences have the same truth values they are supposed to have the same meaning. Clearly, it does not make much sense to assert that sentences about different things mean the same thing. As it turns out, the problem is with the arbitrary nature of the symbols. They only mean when they are mapped onto the world, and Putnam demonstrated that it is impossible to find the one and only correct mapping.
Lakoff and Johnson's third argument against the standard theory is based on their analysis of language use and what it implies about cognition. In brief, people frequently use metaphorical language ("He's trapped in his marriage," "Your theory is airtight," "I'm really high today"). Furthermore, Lakoff and Johnson propose that metaphorical language is not just the way people talk, but that it accurately reflects the way people think. Given that theories cannot literally be airtight and that people's emotional states cannot literally be high, it is hard to imagine how cognition could be based on the mappings of arbitrary symbols and produce such (easy to understand) language.
Several other cogent arguments against the use of arbitrary symbols in a theory of meaning can be found in Barsalou (1993, Barsalou et al., 1993) and Shannon(1988). Barsalou and Shannon note that people have a hard time defining many familiar words, and that the definitions can vary greatly with context. This finding is difficult to explain if one believes that meaning of words is a simple list of well-formed propositions. They also note that there is no good account of how propositions composed of meaningless and arbitrary symbols might have evolved or how a child could have discovered them. As Shannon concludes,
"Specifically, it appears that the underlying substrate of mental activity is not a repertory of well-defined, well-structured abstract symbols, and that the workings of mind cannot be generally characterized as the computational manipulations of such symbols. Rather, the substrate in which mental activity takes place should be one that meets the following requirement: It should not be fixed by any coding system that is defined a priori, it should afford maximal sensitivity to unspecified dimensions and distinctions, it should be context-sensitive, and it should be embedded in the framework of the organism's action in the world" (page 80).
That is a call for an embodied approach to meaning.
If we dismiss the standard theory, what is left? Lakoff and Johnson offer a theory of meaning based on the concept of embodied knowledge. Because I will be approaching the problem from the question of "What is memory for," I will develop an idea of embodied meaning that is distinct from the Lakoff and Johnson proposal. Nonetheless, the proposals are clearly related. In outline, my proposal is that perceptual systems have evolved to facilitate our interactions with a real, three-dimensional world. To do this, the world is conceptualized (in part) as patterns of possible bodily interactions, that is, how we can move our hands and fingers, our legs and bodies, our eyes and ears, to deal with the world that presents itself. That is, to a particular person, the meaning of an object, event, or sentence is what that person can do with the object, event, or sentence.
How does this approach answer the objections raised to the standard theory of meaning? Importantly, embodied representations do not need to be mapped onto the world to become meaningful because they arise from the world. In other words, embodied representations are directly grounded by virtue of being lawfully and analogically related to properties of the world and how those properties are transduced by perceptual-action systems (Harnad, 1990, 1993). Thus, the meaningful, action-oriented, component of conceptualization is not abstract and amodal. It reflects how bodies of our sort can interact with objects.
Given that embodied representations do not need to be mapped onto the world to be grounded, there is no need for representations to be Aristotelian nor for the categories in the world to be Aristotelian. Furthermore, because embodied representations are not discrete, meaningless symbols, they can reflect subtle, fuzzy variations in the world. How then do categories arise? Objects fall into the same (basic) category because they can be used to accomplish the same interactive goal, such as supporting the body. Because the same object may be useful for accomplishing a variety of goals, categorization can be flexible and context dependent (Barsalou, 1993).
Consider three objections to these claims. The first is that because we have different bodies, we will understand the world in different ways. In fact, that is a valid prediction. For example, what makes an object a chair for a particular individual will depend on whether or not that individual is able to get his or her body into a sitting position using the object. Thus, depending on the height of the object, the width of the flat surface, the object's strength, etc., the object will be a chair for some people (e.g., a child) but not for others (e.g., an aging grandfather).# Nonetheless, our bodies are substantially the same around the world and across cultures. Thus, although there will be variability around the edges, our common human endowments and our common environment ensures a great degree of common center to cognitive structure.
A second objection to the claim that cognitive representation is embodied is that the mapping problem has not been solved; there is still the problem of mapping (arbitrary) words to embodied representations so that we can talk about what we are perceiving and thinking. This is a deep problem (e.g., Harnad, 1990; Plunkett, Sinha, Mxller, & Strandsby, 1992), but it is not one that I intend to address here. The point of the above is that embodied representations allow us to understand how, except for the seriously deranged, we all know the difference between say, horses and ideas, and contrary to what Putnam's analysis shows of the standard theory, we don't ever confuse them.
A third objection is that some things are meaningful (e.g., a beautiful sunset) even when there is no apparent possibility for bodily interaction.# The embodied account of meaning is situated, so that action-oriented meaning can vary greatly with context. Thus, depending on the context, a coke bottle can be used to quench thirst, as a weapon, a doorstop, or a vase. That is, its meaning depends on the context. Similarly, a beautiful sunset is a context that combines with objects and memories to suggest actions consistent with warmth, relaxation, and a good beer.
Later I will discuss how embodied representations can be extended to represent abstract concepts and how they may provide a novel way of dealing with unanalyzed concepts such as association. For now, however, I turn to developing a particular sketch of embodied representations that arises from a consideration of what memory is for. This sketch is not a fully testable theory. The idea is to show how a type of theory that is not subject to the criticisms leveled at meaningless symbol theory can handle problems of memory and comprehension.
Except for the recent blossoming of interest in indirect memory (see section 5.1), the contemporary psychology of memory has been dominated by the study of memorization.# In part, this seems to have arisen from a failure of many twentieth century memory theorists to consider what memory is for. By the end of section 5 I will have concluded the following. Memory is embodied by encoding meshed (i.e., integrated by virtue of their analogical shapes) sets of patterns of action. How the patterns combine is constrained by how our bodies work. A meshed set of patterns corresponds to a conceptualization. Updating memory occurs whenever the meshed patterns change (a change in conceptualization of the environment), and the updating is in terms of a change, or movement, or trajectory toward a new set of meshed patterns. Thus, memory records how conceptualizations blend into one another. This memory works in two broad modes. First, patterns of action based on the environment (projectable properties of the environment) are automatically, that is, without intention, meshed with patterns based on previous experience. This automatic use of memory corresponds closely to implicit or indirect memory. Second, patterns from the environment can be suppressed so that conceptualization is guided by previous experience encoded as trajectories. This is a conscious and effortful use of memory. The ability to suppress environmental patterns contributes to prediction, the experience of remembering, and language comprehension.
We live in a dangerous, three-dimensional world. Given the size, density, and physical capabilities of our bodies, the natural environment is hostile. We are open to predation, and our interactions with the world can lead to injury from freezing, burning, drowning, and falling. Clearly, survival requires the capability to navigate this environment, and just as clearly, our perceptual system has evolved to do just that. For example, we have developed impressive abilities to use information (e.g., optical flow fields) to guide action so that obstacles are avoided. These abilities may not require any sort of representation of the environment, and they may not require memory; responding constrained by characteristics of the environment and our bodies guarantees successful action (for a review, see Bruce & Green, 1985).
On the other hand, it is frequently the case that we need to differentiate. In addition to avoiding obstacles in our path, we need to pick out and follow a particular path, avoid a particular location, or approach a particular person. This sort of differentiation requires a memory system. What makes one person a particular person (to you) or one path the path to your house, is its relevance to you, that is, how you have interacted with it in the past. An optical flow field cannot contain this information; it is the province of memory. This distinction is discussed by Epstein (1993) who uses the term "projectable" to refer to properties of the environment that can be specified by information available in the light and "nonprojectable" to refer to properties that must be signaled by other sources. Thus spatial layout is a projectable property whereas ownership is a non- projectable property that must come from experience.
To support action, the perception of projectable properties is in terms of patterns of possible action: how we can examine, grasp, shove, leap over, or move around an object. This coding depends on the capabilities of our bodies, both as a species and as individuals. Because the world is perceived in terms of its potential for interaction with an individual's body, it is proper to call the perception "embodied."
Patterns of action derived from the projectable properties of the environment are combined (or meshed, section 3.1) with patterns of interaction based on memory. The two patterns can combine because they are both embodied, that is, both are constrained by how one's body can move itself and manipulate objects. The resulting pattern of possible actions is a conceptualization: the possible actions for that person in that situation. For example, turn left to get home.
Thus meaning of an object or a situation is a pattern of possible action. It is determined by the projectable features of the object molded by bodily constraints and modified by memory of previous actions. These memories provide the non-projectable features. As another example, consider the meaning of the cup on my desk. the embodied meaning is in terms of how far it is from me (what I have to do to reach it), the orientation of the handle and its shape (what I have to do to get my fingers into it), characteristics of its size and material (the force I must exert to lift it), etc. Furthermore, the meaning of the cup is fleshed out by memories of my previous interactions with it: pouring in coffee and drinking from it. Those memories make the cup mine.
Note three characteristics of this sort of meaning. First, because bodily actions take place in space, embodied meaning captures spatial (or topological) and functional properties. Thus a synonym for this type of embodied meaning is spatial-functional meaning. Second, because we interact with objects via parts, conceptualization in terms of bodily interaction forms the basis for partonomies (Tversky and Hemenway, 1984) and basic-level categorization. Third, conceptualization in terms of patterns of bodily interaction is very close to Gibson's (1979) notion of affordance.
Thus, what is memory for? Its primary function is to mesh the embodied conceptualization of projectable properties of the environment (e.g., a path or a cup) with embodied experiences that provide nonprojectable properties. Thus the path becomes the path home and the cup becomes my cup. This meshed conceptualization, the meaning, is in the service of control of action in a three-dimensional environment.
How far can this account of embodied meaning be pushed? At the least, there are intriguing results that fit this account nicely and which do not seem to have a natural explanation in cognitive accounts based on meaningless symbols. I will review some of this literature from domains of affect, memory, and imagery.
Van den Bergh, Vrana, and Eelen (1990) presented typists and non- typists sets of letter pairs (e.g., WX and ZD). The subjects were asked to choose the one pair (from each set) that was liked the best. Typists showed a clear preference for pairs typed with different fingers over pairs typed with the same finger, whereas the non-typists showed little preference. (The typing finger was determined using AZERTY keyboards in Belgium and QWERTY keyboards in the U.S.) Van den Bergh et al. argued that for typists, part of the encoding of letters is as a motor program or movement. The incompatible movements generated by letters typed with the same finger resulted in a negative evaluation. It is unlikely that this effect arose from associations to specific letter combinations because the effect was most robust for pairs of letters with low frequency in the language.
Berkowitz and Trocolli (1990) and Berkowitz, Jo, and Trocolli (1993) illustrate the influence of the body on affect judgments. In one experiment, subjects were asked to judge the personality of a fictitious person described in neutral terms. Half the subjects listened to the description while holding a pen between their teeth without using their lips. This activity forces the face into a pattern similar to that produced by smiling. The other subjects listened to the description while biting down hard on a towel. This activity forces the face into a pattern similar to that produced by frowning. The subjects who were smiling rated the person described more positively than did the subjects who were frowning. It is unlikely that this effect arose due to demand characteristics of the experiment for the following reason. The effect was obtained only when subjects were distracted from their activities; when they were asked to focus on the activities, the subjects seemed to compensate for the forced smile (frown) and rate the description more negatively (positively). What can account for this finding? Experienced emotion is embodied. When the body is manipulated into a state that is highly correlated with an emotion, the body constrains other cognitive (that is embodied) processing.
Montello and Presson (1993) asked subjects to memorize the locations of objects in a room. The subjects were then blindfolded and asked to point to the objects. Pointing was fast and accurate. Half of the subjects were then asked to imagine rotating 90 degrees and to point to the objects again. That is, if an object was originally directly in front of the subject and the subject imagined rotating 90 degrees clockwise, the correct response would be to point to a location toward the subject's left. In this condition, the subjects were slow and inaccurate. The other subjects, while blindfolded, were asked to actually rotate 90 degrees and to point to the objects. These subjects were just about as fast and accurate as when pointing originally. Thus, mentally keeping track of the locations of objects, a task that many cognitive psychologists would suspect as being cognitive and divorced from the body, is in fact strongly affected by literal body movements.
Rieser, Garing, and Young (1994) reported a similar finding for children and adults. The participants were tested for the ability to imagine (while at home) their classrooms and to point to objects from various perspectives. When the perspective change was accomplished by actually changing position (at home), the 5-year-olds were correct on 100% of the trials, the 9-year-olds were correct on 98%, and the adults on 100%. When the the perspective change was accomplished solely by imagination, the 5-year-olds were correct on 2% of the trials, the 9-year-olds were correct on 27%, and the adults were correct on 100%. Even the adults showed great difficulty in terms of the time needed to accomplish the imagination-only version of the task. When actually changing position, 100% of the adult responses required less than two seconds, whereas when only imagination was used, only 29% of the responses required less than two seconds.
Findings on the psychophysiology of imagery also point to a close connection between body and cognition. These findings are summarized by Cuthbert, Vrana, and Bradley (1991). Their starting point is Lang's (1979) bio-informational theory, which asserts that encoding of events includes response "propositions", and that imagery (visual and otherwise) is the activation of those propositions. Furthermore, although overt responding is inhibited during an imagery task, there may well be "efferent leakage" that can be measured using psychophysiological techniques. In support of these ideas, Cuthbert et al. note that psychophysiological responsivity is particular to the image being evoked. Thus imagining a fearful situation evokes sweating, imagining positive situations results in measurable activity in muscles associated with smiling, and imagining negative situations results in activity in the muscles associated with furrowing of the brow. There are analogous effects for imagery related to other perceptual/action systems. Thus, in imaging pendular motion, discharges in the eye muscles follow the appropriate frequency, in imaging bicep curls there are discharges in the biceps, and in imagining the taste of a favorite food there is an increase in saliva flow. These results are compatible with the notion of embodied, spatial-functional encoding. In addition, the idea of embodied encoding has an advantage over Lang's response propositions. According to Cuthbert et al. (1991), the function of imagery is to allow new behaviors to be tried out "off-line." It is not clear however, how response propositions can be integrated (other than by concatenation) to effect this rehearsal. In contrast, the integration of responses is basic to the notion of mesh (see Section 3.1) of embodied encodings. That is, given that the information is encoded in terms of bodily interaction, effecting one action (or imagining it) necessarily constrains the operation of simultaneous and successive actions.
Effects of embodiment is revealed by research on memory for subject-performed tasks (Cohen, 1981; Engelkamp & Krumnacker, 1980; Saltz & Donnenwerth-Nolan, 1981; see also a special issue of Psychological Research, 1989). The basic finding is that memory for actions (performing a command such as, "open the book") is better than memory for the verbal description of the commands. One interpretation of this finding is that memory specializes in embodied information.
The nature of our bodies also controls ease of remembering. Consider a series of studies by Tversky and her colleagues (e.g., Bryant, Tversky, and Franklin, 1992). In these experiments, subjects read about and memorized spatial layouts corresponding to scenes viewed from particular perspectives (e.g, in the hotel scene, "To your left...you see a shimmering indoor fountain..."). Objects were located above, below, in front, in back, to the left, and to the right of the observer in the imagined scene. After the scene was memorized, the time taken to retrieve a particular object was measured. For equally well memorized locations, one might expect the retrieval times to be independent of location. Another hypothesis is that the times would be correlated with the degree of mental rotation needed to mentally face the object. The results, however, were contrary to both of these hypotheses. Fastest responding was to objects located on the head/feet axis, followed by the front/back axis, followed by the left/right axis. Tversky argues that these results follow from using a "spatial framework" that is sensitive to environmental asymmetries (such as gravity) and perceptual asymmetries (we generally look and attend to the front). In other words, retrieval processes appear to be sensitive to how we use our bodies.
Klatzky, Pellegrino, McCloskey, and Doherty (1989) demonstrated contributions of the body to symbolic or semantic judgments. They trained subjects to make hand shapes corresponding to descriptions such as "pinch" or "clench." The verbal descriptions were then used as primes for judging the sensibility of phrases such as "aim a dart" (sensible) or "close a nail" (not sensible). The appropriate prime, that is a prime corresponding to the hand shape used in the to-be- judged action, speeded the sensibility judgment compared to a neutral prime. Thus, the hand shape for "pinch" speeded the sensibility judgment for "aim a dart." It is unlikely that this priming effect derives from any sort of verbal mediation: The priming effect was found for subjects trained to make the hand shapes when signaled by non-verbal primes. Also, when subjects were trained to make verbal responses (but not hand shapes) to the non-verbal primes (e.g., saying the word "pinch" when shown the non-verbal signal for pinch), the priming effect was eliminated. Klatzky et al. suggest that the sensibility judgment requires a type of mental simulation using an embodied, motoric, medium. Generating the appropriate hand shape "facilitates constructing the representation and/or simulating the action/object pairing."
Consider this scenario. You have been wandering in the woods, and suddenly you are unsure of the way home. You see what appears to be a path, but you are not certain if it really is a path, yet alone, the path home. You take a few steps and hunt for evidence. As you continue your exploration, you become convinced that this is the right path: the patterns of rocks, twigs, and soil align themselves to form a connected pattern that could be a path. Also, as you move along, you are able to conform your own body to the putative path. For example, the overhanging branches are not so low that you have to stoop or crawl; when you reach a stream, the distance between the rocks forms a series of stepping stones that can be used by an animal of your size and agility.
Recognition of the path as a path arises from an exploration of the environment and a fit between the environment and embodied knowledge. This fit can be conceptualized as a type of constraint satisfaction, but here the constraints are spatial and functional, not associationistic or probabilistic (cf., Rumelhart, Smolensky, McClelland, & Hinton, 1986). Thus projectable properties of the environment (arrangement of rocks, twigs, and soil) are encoded in terms of how you (with your particular body) can interact with that environment (e.g., whether the distances between the rocks in the creek can be broached). Other patterns of interaction come from memory, for example, patterns representing goals such as "get home without getting wet." In conceptualizing the environment as a path, the spatial-functional patterns based on projectable properties from the environment are combined or meshed with the patterns from memory. The meshed pattern dictates how (or if) the body can be moved in a way that simultaneously satisfies both sets of patterns of action (e.g., "Can I, with my body, get from rock to rock without getting wet?"). This sort of mesh is a possibility because all of the patterns are embodied, that is, they are all encoded in terms of how your body constrains actions. When the patterns can be meshed into a plan for coherent action (e.g., stepping across the rocks), the rocks, soil, and twigs become (for you) a path.
I envision mesh of embodied encodings as being analogous to coarticulation in speech production. When pronouncing the initial /d/ in "dog," the articulators are shaped in part by the requirement to enunciate the following vowel, and when pronouncing the vowel, the articulators are shaped not only by the vowel, but by the preceding and following consonants. Furthermore, the constraints on articulation are not consciously imposed, but are constraints that follow from real movements of physical devices: The tongue can only be in one place at one time, and how it is going to move to the next place will depend on where it is now. Thus pronunciation of the word requires a mesh of real physical actions.
An example of cognitive meshing is borrowed from Barsalou, Yeh, Luka, Olseth, Mix, & Wu (1993). Imagine a ball; now imagine that it has yellow and white stripes; now imagine that the ball is deflated (it is a beach ball). Adding each new descriptor is not a matter of adding a simple association or adding a proposition to a list. Instead, each previously constructed representation constrains how the new descriptor is utilized. Thus, the yellow and white stripes surround the ball. Then, not just the ball, but the stripes too become deformed when the ball is deflated. The stripes and the ball deflate together because they are encoded as patterns of action subject to the same spatial-functional constraints. This meshing occurs, not just in imagination, but in memory, comprehension, and perception.
It is the mutual modification of meshed patterns of action that produces emergent and creative features of thought. The deflated beach ball is not simply a deflated ball associated with an unchanging stripes feature. Instead, the fact that the stripes are deflated arises from the operation of meshing. Related concepts will mesh easily, because that is what it means to be related (section 7.3), and with some effort, we can mesh arbitrary concepts. Thus a "tiger bicycle" is one designed for hunting tigers, and it consists of a mesh between the actions required to hunt and those required to ride a bicycle, whereas "colorless green ideas" are uninspired ways of dealing with environmental crises. In short, mesh underlies our ability to understand novel conceptual combinations. Note that the type of mesh I am proposing depends on the analog nature of embodied actions, not just their propositional content.
Meshing patterns of action based on projectable properties of the environment with those from memory changes the way we conceptualize the environment. Thus, the soil, twigs, and rocks are conceptualized not just as a path, but as the path home. There is a danger, however, in allowing patterns from memory to modify conceptualization: meshing of patterns can distort the perception of the environment. Clearly, survival requires seeing the environment for what it is (soil, twigs, rocks), not just for what it means (the path home). To keep the system reality-oriented, it is necessary to ensure that patterns based on projectable properties of the environment are primary. That is, the meshed conceptualization that is achieved cannot be at the cost of distorting the environmental input. I will refer to this as clamping projectable properties of the environment.
Clamping projectable properties ensures that experiences are individuated or situated. We do not experience categories, but individual, particular events (cf. Barsalou et al., 1993). We cannot direct our perceptual system to ignore differences between two paths, just because they are both paths, or between two chairs just because we can fit our bodies into both. Because the projectable properties are clamped, the two chairs, although members of the same category, remain separate chairs.
I have proposed that embodied memory acts as a source of nonprojectable patterns of action that mesh with patterns derived from projectable properties of the environment; the mesh is possible because both sets of patterns are constrained by how the body works. If memory is to be useful, however, it must be updated. That is, new experiences must affect the system so that we come to learn the path home. Because experience is continuous (or at least the environment appears continuous to beings of our size and abilities), we must deal with how it can be captured by a system using a finite brain.
Consider this possibility. Projectable properties are clamped and then embodied memories mesh to produce a particular conceptualization (e.g., the path home). At this point, either an action is taken (e.g., a step along the path) or projectable properties of the environment change (e.g., a barrier appears). In either case, the system is forced to settle into a new conceptualization. Here is the proposal for updating memory: Memory is updated automatically (that is, without intention) whenever there is a change in conceptualization (mesh). The degree to which updating takes place is exactly correlated with the degree to which the conceptualization changes.
Updating is not encoding a new memory trace. Instead the shift from one pattern of possible actions (one conceptualization) to the next is reinforced. That is, what is updated is how one situation flows into another. I will refer to this flow as a trajectory. I am using the term trajectory to imply that the change is not random. Instead, actions humanly possible under the current conceptualization are biased by what was possible in the previous conceptualization, just as pronunciation of a vowel is biased by the pronunciation of the preceding consonant.
The idea of trajectories solves several problems in the psychology of memory. It provides a way of conceptualizing dynamic information in memory that is sensitive to biological and spatial-functional constraints (Shiffrar, Heptulla, O'Shaughnessy, & Freyd, 1993). Trajectories can reflect minimal changes in conceptualization, such as from one step along a path to another, or gross changes such as from a step to a fall. The idea offers the beginnings of a solution to the problem of features. Most theories of memory are based on the idea that memories are multidimensional, consisting of a vector of features, such as animate, red, and smaller than a bread box. None of these theories, however, is committed to a listing of what those basic features might be. In fact, because experience is so varied, it is hard to imagine a complete list. Also, given a feature-based system, it is difficult to understand how people can ever learn anything truly new: We must always conceptualize using the same basic features. In contrast, because embodied patterns of action can be infinitely varied and infinitely meshed with goals (also specified as patterns of action), a system based on embodied concepts and trajectories approaches the ideal of enabling memory to code the full variety of human experience.
Because updating of trajectories occurs only when there is a change in conceptualization, memory is sensitive to frequency and to novelty. To illustrate this, consider once again walking the path home. Three phenomena are associated with repeated actions. (1) Memory for the repeated action (walking the path) will be an increasing, but negatively accelerated, function of frequency (e.g., Logan, 1988). (2) Memory for a particular typical repetition of the action will be poor (Glenberg, Smith, & Green, 1977; Naveh-Benjamin & Jonides, 1984). (3) Memory for a particular unusual repetition of the action will be good (Hunt, 1995). The frequent interactions with the path will result in frequent updating (reinforcement of a particular trajectory), and consequently a shift toward a stable conceptualization (e.g., a shift from possibly the path home to definitely the path home). However, once the conceptualization is stable, little further updating occurs. Thus, each encounter with the path will have less and less of an impact (phenomenon 1). Because typical encounters result in little new conceptualization and little updating, we have little memory for the individual steps down the path (phenomenon 2). However, if reconceptualization is required (e.g., when a log appears across the path, so that now, in terms of bodily constraints on action, the path is a blocked path) memory is again updated, leading to memory for novel events (phenomenon 3).
The meshed conceptualization of the current environment dictates what actions are possible in that environment. Prediction, however, requires simulating how an action will produce a new conceptualization, which in turn can be used to simulate the next action, etc. Two difficulties arise. The first is that simulated action does not change the environment. Thus, changes in projectable properties that would have resulted from a real action cannot be clamped to automatically guide further action. A second difficulty is that currently clamped stimulation is providing the wrong constraints, because those constraints are only relevant before the simulated initial action. I believe that this is a major problem, and that it requires a radical (and dangerous) mechanism: suppression. In particular, I propose that in the service of prediction, we have developed the ability to, if not ignore, at least to suppress the overriding contribution of the current environment to conceptualization. This is a risky operation because it loosens the tie between reality (the current environment) and conceptualization. Perhaps because suppression is so dangerous, it is an effortful process. As we will see, however, suppression results in several serendipitous abilities, including conscious autobiographical memory and language comprehension.#
Once clamping of projectable properties is suppressed, multi-step prediction arises from following trajectories guided by bodily constraints on action. For example, by following trajectories we can envision what will happen when we proceed down the path. We also have the ability to envision arbitrary events (such as what actions are possible if the path is washed out by a storm or blocked by strange creatures) not just events we have previously experienced. Prediction for these arbitrary scenarios is based on seeking a mesh among patterns of action. Some of the patterns are based on well-learned trajectories. Other patterns (e.g., interactions with strange creatures) come from a consideration of how our bodies work. These patterns can mesh to give a coherent conceptualization because they are all based on bodily interaction. Keep in mind, however, that in prediction, the mesh of these patterns may not be guided by stable and projectable features of the environment. To the extent that environmental constraints are suppressed, and to the extent that trajectories are not well-learned, the predictions will tend to be variable and inaccurate. Thus, it is easy to predict the outcome of the next step on a well-traveled path: the simulated mesh is strongly constrained by projectable features of the current environment and well-learned trajectories. It is more difficult to predict what will happen many steps down a new path when the projectable features must be suppressed and trajectories uncertain.
Many of the ideas and much of the terminology introduced in Section 3 are borrowed from connectionist approaches to cognition. Some examples are constraint satisfaction, trajectories as paths through a set of states, and clamping of projectable features. There are two other, perhaps deeper, similarities. As I will discuss in Section 6.2, an embodied conceptualization functions as a preparatory state. Given a particular conceptualization, an organism is better prepared to act when changes in the situation easily mesh with the conceptualization than when changes do not easily mesh (i.e., we are surprised). This notion of preparedness underlies priming phenomena, and it is close to connectionist interpretations of semantic priming developed by Masson (1995) and Sharkey and Sharkey (1992).
The second deeper similarity relates to ideas of context and situated representation. For example, Smolensky (1988) discusses how a distributed representation of "coffee" will depend on whether the coffee is in a cup, in a can, or in a person. Similarly, as I discuss in Sections 3.1 and 7.3, action patterns based on projectable features of an object (e.g., a coke bottle) can mesh with action patterns underlying goals in particular contexts (e.g., drinking or fighting), so that the resulting meshed conceptualization is context-dependent.
Nonetheless, there are important differences between my use of terminology and connectionist systems. For example, connectionist accounts of semantic or meaningful information are based on conceiving of meaning as activation of a limited number of features, at least at the input layer. Unfortunately, most theorists fail to specify what the features are, and they fail to specify how those features might be learned or changed as a consequence of development. In the system that I am proposing, initial coding is not featural, but analog, in terms of patterns of possible action. Furthermore, as one learns more about the interactive capabilities of one's body, objects and actions can be imbued with new meaning: What I can do with that object now.
A second important difference concerns the nature of constraints. In standard connectionist accounts, constraints are, in Palmer's (1978) terminology, extrinsic (but see Regier, 1995, for an exception). That is, a particular constraint represents statistical, or joint occurrence, information, not a necessary feature of the operation of the system. Thus, a connectionist system would be equally happy to learn that a coke bottle can be used as a chair or as a weapon. In an embodied system, constraints arise because of analog coding of projectable features and their implications for human action. In Palmer's terminology, these constraints are intrinsic to the operation of the system. For example, how we think about a coke bottle is constrained not just by particular experiences with coke bottles, but by the actual shape and heft of the bottle, too. Thus, an embodied system would have little difficulty understanding how a coke bottle could be a weapon, but it would balk at learning that it could be used as a chair.
These differences are not unique to my proposal. Lakoff (1988) argues that connectionist systems need to be grounded in the body to give meaning to connections and constraints. As an illustration, he notes that phonology is not arbitrary; instead, it is constrained by the muscles, shapes, and control of articulation. Shepard (1988) makes a related point regarding the abilities of connectionist systems to self-organize and generalize:
"But nontrivial self-programming can take place only if some knowledge about the world in which the system is to learn is already built in. Any system that is without structure has no basis for generalization to new situations." (p. 52).
How the body can interact with the world provides just such a basis for generalization.
These comments should not be taken to mean that an embodied system cannot be simulated using connectionism. In fact, it may well be that connectionism will be the surest route to formalizing these ideas. Nonetheless, it will have to be a connectionism that differs from the sorts currently in use.
The system described so far seems to be useful for negotiating the environment, and it seems to correspond to what some have called semantic (Tulving, 1983) or generic (Hintzman, 1986) memory. Where is episodic memory, that is, our memory for particular, personal experiences? The answer: the same place. I propose that episodic recollection is a type of pattern completion via meshed bodily constraints on action. Furthermore, the episodic character, the feeling that a memory is personally relevant, arises from suppressing clamped projectable properties of the environment. In this case, conceptualization is driven by trajectories rather than by changes in the environment.
To some cognitive psychologists, this idea will seem wrong on the face of it: It denies the difference between episodic and semantic memory; it denies the idea that episodic memory is temporally organized; it provides no distinction between short-term and long-term memory. Before describing how the idea seems right, I will briefly address why these problems are more apparent than real.
I am explicitly equating episodic and semantic memory in the sense that there are no separate episodic and semantic memory systems, hierarchically arranged (Tulving, 1984) or otherwise. Of course, phenomenal memories differ in content, accessibility, etc. But those differences do not imply separate systems. Whereas this equation of memory systems may have been controversial ten years ago, data and mainstream memory theorizing are moving in this direction. In short, there is little data to support a distinction between a memory system devoted to personal experience and one devoted to general knowledge (McKoon, Ratcliff, and Dell, 1986). What appeared to be strong evidence for a memory organized by "semantic" dimensions (Collins & Quillian, 1969), is now known to reflect frequency of experience (Conrad, 1972). Evidence that was taken to indicate the storage of prototypes (Posner and Keele, 1968) in semantic memory, is now taken to reveal the operation of retrieval processes that can average experiences (Hintzman, 1986; McClelland and Rumelhart, 1986). Priming effects that were thought to reflect the spread of activation along permanent semantic links can be easily demonstrated for newly learned (hence episodic) information (McKoon and Ratcliff, 1986a). Thus the distinction between episodic and semantic memory probably reflects a difference in the frequency with which the memories are used, the methods of assessment, and the content of the information, rather than any intrinsic differences in memory systems.
If the framework that I have described is the only memory system, then it explicitly denies a tenet of theorizing about episodic memory: Memory is a record of events that maintains some semblance of temporal order [see, for examples, Murdock's conveyor belt model (1974) or Glenberg and Swanson's (1986) temporal distinctiveness theory]. Almost assuredly, the tenet that episodic memory maintains order derives from the fact that temporally distant information is harder to remember than recent information. This fact does not demand a theoretical explanation that maintains time as a dimension of memory, however. In fact, Friedman (1993) presents a convincing case that episodic memory is not organized temporally. First, there is little priming between temporally contiguous but are otherwise unrelated experiences. Second, memory for time of occurrence of events is not just inaccurate, it shows nonmonotonic scale effects. That is, memory for when an event occurred may be accurate for the day, inaccurate for the month, accurate for the season of the year, inaccurate for the year, but accurate for the decade. Third, as Friedman discusses, for most of human history, memory based on a linear dimension of time would serve little useful purpose. Instead, a memory organized by functional significance, or by recurrent events (seasons, migrations, life cycles), would seem to have much greater adaptive significance.
The idea of a single memory system seems wrong in that there is no mention of separate processes for long-term memory and for short-term or working memory. Much of the evidential basis for a separate short- term store (or working memory, according to Baddeley, 1990) has been eroded. For example, the recency effect is the enhanced recall of items from the end of a list. Because it was thought to be easily disrupted by a short period of distraction, it was taken has a hallmark of short-term store. We now know, however, that recency effects can be very long-term (Glenberg, 1984; Greene, 1986, 1992). Another supposed hallmark of a separate store is acoustic/articulatory encoding (e.g., Hintzman, 1968). That is, short-term store was believed to code information along acoustic/articulatory dimensions, whereas long-term store coded "semantic" information. However, demonstrations of meaning-like coding in short-term situations (Shulman, 1972), as well as long-term memory for articulatory and orthographic information (e.g., Hunt & Elliot, 1980) deny this simple distinction. Also, the quick forgetting demonstrated using the Brown- Peterson distractor technique, is now known to reflect a combination of poor initial coding (Muter, 1980), and interference from previously studied material (Keppel & Underwood,1962; Watkins & Watkins, 1975).
What are we to make of the impressive body of information on apparently separate short-term modules (e.g., Baddeley's articulatory loop, phonological store, and visual/spatial sketchpad)? An alternative theoretical position is to consider the evidence as indicative of skills and strategies effective in particular domains (cf. Kolers & Roediger, 1984), rather than separate modules. This skill-based alternative can easily accommodate findings that might otherwise be interpreted as evidence for new working memory modules. As one example, Reisberg, Rappaport, and O'Shaughnessy (1984), demonstrated an increase in "working memory capacity" by instructing subjects how to use their fingers to code numbers in a memory span task. This evidence might be interpreted as evidence for a new "finger-control" module, but it seems more sensible to view it as a newly-learned skill. As another example, Carpenter, Miyake, and Just (1994) speculate that there may be separate working memory capacities for language production and language comprehension. Again, the alternative that different skills are involved in comprehension and production would seem to more easily accommodate the data.
Nonetheless, one must come to grips with our intuitions of immediate access to some information and difficulty in recovering other information. Consider this proposal. Memory and the perceptual/action system are designed to produce a meshed conceptualization (possible actions) for current stimulation. It is this constantly changing conceptualization (changing because the stimulation changes in response to action) that gives the illusion of a short-term memory. Because the current conceptualization updates memory and provides the starting point for future conceptualization, it will have a strong influence on performance over the next few moments (as does a short-term memory). Distraction (a changing environment) does cause a disruption in short-term behavior because it produces a forced changed in the current conceptualization. Limits on the "capacity" of a short-term store are simply the limits on coherent conceptualization.
This framework also rationalizes some aspects of rehearsal and control of thought. In particular, it seems that some sort of cyclical activity is needed to maintain information in the forefront of consciousness. Baddeley (1990) discusses this as an articulatory loop that must reactivate the decaying contents of a phonological store. But if memory is like a box that holds items of information, why should cyclical activity be necessary? The answer comes from the nature of trajectories. They are not static memory traces; they are reinforced changes from one conceptualization to the next. Thus, there is no holding of trajectories in mind. Instead, to maintain a thought or a conceptualization in the absence of clamped projectable properties, it is necessary to reuse a trajectory, or to replay the same scene over and over.
The major function of memory is to mesh constraints on action based on non-projectable properties with constraints from projectable properties. This is an automatic function of memory in the sense that it is not under conscious control, and it corresponds rather directly to recent work on indirect or implicit memory. There is also an effortful mode of memory. Effortful suppression of projectable properties allows conceptualization to be guided by trajectories. The resulting conceptualization is what underlies personal, autobiographic, conscious recollection.
When we are walking the path home, we do not need to consciously recall which way to turn at each intersection; when we recognize our children in a crowd, it is not because we have subjected each face to a conscious check; and as we read each word in a sentence, there is no need to try to remember back to when we might have last encountered a similar-looking pattern in order to ascertain the meaning of the word. Memory is automatically, that is, without intention, creating a mesh between the projectable properties (the path, the faces, the letters) and patterns of interaction controlled by non-projectable properties. Research on indirect or implicit memory (Roediger, 1990, 1994; Tulving & Schacter, 1990) is tapping this automatic mode of functioning.
Indirect tests of memory do not require conscious decisions that something is remembered. Instead, the tests often measure some form of repetition priming: The extent to which previous exposure to a stimulus facilitates current processing. For example, a list of words (or pictures) can be presented in phase 1 of a repetition priming experiment. In phase 2, subjects are asked to identify degraded stimuli, some of which occurred in phase 1. Repetition priming is the phenomenon that identification of stimuli actually presented in phase 1 is superior to identification of stimuli presented for the first time in phase 2. This finding occurs whether or not the subjects are attempting to remember anything about phase 1 (e.g., Jacoby & Dallas, 1982; Weldon & Roediger, 1987). A more conceptual form of indirect memory can be measured by, for example, presentation of a word and the later choice of that word as an answer on a test of knowledge (Blaxton, 1989). Among the many interesting findings generated by this research, several may be particularly important. First, repetition priming, can be of very long duration. It is not unusual to be able to demonstrate positive effects over weeks and months (Sloman, Hayman, Ohta, Law, & Tulving,1988). Second, repetition priming effects are often sensitive to presentation and test modality. For example, pictures prime pictures more than pictures prime words, and vice versa (Weldon & Roediger, 1987). Third, people with dense amnesia often perform equivalent to non-amnesics on indirect, repetition priming tests (e.g., Musen & Squire, 1991). I address this last finding in section 5.2.4.
Jacoby (e.g., Jacoby, Toth, & Yonelinas, 1993) has made the case that much of repetition priming is due to an automatic component. Jacoby characterizes this component as "familiarity" that arises from "perceptual fluency." In the embodiment framework, the automatic component of memory is the contribution of embodied memories to conceptualization of the current environment. It is a type of perceptual fluency in that it affects how aspects of projectable properties are conceptualized. Because embodied memories do not change the clamped environment, the automatic operation of memory does not help one to literally see more clearly (that is, with greater acuity): Instead it helps one to understand the environment. That is why repetition priming has negligible effects on accuracy (in the signal detection sense of ability to discriminate) while affecting interpretation (or bias, Ratcliff & McKoon, 1993; Ratcliff, McKoon, & Verwoerd, 1989).
Repetition priming is modality specific because it is often based on clamped projectable properties. For example, consider an experiment in which both pictures and words are presented and later subjects must identify the objects in fragmented pictures. To identify the pictured object, subjects must use their memories to mesh with the projectable fragments. Clearly, features of the letters used in spelling the name of the pictured object are irrelevant to this task, so little priming is expected or found between reading words in phase 1 and identifying pictures in phase 2 (Weldon & Roediger, 1987).
Use of trajectories may underlie conceptual forms of repetition priming as well. For example, presentation of "Amazon" in phase 1 will facilitate answering, "What is the longest river," in phase 2. Clearly, words are more than just marks on a page. In reading "Amazon" we think about what rivers are in terms of swimming, fording, etc. This cognitive activity reinforces trajectories from the word "Amazon" to these activities. Later, in comprehending the question, "What is the longest river," we may create a similar conceptualization of rivers. Given the previously reinforced trajectories, the embodied conceptualization of Amazon is easily reachable (that is, meshes with) the embodied conceptualization of "longest river."
In Section 3.4 I discussed the idea that multi-step prediction requires suppression to loosen control of projectable properties on conceptualization. I suggested that suppression is dangerous because projectable properties that should be controlling action (such as walking) are ignored. This analysis leads to several suggestions. First, because suppression is dangerous, it is effortful. The effort is a warning signal: Take care; you are not attending to your actions! Also, the effort forces us to use suppression conservatively (because it hurts to think). Second, there are behavioral indices of suppression. For example, when working on a difficult intellectual problem (which should require suppression of the environment), we reduce the rate at which we are walking to avoid injury. Third, autobiographical memory arises from suppressing the environment: Once the environment is suppressed, conceptualization is controlled by trajectories and bodily constraints on mesh rather than the projectable features of the environment. Thus recollection is similar to prediction. Both are effortful, both depend on trajectories, and both are constrained by the body. On this view, conscious recollection is a type of pattern completion that is inherently reconstructive (Bransford, 1979).
The effort in suppressing the environment can be used to explain standard and non-standard facts of episodic memory. As an example of the latter, consider the phenomenon of averting the gaze when engaged in a difficult memory task. When recollection is difficult and unrelated to the current environment, clamping of the environment must be suppressed to allow internal control over conceptualization. Closing one's eyes or looking toward a blank sky are actions that help to suppress the environment by eliminating projectable properties that would normally be clamped. Glenberg, Schroeder, and Robertson (1995) have demonstrated that people avert their gaze when working on moderately difficult recollection tasks (but not easy ones), and that this behavior enhances accurate remembering.
How people are instructed to think about (i.e., encode) to-be- remembered stimuli greatly affects success in conscious recollection. Interactive imagery (e. g., Bower, 1970), levels of processing (Craik & Lockhart, 1972), and generation paradigms (e.g., Slamecka & Graf, 1978) all illustrate this phenomena. As an example, consider the use of interactive imagery to memorize arbitrary pairings such as "lamp - 88." Success in remembering the pairing is greatly enhanced by imagining, say, a neon light shaped to form the digits 88, compared to rote rehearsal of the words.
Standard analyses based on the notion of abstract, amodal symbols have difficulty with these effects because the abstract propositional description of the to-be-remembered stimuli are the same regardless of the encoding task. That is, for both rote rehearsal and imagery one must remember the same thing, "lamp - 88." On an embodied account, constructing an image requires meshing a conceptualization of a lamp with that of 88. The changes in conceptualization from the orthographic stimulus to the meshed image update memory trajectories. Later, partial information such as "lamp" may be given as a cue for the pair. Reading and conceptualizing "lamp" will be along the lines of the reinforced trajectory. Importantly, the analog shapes of the successive conceptualizations increasingly specify the final conceptualization of the neon 88. Contrast this with a situation in which the encoding task is not interactive imagery but simply reading the two words or engaging in rote rehearsal. There is little mesh created by reading the words: the words are pronounced separately so that there is not a physical mesh such as that produced by co- articulation. Furthermore, there is no conceptual mesh in terms of the patterns of interaction between the two objects named by the words. No wonder that little can be reconstructed from the cue "lamp" alone.#
Why is there a phenomenal feel to conscious recollection? Why does the content of memory appear to reflect personal experience? Why doesn't perception or automatic uses of memory feel that way? The feel of memory comes from the effort of suppressing the environment and the consequent knowledge that conceptualization is being driven by previously created trajectories. This process has the feel of personal memory because of our belief that the achieved conceptualization is free from domination by the projectable properties of the environment.
To the extent that skill in suppressing the environment develops, it suggests explanations for several related phenomena. Consider first infantile amnesia. There is now good evidence that the phenomenon is not as dramatic as initially proposed. In particular, there is evidence for good early retention when it is tested non- verbally. To the extent that a test trades on the automatic operation of memory, it should reveal substantial memory for the infants. In addition, both Howe and Courage (1993) and Nelson (1993) have suggested that what changes around ages 2-3 is the child's ability to code and retrieve information in ways understandable to adults. For Howe and Courage, this amounts to developing a self-concept useful in organizing and retrieving memories. For Nelson, this amounts to learning how to use narrative structures to organize and relate the child's narrative (i.e., self) experiences. Nelson notes that this learning is typically guided by interactions with adults.
Consider the following explanation for the correlation between development of self-concepts and the emergence of recollective experience. Recollective experience requires a) suppression of environmental input, b) use of self-generated information (trajectories) to drive the conceptual system, and c) an attribution that the resulting conceptualization is due more to internally-guided than externally-guided construction. I suspect that a major factor in the development of a concept of self is just the ability to suppress environmental information. Until that skill is mastered, conceptualization is controlled by the clamped environment; after that skill is mastered, conceptualization can be guided by oneself. That is, one can control what one is thinking about. Furthermore, development of language (by interacting with adults) may well be an important experience in learning how to control suppression and recollective experience: Development of language facility is tantamount to learning to use words to guide conceptualization. Thus skill in suppressing the environment is facilitated by language, and this same skill supports recollective experience and the development of a notion of self.
If recovery from infantile amnesia requires learning to suppress the environment's control of conceptualization, perhaps adult anterograde amnesia results from a traumatically-induced reduction in the ability to suppress. Two findings are consonant with this speculation. First, amnesics exhibit poor performance on explicit tests of memory requiring conscious recollection, but not on implicit tests of memory (e.g., Musen & Squire, 1991). According to the framework developed here, it is the explicit, recollective tests that require suppression of the environment, not the implicit, automatic tests. Second, although there are numerous explanations of amnesic abilities and disabilities, none provides any explanation for the feel of memory. That is, when it can be demonstrated that amnesics are using past experience as effectively as normal rememberers (on implicit or automatic tests), why don't the amnesics have any sense that they are remembering? Of course, the same question can be asked of the normal rememberers: When they perform well on an implicit (automatic) memory task, why do they lack the experience of remembering? For the normal rememberers, the feel of memory comes from an effortful suppression of environmental input and the attribution that conceptualization is controlled by the self. When conceptualization is controlled predominately by the environment, as when performing implicit memory tasks, it does not feel like memory. And, this is the usual state for amnesics.
Other memory researchers have proposed ideas similar to the framework outlined here. A particularly good example is the "procedures of mind" approach (Kolers & Roediger, 1984). In fact, the similarities between the approaches are striking. Kolers and Roediger suggest that many distinctions popular in memory theorizing reflect different skills rather than different memory stores. Importantly, while championing a symbolic account, Kolers and Roediger note that abstract, meaningless symbols will not do. Instead, they prefer symbols that retain characteristics of how they were acquired: "We claim that knowledge of objects is specific to the means of experiencing them" (page 419). Thus the symbols are in some ways analogical, as I have advocated. Kolers and Roediger also object to modeling knowledge using psychologists' propositions because "descriptions of events rarely if ever tell a person what to do about the events described" (page 439). Of course, conceptualization in terms of patterns of interaction with the environment was designed to overcome this problem. Finally, Kolers and Roediger eschew the idea that memory is purely a conscious experience. They propose instead that the most important contribution of memory is to the automatic execution of skills.
Given the similarities between the Kolers-Roediger program and the embodiment framework, are there any differences? One is my emphasis on meaning, that the meaning of an object or event is a meshed pattern of possible action. A second difference is the idea of mesh itself. The mesh between the projectable features of an object and non-projectable features from memory can dramatically change the meaning of an object or event (see section 7.3). This sort of combination is made possible by considering both the projectable and the non-projectable features to be patterns of action that can combine as physical, bodily actions can be combined. If separate patterns of action can be forced into a coherent pattern of bodily movement, then we can comprehend the combination; in this way rocks, twigs, and soil combine to form a path for a particular person. It is not clear how the skills described by Kolers and Roediger can be combined except through concatenation. Finally, I am attempting to extend the analysis to language comprehension.
I have argued that the same memory system underlies perception, semantic memory, and episodic memory. The meaning of a situation is given by a meshed pattern of possible actions, and that is an embodied conceptualization. The system is updated whenever there is a change in conceptualization. Thus, the environment is comprehended as a series of transformations of embodied conceptualizations. I propose a similar characterization of language comprehension. Language comprehension, like comprehension of the environment, is the successive transformation of conceptualizations which are patterns of possible action.
Like recollective memory, language comprehension requires suppression of the environment, but in two ways. First, the content of the language may have nothing to do with the physical environment in which the language is expressed. Lectures, for example, have little to do with the lecture hall. Thus, we must suppress projectable properties of the environment. Second, and perhaps more difficult, we must also suppress the projectable properties of the language signal itself. That is, to understand the language, we cannot focus on the shapes of the letters, the patterns of spaces between the words on the page, or the chirps and squeaks of the speech signal.
Several predictions follow from the claim that language comprehension requires suppression of projectable properties. The first is that good language comprehenders should be good at suppressing the environment. Second, good language comprehenders should be good recollectors, given that both require suppression. Third, unavoidable or non-suppressable properties of the environment should disrupt language comprehension. Of course, distracting noise or sights will impair comprehension, but a more subtle effect is discussed by Sanford and Moxey (1995). They note that many types of regularity seem to disrupt language comprehension, and hence those regularities are classified as instances of poor style. Repeating patterns of articulation (McCutchen & Perfetti, 1982), phonemes (e.g., "Crude rude Jude chewed stewed foods," from Baddeley and Hitch, 1974), and excessive repetition of particular sentence structures all seem to slow comprehension. Sanford and Moxey propose that these regularities contribute to the computation of coherence, but because the regularities "are irrelevant to the writer's message," the processing rapidly runs to a halt. Here is a different (but related) suggestion. The regularities are regularities in the projectable properties of the environment. The regularities capture attention and contravene the suppression required for conceptualization and comprehension of what the language is about. That is, instead of paying attention to the meaning of the language, we start to pay attention to the language itself.
Fourth, because language comprehension is seen as a general skill, performance in language comprehension tasks should correlate with performance in other comprehension tasks. Gernsbacher, Varner, and Faust (1990) have demonstrated just this.
Suppose, as Taylor and Tversky (1992) claim, that at least one of the functions of language is that "Language is a surrogate for experience" (page 495). If language is to be a useful surrogate, it must make contact with the sorts of embodied representations that we use to characterize the world, and I propose that language does this relatively directly: We understand language by creating embodied conceptualizations of situations the language is describing. In fact, this is the only reasonable story for how we can manage to learn from language.
This story works when language is being used as a surrogate for events that are completely absent, and when language is being used to enhance current experience. Consider a situation, in which a mother is instructing her child. Representations derived from the language must smoothly integrate (mesh) with representations derived from other aspects of the environment. Thus, being told "That plate is hot," must modify the embodied representation of the plate in order to modify interactions with the plate. Tannenhaus, Spivey-Knowlton, Eberhard, and Sedivy (1995) demonstrated just this sort of smooth and immediate integration. Their subjects responded to verbal commands (e.g., "Put the apple on the towel in the box.") to move actual objects arrayed before them. Eye movements were monitored during the task. Movement of the eyes to referent objects was extremely closely time-linked to the verbal command. Additionally, the environment was used to smoothly disambiguate the language. For example, when considering the language alone, the phrase "on the towel" is temporarily ambiguous. It may describe the location of a particular apple (the apple that is on the towel) or where an apple is to be put. Indeed, when there was only one apple in the array, the eye movements indicated uncertainty. When the array contained two apples (one on a towel and one on a napkin), however, then the phrase "on the towel" will almost certainly be meant to specify a particular apple, not a location in which to put the apple. In the two-apple case, the eye movements indicated no uncertainty. Thus, understanding of the sentence made virtually immediate use of the context, in contrast to notions of modularity of syntactic analysis. This sort of integration is possible if both the environment and the language are understood as embodied patterns of action.
This sort of reasoning is compatible with work on mental model theory. The basic claim of mental model theorists is that language comprehension results in representations of what the language is about, not representations of the language itself (e.g., chirps, words, sentences, or propositions). Johnson-Laird (1989, p. 488) writes that a mental model is a representation of a situation so that "its structure corresponds to the structure of the situation that it represents." With an important emendation, this definition can apply to the sorts of representations I have been describing. It seems unlikely that the literal, in-the-head structure of the representation could actually be isomorphic to the structure of the situation (in contrast to Glenberg, Kruley, & Langston, 1994).
Embodied mental models are "models" in the following sense: A model is useful if it can be used to predict the effect of an action in the real situation being modeled. One way to ensure accurate prediction is to build into the model spatial-functional constraints analogous to those of the real situation. For example, a useful model of an airplane will have wings that generate lift when it interacts with air currents, much like a real airplane's wings generate lift. Similarly, a mental model built from language incorporates embodied constraints on action like those derived from comprehension of the environment. This sort of mental model is useful to the extent that it incorporates enough constraints on action to derive predictions.
One difference between embodied models derived from language and those derived from perception is how completely the meshed pattern of possible actions constrains further action and prediction. The multiple projectable properties of the environment, because they are clamped, tightly constrain conceptualization and action. In language comprehension, the patterns of possible action that contribute to a meshed conceptualization are much looser. That is, language is ambiguous in not specifying exact parameters of spatial layout, force, etc. (Talmy, 1988). Thus, conceptualizations derived from language do not constrain action as effectively as conceptualization derived from the environment. This is one reason for differences between expert and non-expert comprehension. The expert's model incorporates tighter constraints on action based on trajectories derived from experience. Given the same text, the expert is able to take (appropriately constrained) actions that leave the non-expert baffled. This effect of expertise in language comprehension parallels the expert guide who can spot the trail (based on trajectories derived from experience) while the novice sees only twigs, soil, and rocks.
I have argued that embodiment in terms of action patterns is just what is needed to facilitate interaction with the environment and prediction. Is prediction an important component of language comprehension? Clearly, language would be of little use if it did not enable better prediction of the environment. But, the question asked in the literature on comprehension is different: Does a mental model serve as a source of "on-line" predictions about the upcoming text? In response to this question, one might ask "Should it?" If the point of language is to be a surrogate for experience, that is, to help us take appropriate action in real situations, it makes little sense to expect the representation to predict upcoming text: It should predict changes in the situation. In fact, McKoon and Ratcliff (1992) reached the conclusion that there is little evidence that people make predictive inferences while reading.
Nonetheless, Keefe and McDaniel (1993) presented convincing evidence for what appeared to be just those inferences. Using the standard logic based on psychologists' propositions, Keefe and McDaniel reasoned that pronunciation of a probe word would be faster if the word were part of a recently made inference than if not. For example, subjects read a sentence such as, "After standing through the three-hour debate, the tired speaker walked over to his chair." Following the sentence, subjects pronounced the probe word "sat." Supposedly, pronunciation of the probe word would be facilitated by its having been incorporated into an inferred proposition such as "The speaker sat down." In the control condition, for which an inference including the word "sat" is unlikely, subjects read a sentence such as, "The tired speaker moved the chair that was in his way and walked to the podium to continue his three-hour debate." Indeed, pronunciation of the probe word was faster in the predictive condition than in the control condition. In fact, pronunciation of the probe following the predictive sentence was as fast as when the sentence explicitly continued with "and (he) sat down." Murray, Klin, and Meyers (1993) used a similar methodology and produced a similar effect when the "to-be-inferred event was in focus at the time of test" (page 464). Why is evidence for predictive inferences found only shortly after the predicting sentence? Does this evidence demonstrate that subjects were attempting to predict the upcoming text?
Consider an interpretation of these findings from the point of view that the goal of language comprehension is the creation of a conceptualization of meshed patterns of action. In this case, interpretation of a word, phrase, or sentence, consists of meshing the actions consistent with that bit of language with the patterns of action derived from previous text. After comprehending Keefe and McDaniel's predictive sentence ("...the tired speaker walked over to his chair"), only certain actions can be easily meshed with the conceptualization. For example, the actions implied by "He began to rake the leaves," does not mesh. In contrast, the action of sitting will mesh, and hence interpretation and pronunciation of the probe word "sat" is quick. After comprehending Keefe and McDaniel's control sentence ("...walked to the podium to continue his three-hour debate") the action of sitting meshes about as well as the action of raking, and so pronunciation of the probe word "sat" is slow.
This interpretation of the results is radically different from that used in standard propositional accounts of inference making. In the standard account, an inference corresponds to encoding a new proposition, something akin to, "He sat down," and one would expect some effect of this proposition well after it was encoded. The embodied account is that no "inference" in the standard sense is made. Instead, the action of sitting in the chair is temporarily compatible with the embodied conceptualization. When the situation changes, some actions are no longer compatible with the embodied conceptualization and the "inference" is no longer operative. This notion of temporary compatibility (how well the probe will mesh with the other constraints) may well underlie McKoon and Ratcliff's (1986b) data for "partial" encoding of predictive inferences, and the temporary effect noted by Keefe and McDaniel (1993) and Murray et al. (1993). Of course, this is not to say that language comprehenders might not make forward inferences if induced to do so (e.g., one might be asked to "Guess what happened next"). These sorts of inferences are just the sort of predictions discussed in Section 3.4. However, given that language, unlike the environment, only loosely constrains action, it is more reasonable to wait until what happens next is described.
The procedures used by Keefe and McDaniel (1993) and Murray et al. (1993) follow from the more general notion of "semantic priming." The standard idea is that processing causes activation to spread along permanent links to semantically related information, and this spread of activation speeds processing of the related information. Thus, reading the prime, "doctor" speeds the decision that the target "nurse" is a word (Meyer & Schvaneveldt, 1971). The standard interpretation of semantic priming is embarrassed by demonstrations that priming need not be due to permanent links (McKoon & Ratcliff, 1986a), and that the effective relation between the prime and the target may have little to do with the presumed semantics of categories (Sheldon & Martin, 1992). Might semantic priming be another instance of the operation of mesh? Assume that language comprehension is an attempt to mesh action suggested by the current word or phrase with the pattern of actions already established. Thus, in thinking about what a "doctor" is (the actions taken by a doctor and how one interacts with a doctor), one sets up a conceptualization in which the actions suggested by "nurse" will mesh. Hence, processing of "nurse" is facilitated relative to the case when it is preceded by an unrelated prime word such as "rake."
A report by Hess, Foss, and Carroll(1995) strongly suggests that semantic priming reflects something akin to mesh rather than spread of activation along permanent links. Their subjects read a sentence describing a local context, such as "To complete the assignment, the English major wrote a..." and they then read a target word such as "poem." The question of interest was whether the local context ("English major") would facilitate reading of the target ("poem") regardless of the global situation. This would be expected if priming reflects activation along permanent links such as between "English major" and "poem." In one global situation, the English major was working on a writing assignment, and indeed, reading of "poem" was facilitated relative to a neutral condition. In another global situation, however, the English major was working on a computer program. In this case, reading the target "poem" was not facilitated. The implication is that priming reflects ease of integration (mesh) of concepts, not spread of activation along permanent links.
If embodied conceptualization is a pattern of possible actions, then it must incorporate information about spatial layout, because actions are played out in space. The data from several research projects investigating spatial coding in mental models provide this evidence. First, there have been investigations of how language can lead to accurate, analogical representations of a described layout. For example, Denis and Cocude (1989) had subjects read texts describing the layout of objects on a circular island. After several readings, they were asked to mentally simulate scanning from one object to another. The main finding was of a correlation between distance (if the objects had actually been arrayed) and simulated scanning time. Morrow, Bower, and Greenspan (1989, see also Rinck & Bower, 1995) had their subjects memorize the layout of the rooms in a building (and objects in the rooms) before reading a passage describing the movements of a protagonist throughout the building. Morrow et al. measured time to verify that particular objects were located in particular rooms as a function of the protagonist's described movements. Interestingly, when a described movement (e.g., from Room A to Room C) required passage through an unnamed room on the path of the movement, verification of objects located in the unnamed room was faster than verification of objects in other unnamed rooms off the path. Apparently, subjects were using the spatial information in the building layout while comprehending the text.
Glenberg, Meyer, and Lindem (1987) demonstrated the contribution of spatial information to language comprehension without pre- memorization. Subjects read texts describing a protagonist and a target (e.g., a jogger and a sweatshirt) that were either spatially dissociated (the jogger took off his sweatshirt before jogging) or spatially associated (the jogger put on his sweatshirt before jogging). After a sentence or two in which the protagonist was kept foregrounded but the target was not mentioned, accessibility of the target (e.g., the sweatshirt) was greater in the associated condition than in the dissociated condition. [McKoon and Ratcliff, 1992, have argued that this effect may reflect a type of salience. See Glenberg and Mathew (1992) for a counter to this interpretation.] Along similar lines, O'Brien and Albrecht (1992) demonstrated sensitivity to spatial location of characters in a text well after the spatial information is introduced. Thus, several sentences after reading, "As Kim stood outside the health club, she felt a little sluggish," readers would balk at the sentence, "She decided to go outside..."
One interpretation of these findings is that they reflect a representation that is analogical with respect to space, that is, that the mental model is constructed in an inherently spatial medium. This seems unlikely. Langston, Kramer, and Glenberg (1995) have demonstrated that spatial contiguity, in the absence of other relations, does not have strong functional consequences. In these experiments, subjects read (or heard) texts describing the spatial layout of four objects. In outline, the texts read, "B is to the right of A, C is under B, D is to the left of/right of C." The last sentence in the "close" condition was "D is to the left of C," so that the spatial layout of the objects has D under (that is, close to) A. The last sentence in the "far" condition was "D is to the right of C," so that object D is separated from A. After reading, subjects were tested for availability of the target object, A. If space is represented analogically, and if closeness in that space has functional consequences, then the target should be more available in the "close" condition than in the "far" condition. We tested for availability of A using speeded recognition of A and time to read a sentence referring to object A. Availability of A was never reliably affected by the condition ("close" versus "far"), even though memory for the spatial layout was well above chance.
How are we to understand the contrast between Langston et al. (1995) and the other research that clearly points to an appreciation of spatial relations during comprehension? One possibility builds on the distinction between mental models encoding space in a spatial medium and mental models encoding spatial-functional action and thereby representing space incidentally. Consider a reinterpretation of Glenberg et al.'s (1987) jogger on this spatial-functional account. When the jogger puts on the sweatshirt, there is a mesh between the jogger and the sweatshirt: Wherever the jogger goes, the sweatshirt goes too. Then, later facilitation in reading "sweatshirt" is not due to spatial closeness of the jogger and the sweatshirt, but their functional relatedness. On this account, the texts used by Langston et al. (1995) resulted in encoding patterns of action between the reader (projected into the situation) and each object (A, B, C, and D). Given that spatial layout is not encoded directly, there is little reason to suspect that availability of object A will depend on its spatial distance from object D. In other words, spatial distance only matters when it corresponds to functional distance.
The proposal that embodied mental models reflect a structured space (that is, a space structured by possible actions) rather than a uniform space, is consistent with several research programs. McNamara (1986) and McNamara, Hardy, and Hirtle (1989) adduce evidence that spatial memory is structured and perhaps hierarchical. Bryant, Tversky, and Franklin (1992) argue that the time needed to answer questions about memorized spatial layouts reflects an embodied encoding. They find that retrieval of information aligned on the head/feet axis is faster than for the front/back axis which is faster than for the left/right axis. They interpret these differences as reflecting asymmetries of the body.
If language comprehension is in terms of meshed action, how is it that we come to understand abstract language that is not about concrete objects or situations? Here I adopt a version of Lakoff's (1987) spatialization of form hypothesis. Namely, we understand abstract situations by conceptualizing them in concrete ways.
Talmy's (1988) analysis of force dynamics is a good example of how abstract concepts can be given a bodily interpretation. He notes that we can conceptualize forces as one entity (an agonist) acting against another (an antagonist) and that the entities may have different strengths and different tendencies (either toward action or toward inaction). Importantly, these basic entities and relations can be based on bodily experiences such as pushing and being pushed, moving objects, etc. Talmy suggests that our understanding of causal terms (e.g., "because") reflects an agonist's tendency (toward action or inaction) being overcome by an antagonist. Thus, we understand the sentence "The ball kept rolling because of the wind blowing on it" as an agonist (the ball) with a tendency toward inaction being overcome by the stronger antagonist, the wind. Talmy also demonstrates how this analysis can be extended to psychological instances of causation, social references, and interpretation of modals such as can, may, must, and should. Thus, interpretation of "John cannot leave the house" comes about from assigning John the role of an agonist whose actions are blocked by the unmentioned but stronger antagonist of social or physical constraint. In the case of "should not" the antagonist is a value or belief, and so on. The point is that what has traditionally been treated as prototypically abstract (e.g., cause, force, modality), can be conceptualized in embodied terms, and in so doing brings out important similarities in our understanding of these concepts.
Bowerman (1982, 1985) discusses a number of cases of children's late speech errors that imply an understanding of the more abstract in terms of the concrete. Bowerman classifies an error as a late speech error when it occurs after a linguistic form has been used correctly and when the error does not mirror adult usage. She argues that given these constraints, the error arises from an over-extension of the adult-sanctioned relation between domains. Typically, the spatial domain is extended, so that the pattern of errors is asymmetrical across domains. For example, children import spatial terms into other domains, but rarely vice versa. Bowerman reports that children use the spatial verbs "put" and "take" to describe state changes, such as "put the door locked." Also, it is commonplace to use spatial terms when describing time, e.g., "the week before," and "between spring break and finals week." Is this just a convention, or does it reflect a conceptualization in which we understand time by using spatial dimensions? The late error, "behind the dinner" for "after the dinner" would seem to imply the latter (Bowerman, 1982, 1985). Finally, Bowerman describes the use of spatial terms to speak of non- spatial dimensions, such as looseness of teeth ("They're all the same length of loose") and temperature of water ("I want it the same size as Christy's was").
One final example should suffice. Suppose that we conceptualize abstract trait information (e.g., that Marta is energetic), as a meshing between the person and the trait. That is, the actions that Marta might perform are meshed with "energetic" so that her actions are constrained to be energetic. To test this notion, Fernandez and Saiz (1989) had subjects read texts describing the association or dissociation of a main character and a trait. In a text about Marta, an expert in international business, the critical sentences in the associated condition read (in translation from the Spanish):
(1) She has just been appointed to a government position. Almost everybody considers her an especially energetic person.
Whereas the critical sentences in the dissociated condition read:
(2) She has just been appointed to a government position. Almost nobody considers her an especially energetic person.
After reading one or two filler sentences in which Marta was kept foregrounded (but her energy never mentioned), accessibility of "energetic" was evaluated by speeded recognition of the probe "energetic." On average, responding in the associated condition was over 100 msec faster than responding in the dissociated condition. Thus, readers may well have been conceptualizing abstract trait information as embodied and meshed with an embodied conception of Marta.
Some texts make sense; others do not. The ones that make sense are judged coherent. But, what produces that sense of coherence? A standard answer is that it arises from the connectedness of the psychologists' propositions underlying the text; when the propositions are connected (or can be made connected through bridging inferences, Haviland & Clark, 1974) then the text is coherent. When the propositions do not connect, either bridging inferences need to be made to connect them, or the text will appear incoherent.
This interpretation of coherence is wrong in several respects (see Sanford and Moxey, 1995). Importantly, the account is wrong because whether or not propositions connect and how they connect depends first on interpreting the propositions against a situation. Consider the following example adapted from Sanford and Moxey:
(3) While measuring the wall, Fred laid the sheet of wallpaper on the table. Then he put his mug of coffee on the wallpaper.
(4) After measuring the wall, Fred pasted the wallpaper on the wall. Then he put his mug of coffee on the wallpaper.
A propositional analysis does not reveal that (4) is odd, and thus a propositional analysis cannot indicate "local incoherence" and cannot trigger bridging inferences to maintain coherence (see also O'Brien & Albrecht, 1992). Noticing that (4) is odd arises from a consideration of the situation, that once the wallpaper is on the wall, under normal conditions, it cannot not support a mug of coffee. To state it differently, coherence is a relationship among ideas, and texts do not have ideas, only readers do.
An impressive counter to the claim that coherence derives from connecting propositions can be found in Barton and Sanford (1993). Their subjects read about an airplane crash that occurred in the Pyrenees between France and Spain. The subjects were asked for advice on where the survivors should be buried. In fact, the subjects readily offered advice; that is, they understood the text, judged it as coherent, and were ready to suggest where the survivors should be buried. Nonetheless, only about 60% of the readers noticed that "survivors" are not buried. In a second experiment, when readers were asked where to bury the "surviving dead," only 23% noticed a problem. Clearly, the readers were not forming propositions and checking them for sensibility, because "surviving dead" cannot make a sensible proposition.
An alternative account of coherence is twofold. First, coherence is a matter of degree, and in fact, no bit of language is completely incoherent. Second, the degree of coherence can only be computed from the mesh of a situational representation of what the language is about.
The claim that no bit of language is completely incoherent rests on the analogy between understanding language and understanding the environment. Consider, for example, a percept of a drawing of an "impossible object." There may be no three-dimensional object that could project that two-dimensional outline. Nonetheless, the percept is not incoherent; the percept is of a drawing that has no corresponding three-dimensional realization. Percepts may be unusual or bizarre, but never incoherent because the perceptual/action system is designed to transduce patterns of possible interaction. Similarly, a random collection of words (or even phonemes or features) will be perceived coherently, perhaps as chirps and whistles, and a random collection of sentences will be perceived coherently (correctly) as a random collection of sentences.
Nonetheless, we do get the sense that some collections of sentences are not random. Sentences cohere to the extent that they produce continuous transformations (trajectories) of a meshed set of possible actions. Consider (4) again. The second sentence seems incoherent in that it cannot be incorporated into the the standard situational interpretation of flat wallpaper on a vertical wall in a gravitational field. However, if the initial model is changed so that any of these presuppositions about the situation are eliminated (e.g., the wallpaper has niches in it, the wall is not yet vertical because it will be incorporated into a doll's house, etc.) then the sentences are coherent. Another example is also adapted from Sanford and Moxey:
(5) John ate a banana. The banana was brown. Brown is a good color for hair. The hair of a dog is drink to counteract a hangover.
Sanford and Moxey use this snippet of text to illustrate that sentences that incorporate cohesion markers (e.g., anaphoric reference) can, nonetheless, be judged incoherent. The problem is that the sentences do not update a mental model. That is, the patterns of action suggested by each sentence do not admit to smooth transformation of the mesh from one sentence to the next. Note, however, that as with the previous example, a change in the initial situation can render the sentences (more) coherent. Imagine that John engages in free association whenever he eats fruits. Then, the list of sentences, as descriptions of his free-associations, seem (more) coherent. Similar examples can be constructed for film (e.g., the sequence of cuts seem incoherent unless one has the appropriate model of the film) and for events in the world (e.g., changes in the weather seem incoherent unless one has the appropriate model of weather systems). In short, coherence is a property of models (the ideas that people have), not a property of snippets of language.
As a final example, (6) was taken from the abstract of a talk given in a Computer Sciences seminar at the University of Wisconsin- Madison.
(6) The talk will concentrate on the design of the communications subsystem [of the Meiko CS-2 MPP System]. This utilizes a 'fat tree' network constructed from high performance crosspoint switches. Processing Elements interface to this network via a communications co-processor which contains intelligence to handle virtual addressing and ensures very low message start up times.
This text may well be very coherent for its intended audience, but it is at the low-end of the dimension for me. The problem is not that the propositions do not connect. The propositional relation between 'fat tree' networks and crosspoint switches is virtually transparent; similarly, it is quite clear that a co-processor intervenes between the "Processing Elements" and the network. The problem is that I do not know what a 'fat tree' network is, or what "crosspoint switches" or "Processing Elements" are. I do not know the literal shapes of these things, nor do I know the actions they can take or how I can interact with them. Because I lack that knowledge, I cannot build a coherent spatial-functional model. Presumably, crosspoint switches can be arrayed or interconnected in some way so that they comprise a 'fat tree' network. But for me, the mesh is missing.
The ideas a) that coherence is a function of the mesh in an embodied model, b) that the embodied models constructed to understand language are the same as those that underlie comprehension of the natural environment, and c) that the purpose of perception and memory for the natural environment is to guide action, all lead to a suggestion about how to assess comprehension. Most laboratory comprehension tests require verbatim reproduction of a text, reproduction of "idea units," or speeded responding to words or phrases. A more sensible comprehension test, however, is one that requires action. To what extent can the reader take sensible action (or make sensible predictions) on the basis of the text? (6) is relatively incoherent for me because I can make so few predictions. For example, if the type of switches were changed, I don't know if that would change the network from a 'fat tree' network to some other kind; if the communications co-processor was not intelligent, I do not know if the message start up times would be slower or faster. On the other hand, (6) is not completely incoherent because there are some predictions that I can make. For example, based on knowledge of part- whole relations, I can predict that if the crosspoint switches are eliminated, there will be no 'fat tree' network.
I began with a consideration of the Lakoff and Johnson program and the problem of meaning. In applying their insights to a theory of memory and mental models, the concept of embodiment becomes central. The basic claim is that an individual's memory serves perception and action. Memory meshes non-projectable features with projectable features of the environment to suggest actions for that person in that situation. These patterns of action are what make the environment meaningful to that person. This framework provides a way to address meaning, symbol grounding, recollective and automatic uses of memory, and language comprehension.
The framework provides alternative accounts of standard phenomena and it makes new predictions. Here is a brief review. The concept of embodied knowledge is used to address the problem of meaning and symbol grounding (Section 1.3), why people see the world differently (1.3), effects of bodily activity on emotions (2.3.1), imagery (2.3.2), memory for actions (2.3.3), sensibility judgments (2.3.3), short-term behavior (4.3), and understanding in abstract domains (6.4). Mesh of patterns of action is applied to emergent features of thought (3.1), recollective memory (5.2), interactive imagery (5.2.1), interpretation of semantic priming phenomena including forward inferencing (6.2), and coherence (6.3). Suppression of projectable properties of the environment is seen as critical to multi-step prediction (3.4), the feeling of memory (4.0), the decrease in physical activity when thinking (5.2), amnesia (5.2.4), correlation of language comprehension with recollection (6.0), and effects of incidental patterns on comprehension (6.0). Finally, trajectories are applied to frequency effects in memory (3.3), the nature of rehearsal (4.3), automatic uses of memory (5.1), and expertise (6.1.1).
Can embodied patterns of action underlie all conceptualization? Our experiences of music, taste, and emotions all seem to have aspects that do not fit well into a spatial-functional straitjacket, and one suspects that aspects of these experiences are represented in addition to action patterns. Nonetheless, given the ease with which these sorts of experience combine with spatial-functional experience (consider the contribution of music and mood to the understanding of the action depicted in a film), it is not inconceivable that they may eventually be covered by the same sort of analysis.
Missing from the discussion is a consideration of hedonic valence and motivation to act. It is not as yet clear how pleasure and pain should be represented in an action-oriented system (but see Lang, 1979). What is clear, however, is that hedonic valence affects action and how experiences become meaningful. Our understanding of pleasurable experiences is in part action-toward those experiences, whereas our understanding of aversive experiences is in part action- away. Several ideas follow. Given that action-away does not necessarily specify what the action is directed toward, it ought to be more diffused and variable that action toward. Also, on this analysis, approach and withdrawal are not poles of a single dimension: Withdrawal from one situation does not imply approach toward another. Thus, our understanding of emotional experience should reflect at least two dimensions (e.g., Schneirla, 1959).
Malter (1996) applies these ideas to consumer research, in particular, impulse buying. He proposes that projectable features of a product automatically mesh with affectively charged memories (perhaps imparted by advertisements) to produce an irresistible approach-dominated conceptualization. Thus the consumer experiences a strong desire to approach and manipulate the object, and in most cases that can only be accomplished after purchase. Furthermore, Malter notes that overcoming this urge to buy requires effortful suppression of the projectable features in order to deliberately evaluate the purchase. In the face of a strong impulse to buy, however, that effort may be viewed as unattractive or not considered at all.
There is also reason to believe that an embodied, action-oriented analysis has implications for social psychology. Fiske (1992) traces the history of action-oriented theories of social cognition from James ("My thinking is first and last and always for the sake of my doing..." as quoted in Fiske) to Heider (1958) to current "pragmatic" research. Fiske defines pragmatism as a framework in which "meaning, truth, and validity are determined by practical consequences [and] concrete goal-relevant actions," (p. 886).
Fiske's own analysis of social cognition is compatible with the ideas I have described, and her analysis suggests an important extension. According to Fiske (and Heider, 1958), the key to social cognition is to view others not just as objects that we can affect, but as beings who can effect us in turn. Consonant with this premise, Fiske proposes that our ability to infer traits is in the service of interaction with others.
Among the interpretations that stem from a consideration of embodied representations, the notion of mesh seems most important. Ideas mesh to the extent that the pattern of action underlying one idea can be integrated with the pattern of action underlying another. The patterns mutually modify and constrain one another because the conjoint actions must be possible given our bodies. This mutual modification of patterns of action is what underlies the construction of meaning from words whose senses are jointly modified by the contexts in which they occur.
Meshing patterns of action provides a new way of thinking about componentiality and productivity in language. As an example, consider the coke bottle. Its shape, and thus its affordances for human action, allow it to mesh with many physical situations and goals. It can be used for storing liquid, as a cup, a doorstop, a weapon, a vase, and so on. Thus the meaning of a coke bottle (how we can interact with it) is not fixed, but infinitely varied, depending on the context of use. Importantly, however, the meaning is in no way arbitrary or unconstrained: The meaning of the bottle is constrained by its shape (heft, fragility, etc.) and the implications of that shape for action. Thus the spatial-functional meaning of a coke bottle is componential in that it will mesh with many human contexts. Because that mesh can transform the meaning, however, its use is creatively productive.
This type of componentiality helps us to understand what Barsalou et al. (1993) term "linguistic vagary." When people are asked to describe the features of a category such "coke bottle" there is tremendous variability both across people and from time to time in a particular person's descriptions. Linguistic vagary should be the norm if the meaning of a concept is determined by its mesh with the context.
The idea of mesh may prove to be a concept that can replace "association." Although association has played a central role in theories of cognition, the term carries little theoretical weight. What we mean by an association is little more than a conditional probability; if B is associated with A, then P(B|A) > P(B). There is little or nothing in our theories to help us understand when "laws of association," such as frequency and recency, hold, and when they do not. In contrast, the notion of mesh can, like an association, be used to relate concepts, but the nature of the relation is deeper: When patterns mesh, they modify each other because they must conjoin in a way that respects constraints on bodily action. Thus, "coke bottle" is difficult to mesh with "chair."
Mesh provides a rationale for Thorndike's (1932) concept of belongingness (see also, Ohman, Fredrikson, Hugdahl, & Rimmo, 1976) as well as various ideas put forward by Gestalt psychologists. Furthermore, mesh may help to explicate species-specific differences in associability of stimuli. Rats find it easier to associate a novel taste with illness than to associate a novel sight with illness (Garcia & Koelling, 1966). In contrast, pigeons find it easier to associate a novel sight, rather than a novel taste, with illness (Wilcoxin, Dragoin, & Kral, 1971). If learning comes about through meshing patterns of bodily action, then, given species differences in anatomy, physiology, and possible actions, the fact that stimuli will mesh differently for different species is a foregone conclusion.
If knowledge is embodied, then commonly-used laboratory paradigms for studying memory may well be missing the mark. Many of these paradigms use random lists of words as the objects of memory. Whereas there are reasons for using lists of words, these are reasons related to history and convenience, not to any analysis of the design of memory. A favorite argument to justify the verbal list format is that each word corresponds to a mini-event, and from memory's point of view these mini-events are similar to other events in the world. This argument looses much of its force, however, if memory is embodied and designed for negotiating a three-dimensional environment.
The discrepancy between the design of memory and the design of the tools used to analyze it may account for undesirable characteristics of memory research. Importantly, memory researchers have not made much progress in understanding the nature of memory. We know about many phenomena (Greene, 1992), but there is little agreement as to the interpretation of those phenomena, how they fit together, or whether a particular phenomenon is of any importance. Even with something as basic as the effect of repetition, the theoretical diversity is astounding: We have theories in which repetitions enhance the strength of a single representation (Gillund and Shiffrin, 1982), theories in which repetitions are individually preserved (Hintzman, 1986), and theories which treat memory much like a hologram (Metcalfe, 1993). We have multi-store theories and single-store theories; single system theories and multiple-system theories. All of these positions receive support from some aspects of the literature. I suspect that this diversity of positions arises because in using inappropriate tools we obtain incompatible views of memory much like the views of the blind men touching the elephant.
Baddeley, A. D., & Hitch, G. J. (1974). Working memory. In G. Bower (Ed.), The psychology of learning and motivation, Vol. 8 (pp. 47-89). New York: Academic Press.
Barsalou, L. W. (1993). Flexibility, structure, and linguistic vagary in concepts: Manifestations of a compositional system of perceptual symbols. In A. C. Collins, S. E. Gathercole, & M. A. Conway (Eds.), Theories of memories (29-101). London: Erlbaum.
Barsalou, L. W., Yeh, W., Luka, B.J., Olseth, K. L., Mix, K. S., & Wu, L-L. (1993). Concepts and meaning. In K Beals, G. Cooke, D., Kathman, K. E. McCullough, S. Kita, & D. Testen (Eds.) Chicago Linguistics Society 29: Papers from the parasession on conceptual representations. University of Chicago: Chicago Linguistics Society.
Barton, S. B., & Sanford, A. J. (1993). A case study of anomaly detection: Shallow semantic processing and cohesion establishment. Memory & Cognition, 21, 477-487.
Berkowitz, L., Jo, E., & Troccoli, B. (1993). Feeling and knowing: Evidence for different psychological processes in emotional reactions. Manuscript in preparation.
Berkowitz, L. & Troccoli, B. T. (1990). Feelings, direction of attention, and expressed evaluations of others. Cognition and Emotion, 4, 305-325.
Blaxton, T. A. (1989). Investigating dissociations among memory measures: Support for a transfer-appropriate processing framework. Journal of Experimental Psychology: Learning, Memory and Cognition, 15, 657-668.
Bloom, L. (1994). Meaning and Expression. In W. F. Overton and D. S. Palermo (Eds.), The nature and ontogenesis of meaning (pp.215- 235). Hillsdale, New Jersey: Lawrence Erlbaum Associates.
Bower, G. H. (1970). Analysis of a mnemonic device. American Scientist, 58.
Bowerman, M. (1982). Starting to talk worse: Clues to language acquisition from children's late speech errors. In S. Strauss (Ed.), U-shaped behavioral growth (pp. 101-145). New York: Academic Press
Bowerman, M. (1985). The spatial metaphor: A universal principle of semantic structuring? Clues from children's speech errors. Oxford University Psychology Department Colloquium.
Bransford, J. D. (1979). Human Cognition: Learning, Understanding and Remembering. Belmont, CA: Wadsworth Publishing Company.
Bruce, V, & Green, P. R. (1985). Visual perception : physiology, psychology and ecology. Hillsdale, N. J.: Erlbaum.
Bryant, D. J., Tversky, B., & Franklin, N. (1992). Internal and external spatial frameworks for representing described scenes. Journal of Memory and Language, 31, 74-98.
Carpenter, P, A., Miyake, A, & Just, M. A. (1994). Working memory constraints in comprehension: Evidence from individual differences, aphasia, and aging. In M. A. Gernsbacher (Ed.), Handbook of psycholinguistics. Orlando, FL: Academic Press.
Cohen, R. L. (1981). On the generality of some memory laws. Scandinavian Journal of Psychology, 22, 267-281.
Collins, A. M., & Quillian, M. R. (1969). Retrieval time from semantic memory. Journal of Verbal Learning and Verbal Behavior, 8, 240-247.
Conrad, C. (1972). Cognitive economy in semantic memory. Journal of Experimental Psychology, 92, 149-154.
Craik, F. I. M. & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671-684.
Cuthbert, B. N., Vrana, S. R., & Bradley, M. M. (1991). Imagery: function and physiology. In Advances in psychophysiology.
Denis, M., & Cocude, M. (1989). Scanning visual images generated from verbal descriptions. European Journal of Cognitive Psychology, 1, 293-307.
Engelkamp, J., & Krumnacker, H. (1980). Imaginale und motorische Prozesse beim Behalten verbalen Materials. Zeitschrift fur experimentelle und angewandte Psychologie, 27, 511-533.
Epstein, W. (1993). The representational framework in perceptual theory. Perception & Psychophysics, 53, 704-709.
Fernandez, A., & Saiz, C. (1989, July). Foregrounding of trait information during text comprehension. Paper presented at the First European congress of Psychology, Amsterdam, NL.
Fiske, S. T. (1992). Thinking is for doing: Portraits of social cognition from daguerrotype to laserphoto. Journal of Personality and Social Psychology, 1992, 63, 877-889.
Franklin, N., Tversky, B., & Coon, V. (1992). Switching points of view in spatial mental models. Memory & Cognition, 20, 507-518.
Friedman, W. J. (1993). Memory for the time of post events. Psychological Bulletin, 113, 44-66.
Garcia, J., & Koelling, R. A. (1966). Relation of cue to consequence in avoidance learning. Psychonomic Science, 4, 123-124.
Gdrdenfors, P. (1995). Cued and detached representations in animal cognition. Man