To be published in Behavioral and Brain Sciences (in press)
© Cambridge University Press 2003

Separate Visual Representations in the Planning and Control of Action

Word Count
Abstract: 151
Main Text: 20,345
References: 4,698
Entire Text: 25,394

Scott Glover
Dept. of Experimental Psychology
University of Oxford
South Parks Road
Oxford, UK
OX1 3UD

Email: scott.glover@psy.ox.ac.uk

Short Abstract: Evidence for a dichotomy between the planning and on-line control of actions in humans is reviewed. It is hypothesized that planning uses a richer visual representation than does control, but that control is faster and more adaptable. Evidence from brain imaging and neuropsychology suggests that planning utilizes a network including the inferior parietal region, whereas control utilizes a network including the superior parietal region.

Long Abstract: Evidence for a dichotomy between the planning of an action and its on-line control in humans is reviewed. This evidence suggests that planning and on-line control each serve a specialized purpose different from the other, and utilize distinct visual representations. Evidence from behavioral studies suggests that planning is influenced by a large array of visual and cognitive information, whereas control is influenced solely by the spatial characteristics of the target, including such things as its size, shape, orientation, etc. Evidence from brain imaging and neuropsychology suggest that planning and control are subserved by separate visual centers in the posterior parietal lobes, each constituting part of a larger network for planning and control. Planning appears to rely on phylogenetically newer regions in the inferior parietal lobe, along with the frontal lobes and basal ganglia, whereas control appears to rely on older regions in the superior parietal lobe, along with the cerebellum.

Keywords: action, apraxia, control, illusions, optic ataxia, planning, PET, reaching

This paper explores the evidence for a distinction in human performance between the planning and on-line control of actions. The planning-control model is put forth as an explanation for human action production. A review of studies from healthy subjects reveals differences between the visual and cognitive processes involved in planning and control. Brain imaging studies support the dichotomy, in that planning in humans is linked with activity in a distributed network including a visual representation in the inferior parietal lobe (IPL) whereas control is linked with activity in a separate network including a visual representation in the superior parietal lobe (SPL). Studies of brain-damaged populations also support the thesis that separate brain regions support planning and control. A review of evidence from brain and behavior suggests that the planning-control model provides an account of the data superior to a model based on a distinction between perception and action (Milner & Goodale, 1995).

Woodworth (1899) was the first to propose a distinction between the planning and control stages of action, based on his seminal study examining the use of visual feedback in on-line control. Since Woodworth’s time, the distinction between planning and control has been the subject of much investigation (e.g., Beggs & Howarth, 1970, 1972; Carlton, 1981; Fitts, 1954; Keele, 1968; Meyer et al., 1988; Vince, 1948; see Elliott et al., 2001, for a review) and the existence of these two stages has generally become accepted as an underlying principle of human motor behavior (Jeannerod, 1988; Rosenbaum, 1991).

We have expanded on the planning and control distinction (Glover, 2002; Glover & Dixon, 2001a, 2001b, 2001c, 2002a, 2002b; Glover et al., 2002) to include separate visual representations in each stage of action. In our "planning-control" model, body movements are selected and executed by means of two temporally overlapping systems. Prior to a movement’s initiation, a motor program is selected based on a broad range of cognitive factors coupled with a visual "planning" representation in the IPL. During execution of a movement, the action comes increasingly under the influence of a "control" system, using a limited but quickly updated visual representation in the SPL, coupled with visual and proprioceptive feedback, and an efference copy of the movement plan.

1.1.1. The need for separate planning and control systems. In the planning-control model, the planning system generally operates prior to a movement whereas the control system operates during execution. The quasi-separation of these two stages is hypothesized to reflect the need of each system to fulfill distinct purposes. For planning, the requirement is to select an adaptive motor program given the environment and the goals of the actor. This will depend on a number of factors related to the object of the action, the surrounding environment, and the internal state of the actor. Conversely, for control, the requirement is the minimization of the spatial error of the movement. This requires a relatively simple but quickly updated analysis of the spatial characteristics of the target and actor. The next two sections expand on the characteristics of the planning and control systems.

1.1.2. The planning system. The planning system has the goal of selecting and initiating an adaptive motor program, given the environment and the goals of the actor. At a high level, planning is responsible for such things as selecting an appropriate target, or choosing to grasp an object in a certain manner. Beyond these selection processes, however, planning also determines the initial kinematic parametrization of the movements, including their timing and velocity.

To fulfill its aims, planning must take into account a wide variety of visual and cognitive information. This information can be classified into four basic aspects of the environment and actor: 1) the spatial characteristics of the actor and the target, including such things as the size, shape, and orientation of the target, as well as the spatial relations between the actor and the target; 2) the non-spatial characteristics of the target, including such things as its function, weight, fragility, and the coefficient of friction of its surfaces; 3) the overarching goal(s) of the action; and 4) the visual context surrounding the target. This information is integrated with memories of past experiences (cf. Rosenbaum et al., 1995). Table 1 lists object characteristics important for action as belonging to either the spatial or non-spatial class, as well as the movement parameters most dependent on each characteristic. The position of the effector is always considered to be a spatial characteristic.

It is important to point out that non-spatial object characteristics are not entirely visual. That is, whereas the spatial characteristics of objects tend to be geometric properties that can be gleaned from low-level visual processes alone, the non-spatial characteristics invariably necessitate reference to stored memories. For example, knowing that a tomato is more fragile than an apple requires that each be identified and these properties called up from memory. The identification process may require the incorporation of spatial information such as the target’s shape and size with other information such as color. Similarly, knowing that an iron bar is heavier than a wooden bar of the same size requires not just being able to identify the material, but also to judge the similarity in sizes of the two objects. Thus, an integration of spatial characteristics with information about an object’s identity is usually required in order to compute a non-spatial characteristic. However, the reverse is not also true: nonspatial characteristics are not required to compute spatial characteristics.

As mentioned, the planning system specifies the majority of the macroscopic and microscopic aspects of the action before initiation. Specifically, planning is responsible for selecting the target; for all movement parameters relating to non-spatial target characteristics; for the initial determination of the movement parameters relating to spatial target characteristics (although these can be modified on-line by the control system); for determining the timing of movements (including reaction times, movement times, and acceleration/velocity parameters); and for the selection of macroscopic (i.e., postural) aspects of the movement.

The integration of such a broad range of information and the computation of a broad range of movement parameters by the planning system requires a relatively long processing time. Further, these varied sources of information can interact, sometimes interfering in the selection of adaptive motor programs. For example, contextual figures can induce visual illusions (Coren & Girgus, 1978; Gregory, 1968) that will impact the computation of both spatial and non-spatial target characteristics in the planning system. This means that visual illusions will have large effects on spatial parameters early in a movement, and large effects on non-spatial parameters over the entire course of the movement. Further, as planning relates closely to cognitive processes, the planning representation will rely on, and be subject to interference from, processes such as language and memory. Again, interference effects of these variables will be evident when they interact with the computation of characteristics of the target. Further, because planning relies on a store of memories of past experiences in selecting an appropriate plan, it will be less able to plan precise movements when the situation is unfamiliar, such as when a novel object is the target, as compared to when the target is familiar or the movement is well practiced. Finally, the interaction of planning with cognitive processes will result in planning being at least somewhat susceptible to conscious influence.

The general operation of the planning system can be illustrated with an everyday example. Imagine an actor with the aim of satisfying his thirst. To achieve this goal, the planning system may select a reach-to-grasp movement directed towards a glass on a shelf. It will first select a glass of an appropriate size depending on how thirsty the actor is (overarching goals). The action of grasping the glass will serve the ultimate purpose of filling it with water and drinking out of it (overarching goals). The planning system will choose to grasp a thin glass with less force than a thick glass because the former is more fragile (non-spatial characteristics of the target). It will determine a path that will avoid contacting obstacles along the way to the glass (visual context). It will plan a fairly accurate movement towards the glass, taking into account its position relative to the effector, its size, its shape, etc. (spatial characteristics of the actor and target). Finally, planning will time the movement such that a sufficient period of time is available for the control system to operate. When a motor program is selected that satisfies these constraints, the planning system will determine when to initiate the movement. Once the movement has been selected and initiated, it will come increasingly under the influence of the control system.

1.1.3. The control system. Whereas the planning system is generally adept at selecting an appropriate motor program given the environment, the control system allows the added benefit of monitoring and occasionally adjusting motor programs in flight. These adjustments are limited to the spatial characteristics of the target, as these are the most likely to change or to be erroneously planned. Specifically, spatial errors may arise either from how the movement was planned (e.g., from interference due to cognitive influences), or during execution of the plan (e.g., due to noise in the neuromuscular system). Also, the spatial characteristics of the target may change in unanticipated ways (e.g., a fruit on a branch blowing in the wind). In contrast, the non-spatial characteristics of the target (such as its weight or function) are almost completely unlikely to change after the movement is planned.

The vicissitudes of the spatial characteristics of the actor and target, along with interference from cognitive or perceptual variables, will make it difficult to always plan a spatially accurate movement. Yet this is of little consequence to the overall adaptiveness of the action. This is because the ultimate success of the movement depends much more on how accurate it is at its end, not on how accurate it was when it was planned. In other words, errors in planning can still be corrected given sufficient time; errors in control are much more likely to cause the act to fail.

To ensure that the movement is spatially accurate, the control system requires a quickly computed visual representation. The speed of processing in this representation is gained by limiting it to the spatial characteristics of the target. This control representation is coupled with visual feedback, proprioception, and efference copy (i.e., a "blueprint" of the movement plan obtained from the planning system prior to initiation). The limit of the control representation to spatial characteristics naturally limits its influence to the spatial parameters of the movement. These include such things as grip aperture, hand trajectory, and hand orientation (see Table 1). As planning and control overlap in time, the influence of control on the spatial parameters becomes increasingly large as the movement unfolds.

Being limited to the spatial characteristics of the actor and target has the benefit of allowing for fast processing and similarly fast on-line adjustments by the control system. Further, the spatial analysis that takes place in the control representation is immune to the interference of such variables as the visual context or cognitive processes. The independence of the control system from such cognitive processes as goal formation and conscious perception also means that control operates outside of conscious awareness and influence.

The visual representation guiding control is transient in nature. This allows for it to be quickly updated as a movement unfolds, but also means that its memory is of short duration. When visual information regarding the effector or target is removed either prior to or during a movement, the control representation will begin to gradually decay over a period of roughly two seconds (cf. Elliott & Madalena, 1987). The decay of the control representation is gradual rather than instantaneous, and during delay periods of less than two seconds, a less dramatic reduction in the influence of control will result (Westwood et al., 2001a). However, when the delay is more than two seconds, the decay will be nearly complete, and movements made after delays much longer than two seconds will be executed entirely "as planned" (i.e., without the benefit of on-line control).

Completing the above example of grasping a glass illustrates the operation of the control system. After the planning system has selected and initiated an appropriate movement directed towards the glass, the control system will be responsible for minimizing any spatial error in the movement itself (spatial characteristics of the target and actor). These might include the scaling of the hand to the size of the glass, or the orienting of the wrist to the appropriate angle. Control will generally ignore any objects surrounding the glass (visual context), the intention to use the glass to drink from (overarching goals), and the fragility of the glass (non-spatial characteristics of the target). Put simply, the control system is focused on the on-line correction of the spatial parameters of the action.

1.1.4. The time course of planning and control. As mentioned above, the two stages of action are temporally overlapping (cf. Desmurget & Grafton, 2000; Wolpert & Ghahramani, 2000; Wolpert et al., 1995; but see Crossman and Goodeve, 1983; Meyer et al., 1988). Prior to movement initiation, planning is entirely responsible for the initial determination of all movement parameters, and continues to be highly influential early in the movement. As movements progress, however, the influence of control on the spatial parameters of the action increases. Such a gradual crossover between planning and control systems has the benefit of allowing for smooth rather than jerky corrections (cf. Wolpert & Ghahramani, 2000).

As planning is generally operative early in a movement and control late in a movement, the influence of each will rise and fall as the movement unfolds. For example, peak grip acceleration, a size-dependent parameter that occurs at roughly 35% of movement duration (Jakobson & Goodale, 1991; Jeannerod, 1984), will reflect planning more than control. In contrast, maximum grip aperture, a size-dependent parameter that occurs at roughly 70% of movement duration (Jakobson & Goodale, 1991; Jeannerod, 1984) will reflect control more than planning. Similar applications can be made to other parameters dependent on spatial characteristics: The orientation of the hand early in a reach should reflect mainly planning, late in a reach it should reflect mainly control, etc..

Because control relies to a significant degree on visual and proprioceptive feedback loops, the more time in which these loops can operate, the greater the influence control will exert. That is, whenever actions take longer or shorter than a stereotypical reach-to-grasp movement, the influence of control will be extended or shortened, respectively. In long-duration movements such as manual tracking or catching a long fly ball, for example, almost the entire movement will be under the influence of control. In contrast, for short-duration movements such as keyboarding, practically the entire movement will be executed as planned.

The planning-control model can also be applied to more complex and continuous movements, such as playing tennis or running over rough terrain. In these movement sequences, planning and control processes would alternate with each other in succession. For example, the intention to swing at a ball in tennis and the initial parametrization of the muscle contractions underlying the swing will be pre-planned, whereas the execution of the swing will be controlled on-line. The ensuing movement of rushing towards the net to anticipate the opponent’s return will require a different set of muscle movements, and a new plan. The planning of this action may begin even while the control process is monitoring and adjusting the initial swing at the ball. In this way, the planning system can use the times in which control is operating to select and parametrize the next movement in the sequence.

Running over rough terrain would similarly engage both planning and control mechanisms. Whereas each step or series of steps could be pre-planned, the execution of the steps would be controlled on-line. Given the difficult nature of the task, and the need to make adjustments based on unanticipated events (such as stepping on a rock that throws the actor off balance), such a task would most likely tax the control system quite heavily. Generally speaking, the more the outcome of a given movement or series of movements is predictable, the more it will rely on planning and the less on control. Conversely, when the consequences of movements are unpredictable, or when unanticipated forces act on the body or target, planning will give way more and more to control processes.

1.1.5. Limitations of the planning-control model. The planning-control model is designed to predict and explain body movements. It is not meant to generalize to eye movements. This limitation is necessary due to the very different constraints that apply to each type of movement. For example, body movements involve complex physical transitions through three-dimensional space. These transitions require consideration of objects in the visual context as potential obstacles. Conversely, eye movements involve relatively simple (in terms of muscle activations) rotations of the eyes in the orbits in which collisions with the visual context are extremely unlikely, if not impossible. Because of these different constraints, the planning-control model is limited to movements of the body, although how eye and body movements are coordinated is undoubtedly an important issue.

In the planning-control model, the two stages of action utilize distinct neural networks in the human brain. Planning involves the use of a visual representation located in the IPL, coupled with motor and related cognitive processes in the frontal lobes and basal ganglia. Control, on the other hand, involves a visual representation located in the SPL, coupled with motor processes in the cerebellum.

1.2.1. Evolution of the parietal lobes and a greater separation of planning and control in humans. The evolutionary divergence of humans and monkeys has coincided with a significant relative enlargement of the parietal lobes in humans (Figure 1). According to the planning-control model, this expansion has allowed humans to integrate a vast array of visual and cognitive information into an action plan. Whereas both planning and control appear to exist (though not necessarily together) in the IPL of monkeys, in humans I contend that planning is largely the province of phylogenetically newer cortex in the IPL, whereas control is largely the province of the phylogenetically older SPL.

Figure 1. Comparison of the human (top) and macaque (bottom) brains. Cortical surfaces of the left hemispheres of both species are shown. Note the large area of expansion within the parietal lobes of the human brain compared to the monkey. Labeled in the figure of the human brain are the two regions of the parietal lobe unique to humans, the supramarginal and angular gyri. Not to scale.

The role of the human IPL in action planning may have arisen quite recently in evolution and may be manifest in the uniquely human population lateralization in hand preference (Hacean & Ajuriaguerra, 1968; Harris, 1993; Hopkins, 1996). This human hand preference may have evolved from the need to have at least one highly coordinated limb that could accomplish fine unimanual manipulatory acts.

1.2.2. Distinct neural systems subserving planning and control. Figure 2 represents a schematic of the neural bases of planning and control. Prior to a plan being formed, visual input travels to the IPL via the temporal lobe and a "third" visual stream (Boussaoud et al., 1990). The temporal lobe input includes both the spatial (e.g., size, shape, orientation) and nonspatial (e.g., weight, function, fragility) characteristics of a target, as well as the visual context surrounding the target (it is less clear what the third stream contributes). Information relating to the overarching goals of the action is provided by the frontal lobes, and the frontal lobes also exert executive control (i.e., they are heavily involved in selecting the target, and in deciding ‘what’ to do as well as ‘how’ to do it). The visual and cognitive information used in planning is integrated with proprioceptive input from somatosensory association areas in the selection of an appropriate motor plan. Simple movements such as reaching and grasping tend to rely more on the IPL for movement selection and parametrization. Complex movement sequences rely more heavily on frontal lobe sequencing and timing mechanisms.

After a movement has been planned, an efference copy of the plan is forwarded to the SPL and cerebellum. This efference copy represents a "blueprint" of an upcoming action. Once the action is initiated, the control regions begin to integrate visual and proprioceptive feedback with the efference copy to monitor and if need be adjust the movement in flight. Monitoring of the body likely involves the SPL more heavily than the cerebellum. Comparing the movement with the motor plan likely involves the cerebellum more heavily than the SPL.

Figure 2. Schematic of the planning-control model showing the hypothesized connectivity and functions of the human visuomotor, cognitive, and somatosensory regions involved in action planning and control. Connections between early visual regions, and between visual and cerebellar regions are putative human homologues of connections in the macaque brain (Boussaoud et al., 1990; Glickstein, 2000; Mishkin et al., 1983). Connections beyond the termination of the two streams are based on common activation (see Section 3).

Although the distinction between planning and control stages of action has a long history (cf. Fitts, 1954; Keele & Posner, 1968; Woodworth, 1899; Meyer et al., 1988), the planning-control model has several features that make it unique among these and other models of motor control (e.g., Arbib, 1981; Desmurget & Grafton, 2000; Jeannerod, 1988; Milner & Goodale, 1995; Wolpert & Gharamani, 2000). First and foremost, only the planning-control model postulates separate visual representations underlying the two stages of action. Even other models that include planning and control stages make no such distinction (e.g. Desmurget & Grafton, 2000; Jeannerod, 1988; Wolpert & Gharamani, 2000; Woodworth, 1899). In pursuance of this, only the planning-control model assigns the inferior and superior aspects of the parietal lobes to their specific roles in computing these visual representations. Second, the planning-control model makes no specific distinction between the information used during reaching versus grasping (as was central to Jeannerod, 1988). Third, in assuming a gradual rather than discrete crossover between the two stages of action, the planning-control model differs from at least some (e.g., Crossman & Goodeve, 1983; Keele & Posner, 1968; Woodworth, 1899) models of motor control that assume that control can only begin when feedback loops have had time to close. Similarly to Wolpert et al. (1995; see also Desmurget & Grafton, 2000; Wolpert & Gharamani, 2000), the planning-control model suggests that efference copy may be used to adjust movements from any time after initiation.

Throughout the remainder of this paper, the planning-control model is contrasted mainly with the perception-action model (Goodale & Milner, 1992; Milner & Goodale, 1995). This is done partly for the sake of brevity, and partly because the two models are the most readily comparable. The three main differences between these models can be summarized as follows: First, in the planning-control model, the two stages of action utilize distinct visual representations in the IPL and SPL respectively. Conversely, in the perception-action model, both planning and control mainly utilize representations in the SPL, although certain classes of stored perceptual information may be imported from the "ventral" stream (see e.g., Haffenden & Goodale, 2000; Milner & Goodale, 1995), and the IPL "third" stream is said to play an important role in spatio-cognitive operations (Milner & Goodale, 1995).

Second, the distinct representations in the planning-control model result in interactions between cognitive and visual information in planning but not control, whereas the perception-action model suggests that vision for action should generally be immune to cognitive influences. In particular, the perception-action model suggests that parameters of movement dependent on the spatial characteristics of the target will be both planned and controlled independent of cognitive and perceptual influences.

Third, whereas both models assume a role of non-dorsal stream visual areas in target selection, and whereas the perception-action model also holds that some action planning processes require an "interaction" between streams, only the planning-control model assumes that the IPL is involved in the kinematic parametrization of all movements, not just those that require information regarding non-spatial target characteristics.

It will be seen in the ensuing review that the explanations of the planning-control and perception-action models often conflict for findings from both experimental psychology and neuroscience. For one, each predicts different effects of visual illusions and semantics on behavior. For another, each predicts different patterns of brain activations in imaging experiments. Finally, each predicts different patterns of behavior following brain damage to specific regions. It will be seen that where these conflicts exist, the planning-control model provides an account of the evidence superior to that of the perception-action model.

There is much to support the planning-control distinction in healthy subjects. Specifically, available evidence agrees with the notion that planning is a relatively slow process that is sensitive to both spatial (e.g., size, shape, orientation) and non-spatial (e.g., function, weight, fragility) visual information as well as cognitive (e.g., goals, semantics) and perceptual (e.g., visual illusions) factors. The data also support the idea that control is a relatively fast process focused on the spatial characteristics of the target and actor, relying on visual and proprioceptive feedback, along with efference copy. Critically, the data from visual illusion and semantic interference effects on planning but not control are much more consistent with the planning-control model than with the perception-action model.

Marteniuk et al. (1987) provided the classic example of how planning incorporates overarching action goals into an action plan. In this study, participants had to reach to and grasp a chip in one of two conditions. In one condition, participants were instructed to place the chip into a small hole while in another condition, participants were instructed to toss the chip into a large cup. Marteniuk et al. found that, when the goal was to "place" the chip, the requirement for a much more precise movement was reflected in longer movement times with extended decelerations, compared to when the goal was to "toss" the chip. This result supports the idea that planning incorporates overarching goals into immediate movements. Similar results have been obtained in other studies involving different types of goal specifications (e.g., Gentilucci et al., 1997; Haggard, 1998); and have also occurred when the overarching goal was two or more steps away (Rosenbaum et al., 1992).

Fitts (1954) observed that speeded pointing movements directed towards targets that were difficult to hit took longer than movements directed towards targets that were easy to hit. In the planning-control framework, this speed-accuracy trade-off ("Fitts’ Law" as it is commonly known) exemplifies how the planning system strategically accommodates the limitations of the motor system by adjusting the timing parameters of a movement. When the target is small and/or distant, planning processes are more apt to result in a large error and control processes will benefit from having more time to correct the error. In the planning-control framework, planning is hypothesized to slow down movements made to "hard" targets in order to allow the control system more time to operate.

Fitts’ aiming task has since been extensively investigated and evaluated (e.g., Beggs & Howarth, 1972; Carlton, 1981; Crossman & Goodeve, 1983; Hay & Beaubaton, 1986; Wallace & Newell, 1983). Many of these studies have found that the availability of visual feedback has a positive influence on movement accuracy, leading to a corresponding reduction in movement times, compared to when visual feedback is unavailable. The slowing of movements under conditions of reduced or absent visual feedback seems to reflect advanced planning related to the need to account for the difficulty of an upcoming movement (cf. Jeannerod, 1988).

The idea that planning is responsible for movements times is controversial. A competing explanation of Fitts’ Law is that increased movement times reflect control more than planning (e.g., Crossman & Goodeve, 1983; Meyer et al., 1988; see also Plamondon & Alimi, 1997). This explanation is based on the fact that longer movements tend to result almost entirely from an increase in the amount of time spent in deceleration. Indeed, it is clear that the lengthening of movement deceleration can either be pre-planned or reflect the need for on-line adjustments that add to the time required to execute the movement (e.g., Paulignan et al., 1991). It is important to reiterate here that in the planning-control framework, effects observed at or near the end of the movement do not necessarily reflect control processes alone, or at all. Just as grip force and other weight-related parameters reflect planning but not control, I suggest that whereas movement time can be extended during control under conditions such as target perturbations, in those cases such extensions may occur as a by-product of the adjustments themselves. Conversely, under most "natural" circumstances (e.g., when the target remains stationary) movement times mainly reflect processes that go on before movement initiation (i.e., during planning).

It has often been shown that parametrization related to the spatial characteristics of the target is evident well before the movement is complete. For example, the opening and closing of the thumb-finger grip aperture in grasping an object correlates with the size of the target well before the target is contacted (Jakobson & Goodale, 1991; Jeannerod, 1984). This early scaling has been observed for many other spatial parameters as well, including velocity/acceleration (Gentilucci et al., 1997; Klatzky et al., 1995), hand shaping (Klatzky et al., 1995), and hand orientation (Desmurget et al., 1995, 1996; Jeannerod, 1981).

It has also been shown that planning considers the non-spatial characteristics of the target. For example, the weight of an object affects the amount of force used to grasp and lift it (Gordon et al., 1991). The coefficient of friction of an object’s surface affects the velocity of the reach (Fikes et al., 1994; Fleming et al., 2002). Other examples of planning utilizing non-spatial target characteristics are available from everyday observation. For example, people normally acknowledge the function of tools by grasping them by the handle, they generally avoid contacting hot or sharp surfaces, etc.

In contrast to the large number of variables that are hypothesized to affect planning, the planning-control model predicts that the visual information used during control will be focused on only the spatial characteristics of the target. Specifically, control will rely on both feedback (visual and proprioceptive) and feedforward (i.e., efference copy) mechanisms to monitor and adjust movements on-line. There is abundant evidence that these mechanisms contribute to on-line control.

2.4.1. Fast visual feedback loops in control. Woodworth (1899) was the first to study the use of visual feedback in on-line control. Woodworth reported that when participants drew lines at a rate of 400 msec per line or faster, the accuracy of the drawings was worse than if the lines were drawn at slower rates. Further, if the same task was done with eyes closed, participants’ performance at all speeds was just as poor as when the task was done quickly with eyes open.

More recent studies have shown that visual feedback can operate much faster than the 400 msec estimate offered by Woodworth. For example, Elliott & Allard (1985) observed a time frame of 170 msec in which visual feedback mechanisms could operate in a study of visuomotor adaptation to distortion caused by wearing prism goggles. Zelaznik et al. (1983) showed that when subjects were aware of whether or not visual feedback would be available, visual feedback aided in the accuracy of pointing movements in as little as 120 msec (see also Carlton, 1981).

2.4.2. Fast proprioceptive feedback loops in control. The importance of proprioception in action is well documented. When proprioception is lacking due to pathology of the brain or peripheral nerves, accuracy is reduced (Gentilucci et al., 1994; Jackson et al., 2000; Jeannerod et al., 1984; Lee & Tatton, 1975; Sainburg et al., 1993). On-line corrections of actions based on proprioceptive feedback have been observed to take place in as little as 50-100 msec (Craggo et al., 1976; Evarts & Vaughn, 1978; Lee & Tatton, 1975; Smeets et al., 1990).

Time frames of less than 150 msec in which feedback mechanisms can operate are markedly less than reaction times to initiate a movement, which Stark (1968) estimated to be at least 250 msec (cf. Jeannerod, 1988). These findings suggest that the planning and control of movements are separate processes. Planning appears to be a slower, more deliberate process in which a motor program is selected and initiated, whereas on-line control is much faster and more adaptable.

2.4.3. Use of efference copy in control. von Helmholtz (1866) was the first to postulate the existence of efference copies ("blueprints" of the motor plan forwarded to control mechanisms). Although von Helmholtz was concerned with the dissemination of information regarding upcoming eye movements, it appears that the brain also uses efference copies to control body movements.

The use of efference copy in control is evident in studies in which participants’ ability to localize unseen body parts was tested. When the participant actively moved the arm prior to localization, the ability to localize the arm with the other hand was relatively intact. In contrast, when the experimenter moved the arm, localization was relatively poor (Eklund, 1972; Jones, 1974; Paillard & Brouchon, 1968). Neurological support for the existence of an efference copy in reaching was found by Bard et al. (1999; see also Duhamel et al., 1992). Bard et al. observed that a deafferented patient was able to partly accommodate changes in a target’s position without the benefit of visual or proprioceptive feedback. As visual and proprioceptive feedback were denied this patient, any on-line adjustments must of necessity have relied on efference copy.

2.4.4. The perturbation paradigm. The perturbation paradigm involves suddenly changing a characteristic of the target, typically coincident with the onset of the movement (e.g., Georgopoulos et al.,1981; Soechting & Lacquaniti, 1983; for a complete review see Desmurget et al., 1998). According to the planning-control model, on-line adjustments to perturbations of the spatial characteristics of the target should occur relatively quickly, whereas on-line adjustments to non-spatial perturbations should take much longer, if they occur at all. For example, the motor system should adjust the grip aperture quickly to any change in the size of the target that occurs after movement initiation, as grip aperture relies on size (a spatial characteristics). However, the motor system should not be able to make a fast change to the force used in lifting the object, as this relies on a new computation of weight (a non-spatial characteristic).

Many studies have demonstrated the ability of the control system to adjust to changes in the spatial characteristics of the target. For example, Paulignan et al. (1991b) studied the ability of the motor system to accommodate a change in object location that coincided with movement initiation. Paulignan et al. (1991b) placed three dowels on a table. By manipulating the lighting of the dowels, they were able to create the impression that the target had changed location on some trials. Paulignan et al. (1991b) found that the acceleration profiles of the participants changed only 100 msec after the appearance of the new target. Similar short time frames have been found for reactions to perturbations of the target’s orientation (Desmurget & Prablanc, 1997; Desmurget et al., 1995, 1996), another spatial characteristic.

Paulignan et al. (1991a) studied the effects of a size perturbation on hand shaping in a thumb and finger grasp of a dowel. They observed that it took upwards of 300 msec for the finger movements used in grasping to be affected by a change in target size. However, a similar study by Castiello et al. (1993) showed that hand shaping could respond to a size perturbation in as little as 170 msec. The Castiello et al. study differed from Paulignan et al. (1991a) in that the participants were free to use as many fingers as desired to grasp the object with; in the latter study participants were required to use the thumb and index finger only, and unnatural grasping pattern for a large object.

Castiello et al. (1998) modified the perturbation paradigm to introduce a simultaneous perturbation of size and location. Such a double-perturbation paradigm would also allow one to change the identity of the target (for example, substituting a fragile target for a hard one). According to the planning-control model, it ought to take longer to accommodate the change in fragility by adjusting the force applied in grasping the object than to accommodate a change in location or size by adjusting the trajectory of the reach or the opening of the hand. It is notable, however, that Castiello et al. (1998) found relatively long adjustment times of grip aperture to the perturbation, suggesting that control processes can be slowed if more than one spatial characteristic is changed simultaneously.

According to the planning-control model, only planning should be susceptible to conscious influence. Indeed, it is intuitively obvious that at least some degree of conscious control can be exerted on many aspects of planning. For example, one will typically choose their targets consciously; on a lower level, one may speed up their movements if they are in a hurry, or change their posture consciously, etc.. Behavioral evidence that conscious systems can influence planning comes from studies showing that participants are consciously aware of the kinds of interactions that can sensibly be had with objects (Klatzky et al., 1987, Klatzky et al., 1989). In contrast to this, several studies have suggested that mechanisms responsible for on-line control operate outside of conscious awareness and influence (e.g., Castiello & Jeannerod, 1991; Gentilucci et al., 1995; Goodale et al., 1986; Pisella et al., 2000; Prablanc & Martin, 1992; Savelsbergh et al., 1991).

2.5.1. Control is immune to saccadic suppression. The ability of our perceptual system to disregard motion of images on the retina during eye movements has the side effect of making it very difficult for us to notice small displacements that occur in the visual world during a saccade. Often objects can be moved several degrees of visual angle without the displacement being noticed consciously. This phenomenon is known as saccadic suppression (see Chekaluk & Llewelynn, 1992, for a review). Despite the inaccessibility of these changes to conscious awareness, the motor system is able to accommodate them without difficulty (Goodale et al., 1986; Hallett & Lightstone, 1976; Prablanc & Martin, 1992).

Goodale et al. (1986) had participants point to a target that, in some conditions, moved during an initial saccade to its location. Although participants were unable to say whether or not the target had moved during the saccade, they nonetheless accurately adjusted their saccades and pointing movements towards the target without vision of the moving hand. It appeared that the control mechanisms had adjusted to a change in the target’s location that had remained consciously inaccessible. The fact that the adjustment occurred during the movement suggests that on-line control mechanisms were almost if not entirely responsible for the adjustment. In contrast, there is no evidence that planning processes play any role in the accommodation of the motor system to saccadic suppression.

2.5.2. Control is involuntary and unconscious. A study by Pisella et al. (2000) examined whether on-line adjustments could be subject to voluntary, conscious intervention. In their study, participants were required to make fast movements to targets that either remained stationary or jumped to a new location. In one condition, participants were instructed to correct their movements on-line when the target jumped. In another condition, participants were instructed to stop their movement when they noticed the target jump.

Pisella et al. found that when movements were fast (i.e., movement times of less than 250 msec), participants were often unable to avoid correcting their movements when the target jumped, even when they were explicitly instructed to stop the movement rather than correct it. This result suggested that fast control mechanisms operate outside of conscious control. Notably, Pisella et al. also found that an optic ataxic was impaired at making fast corrections even when instructed to do so, an important piece of evidence that I will return to in Section 4.

Studies of context-induced visual illusions and action offer an excellent opportunity to test the predictions of the planning-control model because they involve the impact of a factor predicted to influence planning (the visual context) on the classes of visual information used by planning (spatial and non-spatial characteristics) as well as control (spatial characteristics). As illusions can affect both of these classes of visual information, the planning-control model can be used to make clear predictions as to how and when visual illusions should affect actions. Further, these predictions often conflict with those of the perception-action model, allowing for direct empirical tests of the two models.

2.6.1. Predictions of the planning-control and perception-action models. According to the planning-control model, planning (which incorporates the context) should be affected by illusions induced by the surrounding visual context, whereas control (which ignores the context) should be unaffected (for other reviews and viewpoints, see Bruno, 2001; Carey, 2001; Franz, 2001). This means that illusion effects on all aspects of actions should be large early in the movements, but effects on parameters of actions based on the spatial characteristics of the target (e.g., size, distance, orientation) should progressively decrease as the movements unfold. Because the control representation is limited to the spatial characteristics of the target over a time frame of two seconds, the on-line correction of illusion effects on action should only occur when visual information is currently or recently available. In contrast, illusion effects on non-spatial target characteristics should not be corrected on-line, nor should illusion effects on spatial characteristics when a delay of two seconds or more is imposed. Finally, because control relies on both visual and proprioceptive feedback as well as efference copy, removal of any one of these sources of information ought to result in larger illusion effects in the control stage of movements than when all of these sources of information are available to the control system.

The predictions of the planning-control model can be contrasted with those of the perception-action model (Goodale & Milner, 1992; Milner & Goodale, 1995). In the latter model, a single visual representation is said to subserve actions whereas a separate representation subserves perceptions. According to the perception-action model, actions should generally be less susceptible to illusions than are perceptions (with some notable exceptions, such as movements made under delayed conditions – see Milner & Goodale, 1995).

Critical to the ensuing review is the contrast between the planning-control model, which predicts dissociations in the effects of illusions on planning versus control, and the perception-action model, which predicts no such dissociations. In contrast, in the perception-action model, the effects of visual illusions should be consistent (either small or large) throughout the movement. Further, the perception-action model predicts that the effects of illusions should not depend on the availability of visual feedback during control.

Table 2 summarizes the literature on visual illusions and body movements (For studies examining the effects of illusions on eye movements, see e.g., Binsted & Elliott, 1999; Binsted et al., 2001; Mack et al., 1985; Wong & Mack, 1981). It is clear from Table 2 that parameters determined by planning (e.g., lifting force, posture choice, movement time, grip acceleration) tend to exhibit large illusion effects, whereas those heavily influenced by control (e.g., maximum grip aperture, pointing accuracy, final hand orientation) tend to exhibit small or nonsignificant illusion effects. For example, when a delay of two seconds or more is imposed between the offset of the visual stimulus and the initiation of the movement, illusion effects are larger than when no delay is imposed, consistent with the idea that the control representation has decayed during the delay, and the index of action is measuring planning processes only (this result is also consistent with the perception-action model). More critically, when illusion effects on a spatial parameter are compared at different times in the movements, the effects tend to larger early in the movements than later.

Columns from left to right: the type of illusion; the measurement of action taken; the availability of visual feedback; whether the measure of action should reflect planning or on-line control; sample size; effect on perception (standard error), effect on action (standard error), and reference. ‘Acc.’ = accuracy; ‘Max. Grip Ap.’ = maximum grip aperture; ‘Peak Grip Vel.’ = peak grip velocity; ‘Orn.’ = orientation; ‘P’ = planning only, ‘MP’ = mainly planning; ‘MC’ = mainly control; ‘C’ = control only; ‘-’ = perceptual effect not tested; ‘N’ = Newtons force; ‘*’ = statistically significant at the p < 0.05 level; ‘ns’ statistically nonsignificant. Notes: 1) In some cases standard errors and/or significant tests were not reported or deducible. 2) Effects on Max. Grip Aperture are scaled to reflect the typical grip aperture-target size correlation of 0.8 (Jeannerod, 1988) at the time of MGA (cf. Franz et al., 2001; Glover & Dixon, 2001a). All other data are as reported in the studies or estimated from reported statistics or figures. 3) Max. Grip Aperture typically occurs at 60-75% of movement duration (Jeannerod, 1988).

2.6.2. Illusions affect planning. Many findings have been consistent with the idea that illusions affect planning. For example, van Donkelaar (1999) reported that the Titchener (or Ebbinghaus) size-contrast illusion affected movement times, but not the accuracy of pointing movements. van Donkelaar (1999) had participants point to the center of a target circle subject to the Titchener illusion as quickly and accurately as possible. Whereas the accuracy and variable error of the pointing movements were not affected by the illusion, the movement times were affected. Consistent with Fitts (1954), pointing to targets that appeared smaller took longer than pointing to targets that appeared larger. This occurred even though the targets were in fact identical in size. Assuming that movement times reflect the timing of actions and thus planning processes (as I suggested in Section 2.2), this study supported the idea that illusions affect action planning. Further, others have also have also found effects of illusions on movement times (Franz et al., 2001; Gentilucci et al., 1996; Smeets & Brenner, 1995; but see Fisher, 2001). These results are generally consistent with the notion that illusions affect planning.

Brenner & Smeets (1996) and Jackson & Shaw (2000) studied the effects of the Ponzo size illusion on visuomotor estimates of weight. The Ponzo illusion results when an object is placed on a background of radiating lines. If the object is placed near the end of the background where the lines originate, it appears larger than if it is placed near the end where the lines are maximally spread. Brenner & Smeets (1996) found a significant effect of the Ponzo size illusion on lifting force (measured as the velocity with which the object was raised). Similarly, Jackson & Shaw (2000) found the Ponzo illusion affected grasping force (the force applied to the sides of the object in lifting), another index of perceived weight. From these two studies, it appears that illusions affect motor parameters related to object weight (a non-spatial target characteristic), and that illusion effects on these parameters were not corrected on-line.

2.6.3. Illusions do not affect on-line control. Aglioti et al. (1995) measured the maximum grip aperture (distance between thumb and forefinger) as participants reached out to pick up a thin disc surrounded by a Titchener illusion display. They found that the maximum grip aperture was less affected by the Titchener illusion than were perceptual estimates of target size, the latter being measured using the method of comparison. (For difficulties associated with Aglioti et al.’s means of comparing perception and action, see Franz et al., 2000; Franz et al., 2001; Pavani et al., 1999). However, maximum grip aperture was still affected by the illusion to some extent, suggesting that the context had played some role in the shaping of the hand. Haffenden & Goodale (1998; Haffenden et al., 2001) also showed that maximum grip aperture was less affected by the Titchener illusion than perceptions when participants were denied visual feedback of the moving hand. Although these authors argued that maximum grip aperture represented an index of action planning, this parameter occurs well into the control phase of the movement and allows ample time in which on-line corrections could take place. As such, this result is consistent with both the planning-control and perception-action models.

2.6.4. Availability of visual feedback reduces illusion effects. One important test of the planning-control and perception-action models is that only the planning-control model predicts a reduction of illusion effects when visual feedback is available. In contrast, the perception-action model argues that illusion effects should be just as small during planning as they are during control, and thus the size of the effect should not depend on whether visual feedback is available during on-line control.

Gentilucci et al. (1996) demonstrated the influence of visual feedback on pointing movements subject to the Muller-Lyer illusion. Participants were asked to start with their finger at one end of a Muller-Lyer shaft and move it to the other end. Gentilucci et al. (1996) manipulated the amount of visual feedback available to participants, ranging from full feedback of the moving hand and target, to a no feedback condition with a five-second delay between lights out and movement initiation. Whereas only small effects of the illusion on accuracy were found in the full visual feedback condition, the effects of the illusion increased continuously over each condition in which less (or less recent) visual feedback was available. Similar interactions between illusion effects and the availability of visual feedback have been observed in other studies as well (Glover & Dixon, 2001c, 2002a; Westwood et al., 2000a, 2000b).

These results suggested that visual feedback could play a significant role in reducing illusion effects on action. This is consistent with the planning-control model, in which visual feedback contributes to the on-line correction of illusion effects. It is not consistent, however, with the perception-action model, in which illusion effects should be small on both planning and control, whether or not visual feedback is available during the movement itself.

2.6.5. Delays increase illusion effects. It is clear from several studies (Bridgeman et al., 1997; Gentilucci et al., 1996; Westwood et al., 2001) that illusion effects on action increase when a delay of two seconds or more is imposed between offset of the visual stimulus and initiation of the movement. This is consistent with the notion that the control representation has decayed in the interval, and that on-line corrections are absent under these conditions. According to the planning-control model, illusion effects after delays of two seconds or more reflect the influence of planning only, as control is unable to operate after delays. Note however that these results are also consistent with the perception-action model, as delays are said to result in a decay of the action representation and the use of a perceptual representation.

Another explanation may also be considered here. Franz et al. (2000; Franz et al., 2001; Franz, 2001) have argued that both perception and action utilize a single visual representation, and that both are equally affected by illusions. On this account, apparently small illusion effects on action have arisen because of a failure to adequately match the attentional requirements of the task. For example, Franz et al. (2000) argued that small effects of the Titchener illusion on maximum grip aperture relative to perceptual judgments in the Aglioti et al. study were a consequence of the need to attend to only one of the Titchener displays in the action task, but to both in the perception task. This hypothesis is intriguing, but would not seem able to account for differential effects on action and perception tasks when the two are adequately matched, as they are in most studies of illusions and action reported in Table 2. Further, this model would have to predict that illusion effects on perception would increase when delays are imposed, just as they do for actions. Evidence to contradict this hypothesis has been found in a study comparing delay effects on pointing and perceptual judgments (Bradshaw & Watt, 2002).

2.6.6. Dynamic illusion effects in reaching. Although the planning-control model provides a ready account of the findings described above, we aimed to investigate illusion effects on planning and control more directly (Glover & Dixon, 2001a). This was done using a task in which participants grasped a small bar placed at various orientations. We manipulated the perceived orientation of the bar by placing the bar on a background grating that was misoriented with respect to the participants’ sagittal plane. When the grating was rotated 10˚ clockwise or counterclockwise from sagittal, participants’ perceptions of the bar’s orientation were overestimated in the opposite direction. This orientation illusion effect was found to be roughly 2˚ in a perceptual task in which participants were required to align the bar with their sagittal planes.

In one experiment, we gave participants a choice between abducting or adducting their hand in picking up the bar. Abducting the hand results in the thumb being placed on the rightward edge of the bar (from the participant’s perspective), whereas adducting the hand results in the thumb being placed on the leftward edge of the bar. It has been shown that when an object is moved through a range of positions or orientations, participants’ choice of postures will typically switch from one posture to another within a fairly narrow portion of that range (Kelso et al., 1994; Rosenbaum et al., 1990; Rosenbaum et al., 1992; Short & Caraugh, 1997; Stelmach et al., 1994). The question in this experiment was whether the orientation illusion would affect the threshold at which participants switched from one posture to another. Our assumption was that the choice of postures would be made during pre-movement planning, and although it would still be possible to change the choice made once the reach was underway, the costs would usually outweigh the benefits. Thus, we reasoned that posture choice would provide a relatively uncontaminated measure of the illusion’s effect on planning.

We found that the choice of postures was affected by the orientation illusion. The threshold for switching from a hand-abducted to hand-adducted posture was adjusted approximately 2˚ by the orientation illusion, an effect roughly equivalent to the effect on perceptual judgments. This finding supported the idea that macroscopic aspects of planning are affected by illusions.

In another experiment, we set out to test the planning-control model more directly. Here, we had participants again pick up the bar, but had them use a hand-abducted ("thumb-right") posture on every trial. The orientation of the hand was measured throughout the movement using optical recording equipment. We observed that the orientation of the hand was linked to the orientation of the bar, and this was evident early in the reach, as has been found elsewhere (Desmurget et al., 1995, 1996; Desmurget & Prablanc, 1997; Glover & Dixon, 2001b, 2001c). More interesting was the fact that the orientation illusion affected the orientation of the hand. The effect of the illusion on the orientation of the hand was large early in the reach, but decreased to near zero as the hand approached the bar. This "dynamic illusion effect" supported our prediction that participants would correct for illusion effects in flight. Large effects early in the movement presumably reflected the illusion’s influence on planning, whereas continuously decreasing effects thereafter reflected the relative immunity of control.

It is difficult to accommodate the results of this study within the perception-action framework, however. For one, posture choice would seem to have to rely on an illusory "perceptual" representation, even though the bar and display were visible throughout each trial, and presumably enough information was present in order to select an appropriate posture. For another, it would appear that in reaching, the perceptual representation would have had to subserve the initial planning of the reaches towards the bar, whereas the action representation would have been responsible for the balance of the movements.

One study by Dyde & Milner (2002) looked at the effects of a simultaneous tilt illusion on ‘posting’ behavior. In this study, participants aimed a card towards a rectangular figure drawn vertically on a background grating. Although Dyde & Milner (2002) did not specifically analyze illusion effects over time in this study, they did find a large effect of the illusion on posting behavior at the end of the movement. This result seems to contradict our own results using a very similar illusion (Glover & Dixon, 2001a, 2001b, 2001c), and Dyde and Milner interpreted their results as suggesting that the orientation illusion arises at V1, thus affecting both perception and action. While this hypothesis does not explain why we did not find similar effects at the end of the movement in any of our experiments, an explanation of the discrepancy between studies may be given. It is possible that the discrepancies resulted from the use of vertically-oriented targets by Dyde and Milner, whereas ours ranged in orientation from 5 to 35 degrees. In their study, the posting task may have included a strong demand to ‘match’ the tilted appearance of the target with the card; in our study the orientation illusion was much less noticeable and so this demand was not present.

2.6.7. Dynamic illusion effects in grasping. We have also applied a similar kinematic analysis to the grip aperture in a reach to a disc subject to the Titchener size illusion (Glover & Dixon, 2002a). The data in this study also supported the planning-control model. Illusion effects on grip aperture were largest early in the reach and decreased as the hand approached the target. Dynamic illusion effects have also occurred when vision of the hand and target were blocked during the reach (Glover & Dixon, 2001c, Glover, 2002a), suggesting that proprioceptive and efference mechanisms can play a significant role in the on-line correction of illusion effects.

The results of this study involving the Ebbinghaus illusion have been contentious, however. Danckert et al. (2002) re-analyzed two of the experiments previously carried out in the Goodale lab. Based on the re-analysis, they suggested that there was no evidence for an illusion effect at any time up to the point where maximum grip aperture was reached (i.e., at roughly two-thirds of movement duration), except for an effect in one experiment at the time of maximum grip aperture. I have questioned this conclusion, however, because there was no scaling of grip aperture effects to the changing dependence of the grip aperture on object size at different times during the reach (Glover, 2002). Such scaling is vital in any analysis of the effects of cognitive or perceptual variables effects on action (for explanations see e.g., Franz et al., 2000; Franz et al., 2001; Glover & Dixon, 2001a, 2002b).

Gentilucci and his colleagues (Gentilucci & Gangitano, 1998; Gentilucci et al., 2000) recently demonstrated interesting effects of semantics on actions. In these studies, words were printed on objects and participants reached to and grasped them. It was observed that the meanings of the words printed on the objects affected the kinematics of the participants’ movements directed towards those same objects. For example, Gentilucci et al. (2000) observed that the maximum grip aperture was larger for objects on which the word "GRANDE" ("large") had been printed than for words on which had been printed "PICCOLO" ("small"). This suggested that the motor system had been influenced by the meanings of the words when a motor program was selected. Effects were also observed for several other word pairs, including (the Italian equivalents of) "long" and "short", "near" and "far", and "high" and "low". Gentilucci and his colleagues interpreted these results within the framework of models that argue for a close relationship between language and motor processes (e.g., Kimura, 1979; Rizzolatti & Arbib, 1998).

We have recently extended the work of Gentilucci and his colleagues in order to measure the effects of words on action throughout the course of the movement (Glover & Dixon, 2002b; Glover et al., 2002). In particular, we aimed to test the predictions of the planning-control model with respect to semantic effects. In the planning-control model, a cognitive process such as semantics should affect the planning of the movements, but not how they are controlled on-line. Thus, a similar result should occur as occurred with visual illusions: i.e., there ought to be large effects of the words early in the movements but continuously decreasing effects as the hand approached the targets. And in fact these were the exact results we obtained. In one study, we found that participants had larger grip aperture early in the reach for objects on which was printed the word "LARGE" than for objects labeled "SMALL". However, as with illusion effects on action, these word effects faded as the hand approached the targets (Glover & Dixon, 2002b).

A similar result was obtained when we had subjects first read a word then grasp an object (Glover et al., 2002). In this study, the words could represent either relatively large objects (e.g., "APPLE", "BASEBALL") or relatively small objects (e.g., "PEA", "GRAPE"). Here, we observed that the size of the object represented by the word had a large effect on the grip apertures early in the reach. For example, reading words such as "APPLE" led to larger grip apertures than reading words such as "GRAPE". Again, however, this effect faded as the hand approached the targets, and participants were able to execute the grasps without difficulty.

These "dynamic word effects" provide further support for the planning-control model, but would be difficult to incorporate within a perception-action model. In the latter model, the motor system could presumably plan and execute the movement based simply on the relevant spatial characteristics of the target alone, independent of semantic processes. Conversely, if the perception-action model were extended to include the supposition that semantics could influence action by an interaction between perception and action systems, it would still have difficulty explaining why the words affected planning only and not control. In short, the effects of semantics on action seem to be much more consistent with the planning-control model than with the perception-action model.

Studies of healthy participants have demonstrated the distinction between planning and control. Whereas planning represents a process that is relatively slow and complex, control mechanisms appear to be much more flexible, faster, yet limited in scope. Planning selects an action based on an integration of a broad range of visual and cognitive information. In contrast, control operates using a fast visual representation limited to the spatial characteristics of the target, coupled with visual and proprioceptive feedback as well as efference copy, in monitoring and adjusting movements in flight.

Because planning incorporates the context surrounding the target, it is affected by context-induced visual illusions. Conversely, because the control representation excludes the context, it is relatively immune to these same illusions. This distinction is reflected in the pattern of illusion effects on action. Whereas indices of planning are affected by illusions, indices of control are much less affected, if at all. Critically, when measures of illusion effects on spatial parameters are taken throughout the movement, the effects of the illusion are large early in the reach, but decrease as the hand approaches the target (Glover & Dixon, 2001a, 2001b, 2001c, 2002a). Similar results have occurred for the effects of semantics on action (Glover & Dixon, 2002b; Glover et al., 2002), also consistent with the notion that cognitive influences on planning are corrected on-line during control. The results of these studies were much less consistent with the perception-action model, however.

Future studies may be aimed at expanding and clarifying the nature of the dissociations between planning and control. According to the planning-control model, these dissociations should generally take the form of influences of many visual and/or cognitive variables on planning, but a lack of an influence of these same variables on control. For example, the planning-control model predicts that the pattern of effects of semantics on action ought to be identical to the pattern of illusion effects (i.e., large effects on indices of planning, small or nonexistent effects on indices of control). More generally, non-spatial characteristics of the target, conscious awareness, visual context, memory processes, and overarching goals should all interact to affect planning, but only the spatial characteristics of the target should affect control.

Here, I briefly describe the anatomy and connectivity of the brain before discussing the neural bases of action. Figure 3 shows the main visual cortical pathways in the human and monkey brains. In the monkey, the visual pathways fall into two main "streams", a dorsal stream terminating in the inferior parietal lobule and a ventral stream terminating in the inferior temporal lobe (Mishkin et al., 1983). Boussaoud et al. (1990) also proposed the existence of a "third" visual stream, terminating in the superior temporal sulcus.

Figure 3. Illustration of putative visual pathways in the posterior regions of the human brain (top), and macaque brain (bottom -- based on Boussaoud et al., 1990; Mishkin et al., 1983). Note that whereas the dorsal stream in the macaque brain terminates in the inferior parietal lobule, in the human brain it is hypothesized to terminate in the superior parietal lobule. The medial or "third" stream is hypothesized to terminate in the superior temporal sulcus in monkeys, but in the inferior parietal lobule in humans. Not to scale.

Originally, the two main visual streams were hypothesized to serve functions related to object identity and spatial localization (the "what versus where" distinction; Mishkin et al., 1983). However, numerous researchers have suggested a role of the primate parietal lobe in sensorimotor transformations (e.g., Mountcastle et al., 1975; Stein, 1991, 1992). Some researchers have contended that the roles of the two streams may best be described not as "what versus where", but rather as "what versus how" (Milner & Goodale, 1995; Goodale & Milner, 1992), or similarly, as "semantic versus pragmatic" (Jeannerod, 1994, 1997; Jeannerod et al., 1995; Jeannerod et al., 1994).

Such a reformulation of the functions of the two streams has drawn evidence largely from neurophysiological studies of monkeys. Numerous authors have argued that the dorsal stream pathway encodes the sensorimotor transformations necessary for goal-directed actions (Goodale & Milner, 1992; Jackson & Husain, 1997; Kalaska & Crammond, 1992; Milner & Goodale, 1995; Sakata et al., 1997; Wise & Desimone, 1988). Support for this notion comes from studies showing activity of cells in the dorsal stream of monkeys related to the visual guidance of reaching (Taira et al., 1990) and grasping (Murata et al., 1996). Cells in the posterior parietal lobe of monkeys have also been shown to be sensitive to changes in motor plans independent of changes in visual attention (Gnadt & Andersen, 1988; Snyder et al., 1997, 1998).

3.1.1. Functions of the two streams in monkeys may not map onto the human brain. A reasonable assumption to make is that human brain organization and function will closely parallel that found in monkeys. However, it will be seen that the data from both brain imaging and neuropsychology raise serious doubts about drawing such parallels in high-level vision (see also, e.g., Eidelberg & Galubardi, 1984; Vandufell et al., 2000). In particular, I will show that the "what/how" distinction found in monkeys cannot be easily translated into an explanation of human brain organization. Rather, I suggest that the evolution of the human brain has resulted in the localization of planning processes in the phylogenetically newer cortex of the inferior parietal lobule. Tool and object use in particular has required that human motor planning processes integrate ventral stream functions related to object identification and context. I hypothesize that this integration occurs in the IPL. The role of the IPL suggested here is thus dramatically different from that proposed by Milner & Goodale (1995), who emphasized the role of the IPL in spatio-cognitive operations and praxis. These authors suggested that the dorsal stream generally possessed sufficient visual information for both the planning and control of actions, although on occasion it may be necessary for the streams to "interact", such as in movements to remembered targets, or movements made after a delay (Milner & Goodale, 1995). In contrast to the perception-action model’s predicted interactions between dorsal and ventral and/or IPL visual systems in some actions, only the planning-control model maps planning and control specifically onto the IPL and SPL, respectively.

The planning-control distinction can also be plausibly extended to other brain structures. Specifically, I hypothesize that the planning system includes regions in the frontal lobes, basal ganglia, and cerebellum. Similarly, although the visual representation in the SPL is said to be the crucial factor underlying on-line monitoring and control, I hypothesize that it operates in concert with other control centers in the cerebellum.

The planning-control and perception-action models can be used to make specific predictions regarding the brain regions active during motor behaviors. In the planning-control model, activation of the two systems should depend on the task at hand. Specifically, tasks that require heavy involvement of planning systems, such as selecting an appropriate posture, should preferentially activate planning regions. Conversely, tasks that require major involvement of on-line control systems, such as manual tracking or adjustments to target perturbations, should preferentially activate control regions.

These predictions are in contrast to the predictions of the perception-action model (Goodale & Milner, 1992; Milner & Goodale, 1995). In this model, visual "action" representations are said to reside in the SPL. As such, the perception-action model predicts that motor behavior should generally lead to activation of the SPL but not the IPL, and that this should be true during both planning and control.

Three types of brain imaging paradigms have been used to study the neural underpinnings of planning and control. One paradigm involves motor tasks that encompass both planning and control and thus reveals structures active during one or the other, or both stages. A second paradigm focuses on the neural structures involved in either planning or control, respectively. A third paradigm employs the use of "motor imagery", imagining the production of a movement without actually carrying it out. As predicted by the planning-control model, increased activity in regions of the planning system is strongly associated with the planning phase of action, whereas increased activity in regions of the control system is strongly associated with the on-line control phase of action.

3.2.1. PET and the motor brain. Several PET studies have shown increased activation of both planning and control regions during motor tasks. Kertzmann et al. (1997) studied the changes in brain activation during visually guided pointing. They found that activity increased in both the inferior and superior regions of the parietal lobe, the premotor cortex, and the basal ganglia (the cerebellum was not scanned) when pointing movements were made. Inoue et al. (1998) studied how pointing with or without visual feedback affected brain activation. Whether or not feedback was allowed, significant increases in activity occurred in both the inferior and superior regions of the parietal lobe during movement compared to rest trials, suggesting that the involvement of neither region was contingent on the availability of visual feedback. Activity was also observed in the frontal lobes, temporal lobes, basal ganglia, and cerebellum. Rizzolatti et al. (1996) studied the neural underpinnings of grasping movements using PET. Grasping an object resulted in increased activity in both the IPL and posterior regions of the SPL, as well as the basal ganglia and cerebellum.

The studies reviewed above show that motor behavior results in increases in activity in both the inferior and superior regions of the parietal lobe, as well as the frontal lobes, basal ganglia, cerebellum, and in one case, the temporal lobes. Whereas these studies are consistent with activation of both of the networks I have proposed in this paper, they do not allow one to dissociate activity related to planning from that related to control. For that purpose, it is necessary to turn to more direct investigations of planning versus control.

Figure 4. Summary illustration of increases in brain activity in the posterior parietal lobe related to motor planning, on-line control, and motor imagery. Filled squares: increased activity related to planning; empty squares: increased activity related to on-line control; filled circles: increased activity related to motor imagery. See text for details.

3.2.2. PET and planning versus control. Figure 4 summarizes the results of studies comparing brain activity during the planning and control of actions, respectively. Grafton et al. (1998) measured changes in brain activity when participants made a choice between a power and precision grasp, the former involving the entire hand and the latter involving the thumb and forefinger only. In two conditions, grasp choice was mandatory. In another condition, a cue occurred prior to each trial that informed the participants of which grasp to use.

Significant increases in activation occurred in the IPL and basal ganglia in all movement conditions, whereas increased activation in the SPL only occurred in the cued versus mandatory comparison. That is, only when participants had to focus attention on an upcoming cue was activity in the SPL elevated. This finding concurs with other PET studies that suggest that the SPL is involved in visuospatial attention (Corbetta et al., 1993; Corbetta et al., 1996; Jovicich et al., 2001; Haxby et al., 1994; Nobre et al., 1997). Increased activity also occurred in the premotor cortex in the cued versus mandatory comparison.

Deiber et al. (1996) also studied brain activation during movement selection. This study had the added control of having delays imposed between cue and response. This allowed for a greater proportion of scanning time being devoted to measuring regions involved in planning. Flexion or extension of either the forefinger or little finger was required, and cues could inform participants of the finger to be used, the movement required, both the finger and movement required, or neither. Another condition allowed participants to freely decide which movement to make with which finger.

Deiber et al. (1996) showed that increased activation of the IPL, premotor cortex, and cerebellum occurred in all of the selection conditions relative to a rest condition, whereas increased activation of the SPL was present only when the cue was uninformative or only partly informative. Similar to Grafton et al. (1998), the results of Deiber et al. (1996) suggest that the IPL was activated during planning whether the cue was informative or not, whereas the SPL was only activated during the planning phase when attentional demands were high.

Krams et al. (1998) utilized a task in which participants were required to copy a hand posture shown to them on a screen. Three conditions manipulated the relative import of planning and control. In one ("execute only") condition, participants simply copied the movement immediately after its presentation. In another ("plan and execute") condition, a pause was given between the presentation of the movement and a subsequent cue to imitate. Participants were instructed to use this delay to prepare the movement. A third ("plan only") condition had participants prepare the movement, but withhold its execution.

The design of Krams et al. (1998) allowed them to dissociate the changes in activity during planning from the changes during on-line control. When the "plan only" condition was compared to the rest condition, increased activity was found in the supramarginal gyrus of the IPL, premotor cortex, basal ganglia and cerebellum. No increase in activity was observed in the SPL. Increased activation of the IPL and premotor cortex was also observed in the "plan and execute" condition compared to the "execute only" condition, another measure of planning. However, there was no increase in activity in the SPL, cerebellum, or basal ganglia in this comparison.

The reverse comparisons (i.e., "execute only" versus rest, "plan & execute" versus "plan only") showed what happened during the execution phase of a movement. In the former of these comparisons ("execute only" versus rest) increased activity occurred in the intraparietal sulcus bordering the IPL and SPL, the frontal lobes (including premotor and primary motor cortex) and the cerebellum, suggesting that these regions were involved in executing the actions. The latter comparison ("pla