BEING ABOUT Chapter 5. Imagining
Presence and simulationA perceptual system that has become sensitized to certain invariants and can extract them from the stimulus flux can also operate without the constraints of the stimulus flux ... The visual system visualizes. But this is still an activity of the system, not an appearance in the theatre of consciousness. Gibson 1966, 256 ... activation of the same central neural systems ... in the absence of the original sensory activity Hebb 1966, 68 ...reinstate the whole situation of seeing, or having, the object Ogden and Richards 1959, 235 Perception and action are basic. We stay alive only if we, or other people for us, stay in touch with where we are. Perception is, and action requires, responsive restructuring: we are able to perceive and act because our bodies are immediately related to, effective in, adapted to, structurally about, our surroundings. It is however one of the extraordinary, complicating facts of human experience that we are also able to be structurally about circumstances we are not with: we are able to simulate aboutness. To simulate is to be (in some ways) structured as if we were in circumstances other than our actual circumstances: it is to be as if about, as if in reference to. Simulation is situationally disengaged aboutness. The presence/simulation distinction is a version of the perception/imagining distinction, with an important difference. Classical representationalists speak about imagining as a kind of quasi-perception. Understood as a variety of structural aboutness, the notion of simulation can apply to non-perceptual structure as well as perceptual structure. Limbs, muscles and glands, the heart and lungs, as well as neural structures serving these organs, may also be in simulational states -- that is, in states primarily relevant to circumstances other than the body's present circumstance. Action and emotion, as well as perception, can be simulational. One of the advantages of a non-representationalist account of imagining is that we do not have to ignore aspects of simulational function that are not perceptual or sentient. The distinction between presence and simulation is epistemologically central, but, given what we now know about cognitive embodiment, it cannot be applied to particular states with any precision. We are very able to perceive and imagine at the same time; we often do so. Living bodies are in touch with their actual circumstances in some ways, even while they are out of touch in others. If this were not true, sleeping bodies could not be wakened. The body's normally stable contact with its surroundings is what allows the sorts of limited inattention we exploit for various purposes. Situationally engaged states of perception and action often include simulation. There are also many kinds of simulational preparation. Prepared in these ways, we can respond more quickly. Action or motion often set us up to project their usual sensory consequences. If a bird flies behind a chimney we know where and when to expect to see it again; we have been as if tracking it while we cannot see it. Experimental subjects shown part of a video of a truck backing up remember the truck as having moved further than it had in the segment they saw (Freyd 1988). When we hear ten notes of a familiar tune we will seem to hear the eleventh, or, when a spoken phrase is missing the first phoneme, we will supply one suited to the context (Grossberg 1995). These sorts of anticipatory or priming results can be understood as dynamic completion effects in neural nets. Sensing in one modality also primes us to anticipate the usual concommitants in other modalities. These are the ordinary synaesthesias: the part of seeing a texture that is seeming to touch it; the part of hearing that is seeming to see; the felt subvocal tensions when we read or listen. More extended forms of simulation can be ways of making use of perception/action circuits to consider things that have happened or might happen, or things we might want to do. Evoking the physical consequences of circumstances we are not in, we are able to plan. Damasio (1994) makes the point that planning includes simulational emotion that acts to mark value consequences of a proposed act. Developed simulation can also be used in self-management. We can simulate to change our state, imagining we are safe if we need to be calm, imagining danger if we want to be more alert. Perception and action occur by means of physical structures strongly ordered by present sensory contact with environmental pattern and energy. Simulational structure is energetically and organizationally more autonomous, and so it tends to be less robust and more transient. When simulation must be sustained for purposes of planning and self management, we often make use of contact with representational artifacts. We draw a diagram to help us sustain complex spatial planning, or talk to ourselves to sustain the fantasy that sustains a mood. Using representational practices in these ways is like seeing animals in moving clouds -- it is perceptually entrained simulation. An account of cognition grounded in a vision of bodies built in contact with the actual world needs a plausible description of simulation for at least three reasons. One is that, in dualist stories about mind, imagining states are often taken as evidence that minds 'transcend' embodiment. A second is that understanding simulation correctly is essential to understanding our use of representing artifacts and events. When we understand how we use representing artifacts and events, we also understand what it is that is wrong with the representation metaphor for cognitive structure. A third is that, given the importance to any animal of moment-by-moment adaptation to its actual environment, we need to be able to account for the evolution of an ability to be structurally non-present. In humans, the ability to simulate is culturally much elaborated and astonishingly developed. In its madder manifestations it can defeat basic level function and cause death. In these instances, is it to be understood as a concommitant of abilities that are vital to basic level flourishing? If we understand it as anticipatory self-organization, the sorts of involuntary short-term dynamic completion described above have obvious evolutionary advantage (Dennett 1991), as does deliberate planning. Seen at the level of their cortical organization there is, moreover, a functional overlap or continuum from perceiving to imagining, which can suggest how the ability to be non-present can have evolved in stages, in parallel with a developing ability to be more present. The participation of prefrontal cortex in attentional modulation is central to the story. In what follows I will trace the role of counterflow activity in a continuum of deliberate attentional situations, from overt orientation, to covert attention, to orientation maintained through a delay, to environmentally uncued deliberate simulation. I will describe two levels of counterflow participation. The first -- which does not involve prefrontal cortex -- is a posterior system in predominantly sensory cortex behind the central sulcus; it is a counterflow from posterior association areas to primary and secondary sensory areas. The second, more comprehensive, with longer loops, is a counterflow from frontal into posterior cortex. Both of these sorts of back-connection are important for simulational purposes. The second is primarily important to deliberate simulation. The first is normally included within the second as a subsystem, but when it runs independently it is primarily important to externally-cued or involuntary simulation (and thus to representational practices, which will be described in the following chapter). The material in this chapter is quite technical; understanding the intricacies of backflow effect seems to be worth some effort since this understanding works so effectively to re-embody our notion of imagining, and with it our notions of high cultural cognition in general. Counterflow and attentionIf cortical structure is organized by back-to-front through-streams from posterior sensory to anterior motor cortex when we act in response to environmental conditions, how is cortical structure organized when it is not being organized from the sensory periphery? The best approach to an answer is probably to consider skilled motor behaviors in the more complicated animals. Skilled behaviors require deliberate attention, and deliberate attention requires modulatory front-to-back activity. Such counterflow activity can also organize simulational nets. Focal attention may thus be the common key to both presence and simulation. Engaged or attentive aboutness requires alertness and selectivity. Simulational kinds of attention are not, by definition, engagement, but they are seeming engagement, and as such they also require alertness and selectivity. Recall that the large-scale attentional network described by Mesulam (1981, 1990) includes a subcortical node that sets levels of general arousal or alertness. [5-1 Mesulam's attentional net] Selectivity is a result of orientation and priming. Mesulam's network also includes dorsal foci organizing saccades, fixations, head-turns, and reaches -- the various forms of orienting action that begin the process of setting up a task axis. Priming is a pre-modification of neural characteristics that alters the effect of incoming activity. It may be inhibitory or facilitory, and it occurs in a number of ways: direct counterflow activity from upstream areas is one of the most important; a second involves indirect back-connections via the subcortical thalamus, which sends inhibitory and facilitory projections to most cortical areas.
Posterior counterflow -- in cortical areas behind the central sulcus -- back-connects association areas in both ventral and dorsal streams to primary and secondary sensory areas. These connections modify sensory flow as it begins to propagate forward toward motor areas. Martha Farah hypothesizes that object vision, for instance, needs converging activity from both dorsal and ventral streams onto early visual areas in V1/V2. She cites lesion evidence that successful visual attention requires dorsal activation priming response to a selected location, and ventral activation priming response to object form. Touch-sensitive cells among the sensory-motor gradients of the superior parietal have been found to project backward onto primary somatosensory cortex. By means of these back-connections, SPL touch areas help to organize haptic perception of objects. Fuster (1995, 250) suggests that SPL cells with auditory response may similarly project back onto primary auditory cortex and help organize perception of auditory sources. When an object is being explored by all or several of the senses, counterflow activity from SPL cells all tuned to the same location could thus be critical to perception of an object as such. Visual counterflow activity is important in determining which thing will be seen, and then also in determining which aspect of that thing will be emphasized. As many as seven retinotopic areas distribute counterflow projections across the same layer of primary visual area V1 (Pollen 1999, 13). Form and color areas in the ventral stream have, for instance, counterflow connections that can prime neuronal groups responsive to a particular color or form. At the same time, the pulvinar nucleus of the thalamus is known to prime early visual cortex from below, and may be used for indirect counterflow priming. According to Edelman's core dynamics hypothesis, it is bidirectional flow of neuronal activity that establishes the synchronized subnet essential to sentient perception. Pollen similarly guesses that counterflow activity may be necessary to integrated conscious perception of objects having sensory qualities of different kinds: "consensus of neuronal activity across ascending and descending pathways linking multiple cortical areas ... may underlie the normal unity of conscious experience" (Pollen 1999, 4). I have described these posterior networks for perceptual attention separately from the more widespread prefrontal counterflow networks because later in this chapter I will be describing kinds of simulation (such as dreaming) that use this level of attention exclusively. Where we are wanting to talk about voluntary, sustained, detailed, or complex many-part attention, however, we have to include the fourth node of Mesulam's attentional net, which is found in frontal rather than posterior cortex. Prefrontal counterflow networks normally include posterior networks described above and act by modifying them. Prefrontal counterflow
The frontal lobe includes prefrontal, premotor and motor divisions, as described in Chapter 4. Prefrontal and premotor cortex are distinguished on the basis of differences in thalamic connectivity; prefrontal cortex is defined as the parts of frontal cortex connected to the forward nucleus of each thalamus. There are also developmental and cytoarchitectural differences. Like inferior parietal areas to be described in Chapter 8, prefrontal cortex is phylogenetically and ontogenetically recent, and like other recent areas, it is one of the last neocortical regions to mature (Fuster 1995, 174). [5-3 Human prefrontal, premotor, motor] In humans, prefrontal cortex makes up one-third of neocortex. Human prefrontal areas include Brodmann's areas 9, 10, and 46. Parts of Broca's area (44/45) are sometimes also considered prefrontal rather than premotor. Cortical subdivisions are often defined on the basis of retinotopic, cochleotopic, somatotopic or movement-related arrays discovered in a cortical region, but Patricia Goldman-Rakic says there is no evidence of this sort of array in prefrontal cortex (1987). Goldman-Rakic believes subdivisions of prefrontal cortex may instead be defined by tracing interconnections with subareas of posterior association cortex. She believes there are six prefrontal subareas in macaques, and ten in humans, as determined by this criterion. [5-4 Human prefrontal subdivisions] There is little agreement about the details of prefrontal function, but prefrontal cortex is often described as a module for executive or top-down control. When paired with 'executive' and 'module', 'top-down' suggests independent or one-way origination. While it is true that prefrontal cortex seems to be essential to many kinds of human autonomy and agency, it is only one of the interdependent nodes of many-part networks spanning the brain. A person controls many things in an environment, and particular brain areas facilitate the decisive acts of persons in various ways. Less metaphorically described, what is known about prefrontal function is that, in the most general terms, "attentional, mnemonic, and motor-control deficits occur after lesions in all areas ... although in relation to different tasks" (Goldman-Rakic 1987, 403). The functional overlap of attention, memory and motor organization discovered in prefrontal function is important to an understanding of complex and voluntary forms of simulation.
The basic-level importance of prefrontal function may be thought to be its role in complex physical action. Prefrontal cortex is the frontal apex of both ventral and dorsal through-streams from sensory to motor cortex. It is sometimes called heteromodal cortex, because it is high-order association cortex, connecting to multimodal association areas in dorsal and ventral sensory streams as well as to motor association cortex in premotor areas. Not all sensory-motor paths are routed through prefrontal cortex: we have seen that the same muscles can be set in motion from through-streams at different anatomical and evolutionary levels (Fuster 1995, 275-6). Motor behaviors may be sequenced and released by activity in subcortical, parietal, motor, premotor and prefrontal regions, each succeeding level adding context. Environmentally controlled action is organized by relatively direct through-lines from sensors to effectors, but deliberate action, mediated by prefrontal cortex, resists or delays the effects of immediate environmental and body-internal conditions. Fuster describes prefrontal involvement in motor organization in terms of an override function; regions of the prefrontal have direct connections to premotor and to subcortical motor structures, and are therefore in a position to inhibit or facilitate lower-level motor routines. Learned and complexly contextualized behaviors are "mixed," Fuster says; "They contain segments that are novel and unrehearsed as well as segments that are old and thoroughly practiced" (1995, 170). Fuster believes that coordinating "the continual interplay of voluntary and automatic action that makes up normal sequential behavior, both in the motor and in the speech domain" is the motor specialization of prefrontal cortex (195).
At the same time as it is overriding, releasing and combining motor routines, prefrontal cortex is also able to organize motor response indirectly by altering perceptual structure. Skillful action requires that we direct and maintain attention while negotiating the stages of a complex enterprise. In behavioral terms, this means acting to establish, maintain, monitor and correct spatial orientation, in spite of distracting new events or even through a temporary obscuration of what is being attended; it may also involve orienting to detail, for instance to a part or aspect of something normally perceived more generally. In neural terms, it means selecting, maintaining, accumulating and refining structure in many parts of the moment's wide net. Along with its forward connections to premotor cortex, prefrontal cortex also has back-connections to association cortex in both ventral and dorsal streams. Prefrontal cortex may shift or maintain orientation through back-projections to parietal areas. It is also able to inhibit and facilitate early sensory structure and ventral association cortex. By these means prefrontal cortex can increase perceptual activation and maintain it long enough so that complex motor over-ride and release decisions can be made. The particular role of prefrontal cortex in the attentional net thus seems to be simultaneously to modulate what happens downstream in multiple motor areas and upstream in multiple sensory areas, so that structure in different brain regions is selectively inhibited or facilitated in combinations appropriate to a task (Knight et al 1999, 172). It is this parallel and interactive prefrontal modulation of neuronal activity in motor and sensory cortices that "results in the higher level functions attributed to prefrontal cortex," Knight suggests (161). Since areas with increased and sustained activity may join the core dynamic subset of the moment's wide net, one effect of prefrontal function may also be to make action conscious. It should be noted that, because of its role in multi-level coordination, prefrontal cortex is itself the target of neurotransmitter projections that can rapidly alter its global operating state. A medial area of the prefrontal, inside the fissure dividing the cortex into two hemispheres, is a relay from subcortical saliency structures.
Two frontal regions, intensively interconnected, are particularly important to orientational aspects of deliberate attention. Both are crucial to spatial attention by being crucial to deliberate eye motion. The older of the two regions, the frontal eye fields, seems to be both a motor association and a multimodal sensorimotor association area. The more recent region is dorsolateral prefrontal cortex. The two areas work in concert to establish spatial attention in perception/action situations, and have recently been found important to simulational attention as well. The FEF are near eye-related motor cortex, to which they have extensive forward connections. They also have back-connections to posterior association areas in the parietal, and to subcortical matrices active during reflex saccades. In the macaque the FEF are on the anterior bank of the arcuate sulcus (Courtney et al 1998, 1350). In humans they are higher and further back, in Brodmann's area 8. Passingham speculates that the frontal eye fields come into play for deliberately sequenced eye movements and other learned forms of visual orientation; monkeys and humans with lesions in lateral area 8 have difficulty directing their eyes on the basis of learned rules (1993, 122). Recall that the subcortical superior colliculus is active for reflex saccades to a source of sound or visual motion. If both the superior colliculus and the FEF are damaged, saccadic eye movements are no longer possible. Mesulam's notion is that "The superior colliculus may be important for foveating the general area of interest, whereas neurons in the FEF may be more important for the finer analysis of that region" (1990, 599). Single cell studies find units in the FEF that fire not only during a saccade to a particular location, but also immediately before the saccade. The FEF are also found to be active when a monkey is being trained in antisaccades, that is, when a monkey is learning to look away from a stimulus that would normally draw its gaze (Goldman-Rakic 1994, 115-116). Mesulam's summary of the role of the FEF is that, along with its dorsal back-connections to the parietal and forward connections to motor cortex, they are a "mechanism for specifying whether a location in space (and events within it) will become the target of enhanced neuronal impact," and thus of visual grasp, manipulation, or exploration (1990, 599). (Through-lines from parietal and temporal areas often also terminate in an area near but outside the frontal eye fields. This area is reciprocally connected to areas showing auditory orienting response.) Dorsolateral prefrontal cortex is a large area of frontal cortex anterior to the frontal eye fields. The rate of activity of this area during many sorts of tasks seems to be closely correlated with degree of deliberate attention. In the macaque, dorsolateral prefrontal cortex is the area in and around the principal sulcus, anterior to premotor areas F4 and F5 (which include reach and grasp areas described in Chapter 4) as well as to the frontal eye fields. The macaque principal sulcus may be called area 9 or area 46; Goldman-Rakic prefers to call it area 46, to distinguish it from a neighbouring area with different response properties. The human homologue of the macaque principal sulcus is generally thought to be Brodmann's 46, but, because the human FEF have been pushed backward and upward by expansions of association cortex in more ventral and more anterior areas of prefrontal cortex, the human homologue to the principal sulcus may also be higher and further back. The macaque principal sulcus has numerous connections to the frontal eye fields, to a second, more medial premotor eye region, and to the superior colliculus and other subcortical oculomotor areas (125). Passingham believes dorsolateral prefrontal cortex has evolutionary ties to the FEF: "area 46 presumably differentiated out of areas 8 and 45; it is similar in being involved in the selection of eye movements. In macaques it is similar to area 8 in having inputs from visual areas, though it differs in that these come from later stages of visual processing" Passingham 1993, 257 The organization of the macaque principal sulcus is notable in two ways. The first is its parallel connectivity to counterpart locations in and around the intraparietal sulcus of the same hemisphere. Each subdivision of the macaque posterior parietal seems to be reciprocally connected with a subdivision of principal sulcus: there are particularly dense projections from eye-related area 7a, for instance. There are also connections from reach-related 7b, and from 7m, which is thought to coordinate eye, arm and hand in the service of reaching and grasping. [5-12a Principal sulcus-ips connections] [5-12b Principal sulcus-ips connections] The functional properties of neurons in principal cortex "in many respects resemble and perhaps mirror the properties of those neurons described in the posterior parietal association cortex, as might be expected from the intimate and topographically organized connections between these two areas," Goldman-Rakic believes (1987, 383). It is tempting, therefore, to speculate that the reciprocal projections between the principal sulcus and the intraparietal region participate in a reverberating circuit for short-term maintenance of location-related structure, she says; maybe "reciprocal circuitry ... provides a regulatory mechanism for selecting, adjusting, and maintaining a flow ... from the parietal to the prefrontal cortex", providing orientational and other attentional organization (387-8). The second important structural feature of the principal sulcus is that projections from principal sulcus to parietal areas of the same hemisphere originate among connections from the principal sulcus to the principal sulcus of the opposite hemisphere. Parietal projections "are not uniformly distributed within their prefrontal targets but are segregated in the form of vertical columns that alternate in rather precise geometric fashion with columns of callosal afferents". [5-13 Contralateral and ipsilateral connections] That is, neurons in the principal sulcus send axons both to contralateral prefrontal cortex and to ipsilateral parietal cortex (387) and would thus seem to be in a position to integrate oculomotor attention to both sides of the body. As we saw in chapter 4, eye direction is a central aspect of an established task axis: forms of environmental engagement such as reach and grasp are usually calibrated to eye position. By being in a position to inhibit, release and combine motor routines that direct saccadic eye motion, through its connections to the FEF or through parietal counterflow, dorsolateral prefrontal cortex is thus also in a position to establish and maintain more comprehensive forms of deliberate aboutness.
Parietal-frontal connections seem also to be involved in more subtle forms of attentional regulation. Experimental subjects, monkeys and humans, can be trained to fixate their gaze on a central target while actually waiting for a signal that will appear at some distance from the target. The task is called a covert attention task, since the subject is looking at one location while actually attending to another. Covert attention is a form of spatial attention in which orientational axis and attentional axis seem to be distinct, and yet fMRI scans reported by Nobre et al find that overt and covert spatial attention tasks evoke overlapping frontal-parietal networks. Since both networks include areas responsive during saccadic eye movements, Nobre et al conclude that covert spatial attention is intrinsically bound to sensorimotor systems that direct overt behavior, and that covert spatial attention tasks "may be considered as covert analogues of oculomotor tasks" (2000, 215). These findings are supported in earlier work by Posner (1990), who found that lesioned subjects who had difficulties with overt orienting had parallel difficulties with covert orienting. We have seen that, along with eye motion and foveation, overt visual orienting includes neural facilitation and inhibition in sensory areas. Lower thresholds and more rapid response are found in the parts of sensory matrices that respond to the particular distance and direction of the moment's target. Normally this area of facilitation (and an area of inhibition surrounding it) is pegged to the gaze; but in many areas it also begins before the eyes have moved. It seems that these invisible, priming aspects of orientation may also be evoked in the absence of overt saccades and fixations, while remaining "closely tied to the mechanisms involved with saccadic eye movements" (Posner 1990, 28).
There are prefrontal connections to object constant and object feature fields in the ventral what stream as well as to sensorimotor fields the dorsal where stream. Ventral prefrontal areas (as opposed to dorsal prefrontal areas) are frontal termini of the ventral stream with its specializations for object perception and recognition. Because it is connected to auditory areas in the superior temporal lobe and visual areas in the inferior temporal, the ventral prefrontal is said to be the "prefrontal area for the temporal lobe" (Passingham 1993, 159). Two subparts of the ventral prefrontal -- a lateral portion and a portion on the orbital or under surface of the frontal lobe -- have direct connections to the temporal lobe via the uncinate fasciculus (157). There are also indirect connections via the limbic system. [5-14 Human ventral prefrontal with connections] These limbic connections, along with connections to the autonomic and endocrine systems, and to taste and smell areas, make the orbital portion of the ventral prefrontal important to emotion and to emotion-driven aspects of learning. Passingham describes the orbital prefrontal as "social-affective and motivational". Passingham also suggests that the ventral prefrontal as a whole "selects the goal -- for example an object -- given the current context" (171). In the macaque, the lateral portion of the ventral prefrontal is in the inferior convexity, below the principal sulcus, and inside the lower curve of the arcuate sulcus. (F5, grasp-related premotor cortex, is just behind this curve.) The rostral or forward two-thirds of the inferior temporal lobe projects to the more anterior part of the inferior convexity, and the rear third of IT projects to the more posterior part, closer to the sulcus (401). Like visual association areas in TE in temporal cortex, the ventral prefrontal has foveal and color sensitivities: it is a continuation of parvocellular response into the frontal lobe. The ventral prefrontal is active when a monkey pays deliberate attention to a particular object, or to a detail or aspect of an object. Ventral prefrontal counterflow activity hypes preferred response to particular objects or parts of objects, and inhibits response to others. Inhibition of sensory distraction is set up through direct prefrontal-sensory or through indirect prefrontal-thalamic connections. Inhibitory modulation of sensory flow irrelevant to a task is found as early as primary auditory, visual and somatosensory cortex; an example would be inhibition found in V1 during attentive listening (Zatorre et al 1999). Prefrontal cortex also facilitates sensory flow that is relevant to a task. Imaging studies find that when we attend deliberately to tones delivered to one ear, there is enhanced activity in auditory areas contralateral to that ear, and this enhancement is found so soon after the onset of the tone that it must be an enhancement in A1 (Knight et al 1994, 168). (Facilitative activity of early auditory areas is correlated with prefrontal activity in the same hemisphere; early sensory facilitation is thus considered to be an effect of ipsilateral prefrontal-sensory connections.) Prefrontal-ventral connections, like connections to parietal nodes in the dorsal stream, are both direct and indirect. Knight believes that direct back-connections select targets of perception and action, and indirect connections routed through the thalamus amplify response to targets selected. Milner and Goodale (1993, 327) describe dorsal and ventral aspects of attention as similarly spatial in nature. An inferotemporal cell selects for a particular stimulus in the attended place, whereas a posterior parietal cell typically selects for a response to the attended place. Delay tasksMulti-stage action often requires that we maintain orientation over time or prepare one behavior while enacting another. This sort of temporal maintenance and staged preparation might occur when we are on our way to a target we see or hear at a distance. Similar forms of structural maintenance and staged priming may also occur when we get ready to act toward a target we can no longer see or hear. Delay tasks are experimental procedures that ask a person or a monkey to maintain motor and/or sensory preparation through a forced delay. Performing a delay task requires that we maintain a sensory network and at the same time inhibit, while continuing to prepare, the motor response it calls for. Delay task performance is an indicator of sorts of structural aboutness intermediate between presence and simulation, because structures mediating delayed action are initiated in the presence of a cue at a location, but must then be maintained in its absence. Delay tasks have been investigated for spatial, object perception, motor and language domains.
... our theory that the cerebral cortex as a whole is the primary site of 'working memory' implemented by time-lagged recurrent networks Pribram ed 1993, 305 Delay tasks are generally taken as tests of working memory, which is one of a set of functional terms devised by psychologists in the absence of a structural understanding of cortical processes. (The set includes long term memory, short term memory, and active memory, among others. These terms are used inconsistently in the psychological literature.) Working memory is functionally defined as the ability to 'hold items in memory' temporarily. It is described as limited both temporally and in terms of the number of items that can be remembered (Goldman-Rakic 1994, 114). In structural terms, working memory is the ability to maintain structure for purposes of staged or delayed action. Working memory structure should probably be thought to include any part of an attentional net engaged in maintaining structure in this way, as well as perceptual and motor structure being maintained for purposes of a task. Modular theories of working memory have looked for a locus where representations may be kept active in a buffer, and, because people with prefrontal lesions have difficulty with delay tasks, the 'working memory module' has been thought to be prefrontal. Functional imaging studies have found, however, that although the prefrontal cortex is critical for integrative cognitive functions requiring working memory, it is unlikely that this capacity resides in specialized modules in prefrontal regions. More likely, the spectrum of cognitive capacities involving prefrontal cortex is supported by interactions in the extensive bi-directional connections between prefrontal cortex and numerous cortical, limbic, and subcortical regions ... Knight et al 1999, 161 Sustained activity during delays is observed in temporal and parietal extrastriate cortices as well as in frontal regions (Haxby et al 2000, 145). Nonetheless, there is evidence that prefrontal areas have a specific role in delay tasks: The idea that sustained activity in posterior visual areas reflects top-down influences from prefrontal cortex is supported by the results of deactivation studies. Fuster et al have found that delay activity for object information in the inferior temporal cortex, though not eliminated, becomes markedly less selective during reversible deactivation of prefrontal cortex by cooling. Similarly, Goldman-Rakic and Chafee have found that delay activity for spatial information in the posterior parietal cortex is greatly diminished during prefrontal deactivation. Ungerleider et al 1998, 889 The more complex the task, and the longer the delay, the more prefrontal involvement is found. The role of prefrontal cortex in delay tasks thus seems to be similar to its role in attention without delay: by means of direct and indirect counterflow connections, it inhibits and facilitates both motor and sensory structure as required for a task. Facilitation is needed to keep sensory structure active through the delay, and to prepare motor response. Inhibition is needed both to hold off habituated response until cue conditions are restored and to prevent new sensory conditions from disturbing sensory maintenance.
Short-term sensory maintenance and short-term motor set -- two specialized functions of dorsolateral prefrontal cortex -- are "mutually complementary and temporally reciprocal", Fuster says (1995, 174), because motor preparation is anticipatory while sensory maintenance is "retrospective". Perception-action dualists describe sensory maintenance as memory, because it may be sentient and is thus 'mental', while describing motor maintenance as 'motor set', because it tends not to be experienced as such. Motor and sensory maintenance seem to occur by the same sort of mechanism, however, and they are interactive. Motor memory is sometimes called motor trace, procedural memory, habit, program, plan, or schema. In structural terms, motor memory, like sensory memory, is a structural potential that can be evoked under the right conditions. When activated, the structure that is the memory is also the operative structure that organizes motor behavior (Fuster 1995, 176). Joaquin Fuster believes that, within a delay task, short-term sensory memory and short-term motor set occur in distinct though intermixed cell populations in prefrontal cortex. A case in point is the prefrontal cortex of the frontal eye field where, within a small region (approximately coinciding with area 8) we can find cells that have short-term memory and motor-set properties in the context of a certain aspect of behavior (i.e., ocular motility). Fuster 1995, 277 Sensory response, which is perceptual memory of the cue, is "usually activated to a maximum during or immediately after the cue" and tends to diminish as the delay progresses. Motor activity, meanwhile, tends to increase as the delay progresses, so that "the gradual deactivation of the first as the second becomes activated suggests a transfer of excitation from the perceptual to the motor component of the network" (270). Motor-coupled cells intervene at different times before the delayed response to a cue, Fuster says; there seems to be an inverse relationship between the lead time of the anticipatory discharge of motor cells before movement and the specificity of those cells. Many studies of motor delay have been eye-related, but Rizzolatti describes similar (though accidental) delay response in the monkey reach area of frontal F4. Neurons that normally respond when there is something present in the space adjacent to the skin of the face or arm are found to "continue to fire when, unknown to the monkey, the stimulus previously presented has been withdrawn, and the monkey 'believes' that it is still near its body" (Rizzolatti et al 1997, 190).
Patrica Goldman-Rakic, who studies monkey frontal cortex at the fine grain of single cell testing, describes separate working memory systems for locations, objects and language. The relation between locational attention and locational delay was tested in PET research by LaBar et al (1999); it was found that the same neural populations within frontal and posterior parietal cortices were active in both sorts of task. Areas active in delayed saccades to a target location include motor, frontal eye field and superior colliculus areas as well as parietal and prefrontal cortex -- the same areas as are active during purposive eye movements to targets that are currently visible. As one would expect, the prefrontal area active in working memory for location is at the frontal terminus of the dorsal stream in dorsolateral prefrontal cortex -- the prefrontal area described as active in organizing and maintaining deliberate eye motion. Monkeys with lesions to this area cannot regulate eye movements to remembered targets (Goldman-Rakic 1987). PET studies of delay tasks find that the network active during delays also includes many of the same neural populations within frontal and posterior parietal cortex as are active in covert spatial attention. Parietal areas are also involved in spatial working memory (Murata et al 1996). A study of single-cell response in the grasp-related parietal area AIP found a majority of cells of both object-type visual-dominant and visual-and-motor neurons showing sustained activity during the delay period before manipulation. Cells showed the same selectivity for shape and/or orientation during delay maintenance as during object perception. Like parietal and FEF areas, the macaque dorsolateral prefrontal has both motor and sensory response during delays. Sensory cells may be driven by both auditory and visual events. Visual sensitivities vary with the current position of eye, head, and neck (Goldman-Rakic 1987, 383). The location of a homologous spatial delay area for humans has been in dispute. Brodmann's 46 has been named, but both Ungerleiger (1998, 888) and Courtney (Courtney et al 1998) believe, on the basis of imaging studies, that in humans the region activated by working memory for locations also lies just anterior to the FEF. The ventral prefrontal area that is active when a monkey attends to an object is also active when it remembers an object through a delay. [5-17 Dorsal and ventral delay routes] Courtney's imaging studies find that during working memory for object identity or visual features, there is activity in IT areas 37 and 18/19 as well as in ventral prefrontal cortex, suggesting that ventral prefrontal back-connections are maintaining posterior sensory activity through the delay. In non-human primates, delay activity in temporal cortex, unlike that in prefrontal cortex, is disrupted by intervening stimuli. These results, supported by our results in humans, suggest that the dominant function of temporal cortex is perceptual but that this region also participates in maintaining a working memory as long as it is not recruited for the perception of new stimuli. Courtney et al 1997, 611 Again, the role of prefrontal cortex in working memory maintenance seems to be like its role in attentional maintenance: in the absence of the object, as in its presence, prefrontal counterflow activity hypes preferred response to particular objects, and inhibits response to others. Human ventral prefrontal cortex also includes language areas whose description will wait until Chapter 6. Memory and simulation... under certain conditions a memory network becomes and stays active above a certain level or threshold and, in that state, remains useful for current behavior Fuster, 1995, 238 If we define simulation as a structural state by which we are about, or adapted to, something we are not with at the moment, delay capabilities both are and are not a form of simulation. With delay tasks we begin by being about something we are with, and end by being about something we are no longer with: the process begins as perceiving and ends as remembering. Not all simulation is memory; the wide net of a simulational moment must be able to fall into structures that have never existed before. We can in some sense be about things we have never been with. At a finer grain, though, any sort of imagining will have to be done by means of structure that exists because we or our ancestors were actually with something or other. I will approach the more inclusive class of simulation through a description of memory because the structural principles of the larger category are more easily understood in the context of the subclass. The most important theoretical point here is that memory is understood as structural change allowing subsequent structural reactivation by means including these changes (Hebb 1949, Fuster 1995, Damasio 1999). Rather than being thought of as the function of various kinds of memory modules, Fuster says, memory "can be legitimately construed as a property or state of any of the components of the central nervous system" (1994, ix). In this view long term memory is the long-lasting structural potential -- 'passive' or 'latent' -- that, if triggered, will re-install an earlier structural state. Short term memory is a similar structural potential with a much shorter effective span. Active memory, in contrast with both these terms, would be mnemonic structure active in the moment. It would include active forms of both long and short term memory structures. The term is sometimes used as if it were equivalent to Edelman's core dynamic subnet. If we use active memory in that way, we are, however, left without a term for sectors of the wider net that are mnemonic and active but not part of the sentience subnet. Conscious and nonconscious memory should probably both be considered active memory. Fuster describes two basic hypothetical principles of memory understood as network alteration and reconstruction: 1. The first is that the cortical network that keeps a memory active is largely the same network that supports and defines that memory in its permanent, and, at other times, passive state. The two may, in fact, be identical, as in the case of an internally evoked long-term memory or in the case of our monkey retaining a well-learned stimulus through an enforced delay ... 2. The second hypothetical principle is that recurrent excitation through reentrant circuits plays a critical role in the sustained activation of a memory network ... Here we are postulating that this mechanism is at the basis of not only the short-term process that leads to long-term memory but also active memory in general, whether or not it has a definite term. In addition, we are extending the applicability of the mechanism to the linkage between distant components of the associative cortical memory network in the active state. 1995, 253 The structural principles outlined above apply to any of the forms of memory that have been distinguished. Implicit and procedural memory -- non-verbal forms of memory that include remembering how to do something -- are two variants of structural potential reactivated. The network activated in explicit memory might have more structure included in the core dynamic subnet and might connect to more linguistic structure. Declarative, episodic and semantic memory may similarly be subsumed as forms of a reactivated net; Fuster calls them all perceptual memory. Each will have some wide net differences; semantic memory, for instance, is thought of as anchored in larger networks, with more associations than declarative memory. Other memory differences are differences of cognitive domain, and so presumably differences in areal involvement in a net. As with delay tasks, sensory areas found to be active during long or short term memory are the same general areas that are found to be active during perception in that modality. Memory for places requires hippocampal or parahippocampal involvement, as described in Chapter 4. Memory for faces activates face areas in the ventral stream. And so on. The distinctions that are most valuable in this approach to memory are process distinctions that can be understood in terms of mnemonic dynamics. Separate processes include setting mnemonic structure, reactivating it, maintaining it, and making use of it. Setting and reactivating memory seem to need different subnets; hippocampal lesions that disrupt the former need not disrupt the latter. Reactivation happens in many ways -- in the midst of perceiving and acting, by means of language, or even in purely endogenous ways by shifts of operating state, as when we are beginning to fall asleep. Like other sorts of simulation, memory reactivation can be deliberate or not. Deliberate remembering, like deliberate perceiving, requires prefrontal involvement. Little is known about how mnemonic nets form. It is as if water, rippled by a breeze, is structurally altered so that it will later be able to ripple itself. Fuster describes one sort of embedded network component that may be important to the process. Single cell studies of delay tasks with monkeys have found two sorts of delay-related response. Some cells responsive to the cue respond whether or not the cue is to be remembered. Others, embedded in the same areas, respond only when the cue is to be remembered and is no longer present; response in these cells increases over the delay period. When the monkey has accomplished its delayed task, response in these cells stops, even if the monkey is looking at the restored cue. Fuster thinks of these cells as memory cells. Fuster's memory cells are found in both frontal and posterior cortex. Memory cell response in the prefrontal is sustained through the whole of the delay period, and its magnitude varies with the degree of learning, the difficulty of the task, and the level of performance (1995, 177). Some memory cells have been found as early as V1. Tactile memory cells are common in primary and secondary somatosensory cortex. Prefrontal counterflow and deliberate imagining
Having tracked the role of prefrontal counterflow from overt orientation, to covert attention, to orientation maintained through a delay, we are now in the position to make a jump to the role of prefrontally organized circuits in deliberate imagining. Presence structures are initiated and guided from the periphery of the nervous system, by contact with the world. For simulational structure, initiating and guiding functions must be endogenous. The question of how simulation is possible, then, comes down to a question of how closed-system self-structuring can be initiated and maintained in the absence of exogenous perceptual structuring. In Chapter 4 I described task axes anchored at a particular spatial location as preparing perception and action simultaneously and calibrating them to each other. During perception and action, task axes are maintained and switched by maintaining or switching bodily orientation. Evidence is accumulating that the motor circuits that set up a task axis, particularly the circuits for engaging and disengaging the eyes and for tracking moving objects, are also important to simulational attention. We have seen a continuity of task axis structures from actual to simulational orientation. Object and location delay tasks are instances of focalized, coordinated motor and sensory structures maintained in the absence of a cue. Covert attention is an instance of priming aspects of attention that continue to be keyed to structures for oculomotor orientation even when there is no apparent orientation to what is being attended. A common element is prefrontal involvement in deliberate gaze control. Martha Farah compares the process of voluntary visual imagining with what happens when we attend carefully to something we are seeing (2000, 972); deliberate attention to for instance the shape of a bear's ears requires prefrontal counterflow priming of structures used to perceive an aspect of an object. Imagining could be initiated by the same counterflow influences that come into play when a stimulus object needs particular attention. When we want to examine something in detail, we have to go on directing ourselves toward it and so go on being about it, accumulating and refining structure with which to see it. If we want to imagine something in detail, we will have to go on as if directing ourselves toward it, so that we can accumulate and refine structure with which to seem to see it. Seeming to see something requires seeming to look at it -- that is, seeming to orient ourselves toward it, seeming to direct our gaze, seeming to focus, and the rest.
Representationalist philosophies of imagining have thought of 'mental images' as the product of a separate faculty or module, 'the imagination'; but cerebral blood flow patterns as monitored by PET and fMRI scans show many overlaps in networks active during presence and simulation. What is found in imagining studies has invariably been a wide net similar to the network coordinating action and perception. Like networks active when we are about what we are with, simulational networks include motor as well as sensory structure.
Sports psychologists have discovered that seeming to move can improve later performance as much as actual physical training, presumably because motor performance and imagining involve some of the same central structures (Keil D et al 2000). Crammond's single-cell studies of monkeys found that spatially tuned parietal and pre-central motor control areas respond similarly whether a movement is executed, or planned but unexecuted. Crammond concludes that "motor imagery and execution involve activation of very similar cerebral structures at all stages of motor control with the proviso that the final motor output is not expressed during motor imagery" (1997, 54). Motor simulation studies include a recent PET study of imagined grasping showed foci in prefrontal, premotor and parietal areas (Decety et al 1994). When experimental subjects imagine moving their eyes to look at a target, activity is found in frontal and supplementary eye fields that are important to voluntary control of saccades (Mellet et al 1998, 136). In general, PET studies of motor simulation find premotor activity especially marked (Fuster 1995, 181). Primary motor activity is often, but not necessarily, also reported. Parietal areas are usually involved bilaterally, particularly gaze and reach areas 7a and 7b (area 5 in monkeys). Somatic and other sensory areas may be active. Over all, these networks are very similar to the nets found when we actually perceive and act. There is also behavioral or psychological evidence that networks responsible for motor simulation are like networks responding when we execute behaviors. Experiments comparing simulation and execution tasks find them constrained in similar ways; "we cannot imagine playing the piano faster than we can actually move our fingers", for instance (Crammond 1997, 54). Heart and respiration responses are similar in the two cases. Steven Kosslyn has argued that because part of the process of motor control involves counterflow priming of sensory systems to respond to the expected consequences of motion, motor simulation may be an important organizer of sensory simulation even when action is not being planned (Thompson and Kosslyn 2000, 538). Many studies have found motor involvement in simulation that is ostensibly sensory rather than motor. Kosslyn's studies of 'mental rotation' invariably find motor participation, which can include hand as well as eye areas. He concludes that seeming to see a rotating three-dimensional form actually requires motor participation (1998b, 77).
In a recent PET study of visual imagining, Kosslyn and Thompson report that "a total of 21 areas were activated, with two-thirds of them activated in common by imagery and perception" (2000, 980). There are similar results for auditory imagining: Halpern and Zatorre (1999, 698) report, for instance, that subjects imagining the continuation of a tune show "remarkable similarity in cerebral bloodflow patterns in the perception and imagery conditions". Cortical visual areas that are used both for imagining and perceiving are known to include areas specialized for response to form, motion, location, color, object recognition, and so on. The role of these areas in visual imagining seems to be similar to its role in visual perception, Farah says: in most (but not all) cases of selective visual impairments following damage to the cortical visual system, patients manifest qualitatively similar impairments in mental imagery and perception. Spatial attention impairments for the left side of the visual scene also affect the left side of mental images. Central impairments of color perception tend to co-occur with impairments of color imagery. Higher-order impairments of visual spatial orientation sparing visual object recognition (and the converse) are associated with impairments of spatial imagery sparing imagery for object appearance (and the converse). Finally, hemianopia resulting from surgical removal of one occipital lobe is associated with a corresponding loss of half the mind's eye visual field. Farah 2000, 968-9
Hanna and Antonio Damasio's hypothesis is that simulational reconstruction is triggered and organized from mediational convergence/divergence zones within association cortex: A convergence zone is located within a convergence region. We envision that there are in the order of thousands of convergence zones, which are all microscopic neuron ensembles, located within the macroscopic convergence regions that have been cytoarchitectonically defined and that number about one hundred. 1994, 71 Because they are located at crossings in the distributed streams of activity propagated throughout association cortex, convergence zones can be activated from many directions: In essence, a convergence zone is an ensemble of neurons within which many feedforward/feedback loops make contact. Its connectional structure is as follows: a convergence zone receives feedforward projections from cortical regions located in the connectional level immediately below, sends reciprocal feedback projections to the originating cortices; sends feedforward projections to cortical regions in the next connectional level and receives return projections from it; is influenced by a broad class of cortices concerned with attentional control and response selection ... This rich network of extrinsic connections is complemented by a complex network of intrinsic intralaminar and interlaminar connections. 1994, 71 Activity relayed through convergence sites makes use of local and long-range cortical connections, subcortical connections, cross-stream ventral-dorsal connections, and even cross-hemisphere connections. Convergence architectures originally form during active engagement with environments. The location and connectivity of convergence sites is a function of environment-organism interaction at both genetic and epigenetic time scales. Because environments and bodies are similar, individuals are expected to develop similar large-scale convergence organization; because aspects of convergence function result from individual learning, there will also be individual variability of microcircuitry within convergence regions (Hanna Damasio et al 1996, 505). Since they form at intersections, convergence regions are found near, and between, areas committed to separate aspects of contextualized response. They are associated with areas responding at different categorical levels: object-constant sites in anterior temporal cortex, location-constant sites in the SPL, and act-constant sites in premotor cortex would be examples of basic-level categorical response alternatives, while feature-constant sites like unimodal form and color sites in the inferior temporal would be examples of sub-object perceptual alternatives.
When we use a hammer, multiple convergence sites will be active parts of the network engaged with feeling, seeing and hearing a hammer, using it, and feeling oneself using it. Away from active engagement, convergence sites are points of access through which simulational wide nets can be reconstructed. When we name, remember, think of and recall facts about an object, convergence sites are crucial. Simulational reconstruction would involve a cascade of activity through subnets within convergence sites, each subnet adding its categorical shading while evoking related networks at other levels. Once established, simulational structure, like perception/action structure, would be a network of synchronous, reentrant activity widely distributed through many cortical and subcortical regions. Kosslyn's guess is that deliberate sensory simulation proceeds as follows. Counterflow networks centered in prefrontal cortex first activate subnets in multimodal sites that include frontal-parietal localization gradients, object identity areas in temporal cortex, and inferior parietal areas dealing with inter-object spatial relations. It is possible that structure active in these relatively high order convergence areas may also be included in core dynamic structure (Farah 2000, 972). Cascading activity fanning outward from these high-order sites would evoke lower order convergence sites in unimodal sensory areas like the inferior temporal gyrus and the fusiform gyrus (on the lower surface of the temporal lobe) which are important to color and face perception, and in the middle temporal gyrus, important to motion perception. Activity is usually found in these areas in PET studies of visualization tasks (Thompson and Kosslyn 2000, 537). An area of posterior temporal cortex with a coarsely retinotopic organization has been suggested as supporting summarily or abstractly imagined visual form (Kosslyn and Thompson 2000, 982). Although there is wide agreement that secondary sensory areas are active when we are imagining (Mellet 1998, Roland and Gulyas 1994, D'Esposito 1997, and others), there is conflicting evidence about participation of primary sensory areas (summaries in Mellet 1998, Solms 1997, Pollen 1999). Kosslyn, like Damasio (1994 and 1999), believes all sentient experience requires activity in primary sensory areas. If Kosslyn and Damasio are right, sentient imagining requires that backflow activity from convergence zones in secondary sensory areas further activate structure in primary sensory areas in occipital and superior temporal areas, and in the somatosensory strip of parietal cortex (for visual, auditory, and somatic imagining, respectively). [5-19 Areas active with visual imagining] PET studies of detailed visual imagining often do find activity in occipital cortex as well as in ventral temporal cortex (Farah 2000, 980). People who report vivid visual simulation show more occipital involvement than people who do not (Farah 2000, 969-70). LeBihan et al (1993) find visual imagining tasks activating widespread regions of occipital cortex, including V1/V2. Farah (1997) finds V1/V2 activity when subjects imagine common objects, and Kosslyn et al (1995) find that the location of primary visual activity during visual imagining correlates with size of the object imagined, in the way it would if an actual object were being seen. Ishai and Sagi (1995) find that visual imagining has priming effects on perceptual tasks, and that these effects extend back to V1 at least. When we try to imagine in more visual detail, Kosslyn suggests, dorsal systems for deliberate saccades and fixations will be invoked to as-if look for particulars -- that is, to evoke further structure in secondary, and then primary, sensory areas (Kosslyn and Thompson 2000; Thompson and Kosslyn 2000). The belief that sentient simulational experience requires primary cortex participation has been somewhat weakened by reports that people with V1 damage and cortical blindness can experience vivid visual hallucinations; but this is rare, and cortical localization very uncertain (Solms 1997). Posterior counterflow and dreamingThe large family of simulational phenomena includes forms of imagining that are not, or are not primarily, deliberate. The structural means of these sorts of imagining are evoked and maintained, not by prefrontal counterflow, but in various other ways. Contact with representational artifacts and events is a source of simulational structure that I will describe in the following chapters; in this section I want briefly to describe the paradigmatically involuntary category of simulation during sleep. In slow-wave non-dream portions of the sleep cycle, there is widespread deactivation of both the cortex and its thalamic gateways. The musculoskeletal system is paralyzed. Relative to slow-wave sleep, dreaming is partial or selective waking. During the dream state the metabolic activity of the brain may be equal to or greater than it is during the waking state (Hobson et al 2000, 1341). The sleeper is not paralyzed, but is able to shift position. The eyes perform rapid saccades behind their closed lids. Thalamocortical circuits show oscillation frequencies characteristic of waking perception. PET/fMRI investigation of dream sleep finds high positive correlation with activity in posterior association cortex and high negative correlation with activity in frontal areas described above -- areas necessary to deliberate acting, perceiving, remembering, and imagining. Braun et al conclude that when we are dreaming "association cortices and their paralimbic projections may operate as a closed system dissociated from the regions at either end of the visual hierarchy that mediate interactions with the external world" (1998, 91). Differences among slow wave sleep, dream sleep and waking are differences of global operating state, and as such they are instituted by neuromodulator diffusion. Relative inactivation of frontal cortex is associated with absence of one category of neuromodulator (aminergic), while preferential activation of emotion-mediating subcortical and cortical limbic structures is associated with the preponderance of another (cholineric) (Hobson, Pace-Schott and Stickgold 2000, 1352). As it turns out, the parts of the brain that are most active in dreaming are adjacent and/or heavily connected to these cortical and subcortical limbic and paralimbic structures. Rather than being driven by thalamic-sensory activity originating at sensors on the periphery of the body, or by counterflow activity directed from prefrontal cortex, then, dreaming seems to be driven from motivation- and emotion-related structures at the center of the brain. There is widespread parietal deactivation during sleep, but the inferior parietal lobules, whose medial or internal surfaces abut limbic cortex directly, are among the areas whose activity is positively correlated with dream sleep. The IPL seems in fact to be a critical focus in nets active when we dream. There is global anoneira, a total cessation of dreaming, following inferior parietal lesions, and this occurs equally often with lesions of either the left or the right hemisphere (Solms 1997, 140-141). Solms thinks activity in dream networks flows from limbic structures into the left IPL, from there to the right IPL, and from the right IPL is distributed bilaterally to matrices in temporal cortex. Along with IPL activity, PET/fMRI investigation of REM sleep has found that density of sleep-typical rapid eye movement is positively correlated with activity in occipitotemporal association cortex and in hippocampal and parahippocampal regions (Braun et al 1998). During waking perception, these areas are particularly responsive to objects and to large and small places. During dreaming, activity in these areas occurs in clusters in both hemispheres. Solms notes that "modality-specific disorders of dreaming apparently do not occur with lesions in unimodal (primary and secondary) somatosensory, auditory and motor cortex", but that "modality-specific imagery disorders do occur with lesions in unimodal (secondary) visual cortex" (1997, 169). Higher-order visual areas, especially in the ventral stream, have been found to be more active during REM sleep than they are when we are awake or in slow wave sleep. Focal activity is found especially in Brodmann's areas 19 and 37, which include ventral visual areas V3, V3A, and V4. Movement-sensitive V5 in the dorsal stream is also active. The flow from inferior parietal to temporal cortex can be understood as a form of posterior counterflow, as described earlier in this chapter: it is a flow from multimodal association cortex to unimodal association cortex, and then possibly to primary sensory cortex. This form of posterior counterflow resembles and may be the same as the later stages of deliberate imagining, in which lower order sensory structures are also activated from higher order convergence zones. Martha Farah's 1989 hypothesis that seeing and seeming to see result from activity of a single common system was based on evidence that they are similarly affected by cortical damage. This sort of evidence has been found also for dreaming. Lesions in occipital-temporal areas can result in specific deficits of both the perception and memory of forms, color, faces, letters, spatial relationships, or moving things; revisualization deficits (visual irreminiscence) in these areas are generally paralleled by dream deficits (Solms 1997, 23). Bilateral lesions of form-sensitive V4 in Brodmann's 19 result in 'blank dreams', that is, in dreams without visual involvement. There is controversy about the role of primary visual cortex in dreaming, as there is about the role of primary visual cortex in deliberate imagining. Studies by Roland and Gulyas suggest that activation of associational levels of visual through-streams is sufficient to evoke and sustain simulational visual experience (1994). PET evidence from Braun et al (1998) is that dreaming activates only extrastriate visual areas, and that striate areas V1 and V2 are unchanged compared to wakefulness, and actually less active than they are in deep sleep. Dreaming and imagining can be distinctly visual and at the same time very unlike visual perception. It is not easy to describe the visual difference, because we are not, after all, able to examine visual simulation as if it were a picture: we cannot look at it and see what it looks like. We are able to say quite generally that sometimes we are seeming to see color, and sometimes color is not part of the experience. We can also say that we often don't imagine much visual detail. Dreaming and reverie can be visually quite bizarre, but at the same time quite undetailed. This generality accords well with the generality of function we find in anterior temporal cortex, which responds to objects of particular kinds and of particular forms, regardless of their size, position, color or individual identity. It is possible that when we dream visually, at least sometimes, the core dynamical net responsible for the visual aspect of our experience is active primarily in these unimodal or multimodal associational areas rather than in the much more expansive visual matrices of V1 and V2. But there is evidence also for the importance of V1 and V2 in visual hallucination. Pollen (1999) cites a description by Gloning et al of a patient who underwent a partial right occipital lobectomy for a brain tumor that had been triggering complex visual hallucinations. After removal of right hemisphere occipital areas the patient developed a visual hallucination of a man moving slowly from the left visual periphery to the right visual periphery. When the man reached a place in the left visual field that would normally have activated the portions of V1/V2 that had been removed, he disappeared temporarily, reappearing in the right visual field and moving on to the far right periphery. It seems likely that the amount of counterflow into primary sensory cortex differs across dreams and across dreamers, and it would make sense to think that the difference between dreaming and hallucinating might be a difference in degree of primary cortex involvement. However, Pollen believes -- beyond suggesting primary visual cortex is necessary for hallucinatory visual experience -- that this story suggests the role of higher order visual cortex in experienced visual simulation may be in triggering or organizing primary cortex: ... the hyperactive clusters of neurons within the temporal lobe that trigger complex, well-formed visual hallucinations ... are not sufficient on the basis of their own activity within the temporal lobe, or feed-forward connections beyond the temporal lobe, to generate phenomenal visual experience, because the critical co-requirement for the activation of visual experience was the preservation of early visual cortical areas. 1999, 11-12
Clinical study finds the prefrontal essential to memory, planning, and language, and to deliberation of every kind. The material in this chapter suggests that these quintessentially human specialties are best understood in the context of prefrontal involvement in complex action, which may make use of all of them. Having described some of the mechanisms of deliberate and involuntary simulational perception and action I am ready to go on to representing, which makes use of, and depends on, both. With representing, we come to a vast subclass of biological functions, those socially learned and culturally transmitted. Goldman-Rakic describes prefrontal cortex as a working memory system with sensory, mnemonic and motor components, called into play during motor and sensory maintenance across delays, and also when whatever is being remembered or imagined has not recently been present. She believes interactions and coactivation of these working memory centers within cortical networks together constitute the brain's "machinery" for higher level cognition (1994). Two aspects of frontal function are particularly relevant to these encultured functions. One is the simulational capability described above. Simulational capability is, for instance, important to the ability to learn rapidly from explicit instruction: when we are given instructions we imagine the states or events described, and in the process of doing so begin to structure ourselves to enact those instructions. Motion can be selected on the basis of learned conditions at the level of premotor areas, which are large in monkeys but especially well developed in humans, Passingham says. "Whether the subject is a monkey or a patient we may say that the effect of damage to the lateral premotor cortex is to impair the retrieval of the appropriate movement given the instruction. It does not matter whether the instruction is given by [experimental] learning, by word of mouth, or by mime" (1993, 57). With this ability to organize motion from simulational states, a second specialization of prefrontal cortex becomes essential: it is the ability to monitor and check or correct both action and the processes that are tending toward action. Deliberational self-monitoring is particularly important to creatures able to fall into simulational states. A creature able to space out must also be able to determine whether its current state is relevant to its current circumstance. Courtney et al describe a cluster of regions in the dorsolateral prefrontal active in monitoring mnemonic performance (1997, 610). It is possible that these areas may also be active in determining that one is remembering or imagining.
Part III. Representing and thinking |