BRAIN AND IMAGINING


Should we ever succeed in making the shift, we would be properly at home in our physical universe for the very first time. P.M.Churchland 1989


1. imagining


I am in the small living room of our first house. I am five years old. It is Sunday afternoon. I am alone but I can hear other people in the kitchen. The room is dark but there is bright sunlight in a tree outside the one small window. I have just discovered an ability I didn't know I had. It is this: if I close my eyes I can make pieces of wood fit together to make a birdhouse. I can see them being set together. In real life I have seen a bluebird flying out of a binder-twine can among the farm machines in the long grass behind the pump house, but I have never seen a birdhouse. There is a diagram of the parts of a birdhouse in the encyclopedia and I am aware that what I am seeing is like that diagram. What I have discovered is that I can see it on purpose, without the picture in the encyclopedia, and I can make it move. I can set the parts together and finish the birdhouse. I try again. There it is. What an interesting power.

I don't remember how I happened to find this power; what I remember is the sensation of discovering it. It was the discovery of psychic action -- a kind of action that really happens though it isn't real. It does not change anything in the world and no one can see it, but it is an action, because it is something I do, and it is more completely my own action than anything I have done before. -- Something like that.


2. cognitive order

Imagining is interesting because it is autonomous and yet coherent: it is not a relation to the things and events around us while we are imagining, yet it is organized as if it were a relation to things or events of some kind. The complexity of this organization can be impressive, as when we dream a long journey through landscapes we have never seen or work out the planting scheme for a perennial garden. Or it can be extraordinary: Dorothy Richardson writing Pilgrimage.

When we are perceiving we can think of the structuredness of our experience as given by the things we perceive, but when we imagine it is clear that the structureness of our experience is somehow our own structure. The fact that we are able to imagine implies the existence of cognitive order and the experience of imagining is a demonstration of the particulars of that order.


3. mentalism and imagining

But how are we to understand cognitive order? Aristotle spoke of perceiving and imagining as being like digestion and locomotion in being aspects of the self-structuring activity of animal bodies. For Aristotle, cognitive order is one kind of physical order. That was a good beginning, but without a developed biology, there was not much more to be said along those lines. In the long meantime the locus of cognitive order was taken to be the mind or soul, both taken as distinct from the body. Our ability to imagine contributed to this detour. It is a strange fact that when we perceive, we do not perceive the means by which we perceive. It would be impractical to do so. We perceive instead the sorts of things it has been useful to us to perceive. And when we think and imagine it is the same: we think and imagine by means of physical structures we are not aware of as such.

Given that we are blind to our own nervous systems in action, imagining has seemed to be disembodied. We have tended to think of seeing as presence in the world and of imagining as absence from it; when we imagine we seem to perceive, and when we seem to perceive we really seem to PERCEIVE, so we seem to ourselves to be perceiving elsewhere, off-world, 'among our thoughts', in 'the world of the imagination', in a spirit world.

The seeming detachability of presence from physical presence is as it were the empirical basis for mind-body dichotomies. 'The mind' or 'consciousness' is thought of as the part of us that can be in the physical world, perceiving physical things, and then in a dream world, perceiving dream things, and then in a thought world, perceiving thought things.
A 'mind' or 'consciousness' thus able to perceive, think and imagine even when it has left the physical world cannot be explained in physical terms. Its orderedness cannot be thought of as the orderedness of a perceiving, imagining or thinking body. What happened historically is that cognitive order was thought of in terms of its results rather than its means, but its results were spoken of as if they WERE means: reasoning is taken to be a manipulation of concepts, imagining is accomplished by means of images, and perception is an arranging of percepts. And so on. The fact that we are able to perceive, imagine and think is taken to imply the coexistence of separate faculties of perception, imagination and reason.

We can think of mentalist descriptions of cognitive order as a protopsychological functionalism that was gradually arriving at useful notions of what needs explaining. After all, we have gone on wanting to know how we are able to perceive, imagine and reason. But mentalist descriptions are confused when they purport to be explanations.

Some say the Senses receive the Species of Things, and deliver them to the Common-sense, and the Common-sense delivers them over to the Fancy, and the Fancy to the Memory, and the Memory to the Judgement, like handing of things from one to another, with many words making nothing understood. Hobbes, Leviathan ch. II

We shall unearth this naive ontology of images as a more or less implicit postulate of all the psychologists who have studied the subject
. Sartre 1936, 72


4. transition

The effort to construct a brain-based theory of mind can be seen as part of the long project of slowly working our way into an understanding of ourselves as physical creatures in inalienable contact with the universe that has formed us. We have inherited a discussion of cognitive order that, at least in explanatory terms, has been following a red herring for several millennia. We have been working out the biology of cognition for only a couple of centuries. We are in transition. With the present boom in neuroscience, this transition has been accelerating. We are rushing toward a fully physicalist, causal, brain-based theory of cognitive order. But our present theories are showing the marks of transition: they are strange mixes of mentalist and physicalist vocabulary, metaphor and explanatory style.

This attempt to put together a brain-based theory of imagining is one attempt to get the transition right. Imagining has a long mentalist history. It does have good currency as a phenomenological term. There is no other word for what someone is doing when she seems to see the parts of the birdhouse set together. The ability to do that marvelous thing still wants explaining. But imagining probably does not have current uses as an explanatory term: the ability to imagine is not usefully describe as being given by the imagination. It IS usefully described as given by a combination of general and specific organizational abilities of nervous systems.


5. naturalizing imagining


The picture of imagining I will describe here has been pieced together from suggestions in the recent work of neuroscientists (Changeux, 1985; Damasio, 1995; Edelman, 1989; Pribram, 1991), philosophers (P.S. Churchland, 1989; P.M.Churchland, 1989 and 1994; Dennett, 1991) and psychologists (Kosslyn, 1995). Aspects of these theories were also anticipated by Wittgenstein, Hebb, Gibson, Langer, Sartre, Vygotsky, Piaget, and by even earlier writers including the gestaltists and Freud.

There are three main pushes in a neural redescription of imagining. One is an effort to revise our terminology so we speak more exactly, less mentalistically, about what is going on when we imagine. The second is an effort to understand what imagining is useful for, and how it could have evolved. The third is an effort to imagine what is actually happening in a brain as we imagine something, to picture imagining as a spatiotemporal event. These three pushes depend on one another, but I will have to describe them separately.


6. watching how we talk

Those of us who have stopped arguing about mind-body identity are still working our way into habits of discourse that assume it deeply enough so that our vocabulary and structuring metaphors are no longer at odds with that assumption. When we talk about imagining it is for instance chancy to talk about the image, mental imagery, and image processing, because those sorts of construction set us up. If we talk as if we are really seeing an imagined object rather than seeming to see an object, we are next door to thinking of ourselves as seeing a representation, i.e. as seeing the 'internal' object by means of which we are imagining. Similarly if we talk about image processing we are set up to think of what we are imagining -- that birdhouse diagram -- as being the representional structure that is being processed or transformed, rather than thinking of the whole event -- imagining the parts of the birdhouse coming together -- as being a process.

There is another trap in thinking of imagining as internal; the structures by means of which we imagine are, like the structures by means of which we perceive or initiate action, inside our skulls; but perceiving and acting are not internal -- since they are physical relations between organisms and parts of the world-- and seeming to perceive and seeming to act are, as such, nowhere. The birdhouse I imagined was not in the world, not in my head, not even in 'the imagination'. It is only when I speak of really seeing it that I am led to wonder where it is.

I am suggesting that we talk about imagining as seeming perception and seeming action because it works: the change in locution immediately sorts out disputes about whether, for instance, imagining a birdhouse 'relies on' a pictorial or a propositional format of representation. The answer of course is that the question is nonsense. Pictures and sentences are both things we can see or imagine, but the MEANS by which we see or imagine them are neither pictures nor sentences -- they are four-dimensional shapes of neural activation, which we tend to call representations.

Representation is hazardous in the same way mental image is hazardous: it sets us up to equivocate between what we seem to see and the unseen means by which we seem to see it. That equivocation is the founding equivocation of mentalist theories of mind, and uncareful uses of representation are in fact the most troubling mentalist remnants in would-be physicalist contemporary theories of mind. Several theorists, Edelman and the recent Dennett, try to work without it, and I will do the same.

Something that really isn't an image could be the very thing someone is talking about under the guise of an image. Dennett 1991, 94

A person picturing his nursery is, in a certain way, like a person seeing his nursery, but the similarity does not consist of his really looking at a real likeness of his nursery. Ryle 1949, 72

We must carefully distinguish the mental objects themselves from the operations and computations carried out with these objects. Changeux 1985, 133

... idealist, Cartesian epistemology of the psychologist. Edelman 1989, 13


7. seeming and sentience

There is more that needs to be said about seeming: we are able to perceive and we are able to seem to perceive. With both perceiving and seeming to perceive, it is as if there are two different sorts of things to be explained. One is how we come to perceive or imagine one thing rather than another, and the second is something about the bare fact of sentience -- how one physical entity can see or feel, or seem to see or feel, another, even in the simplest way. The sorts of brain theory we can see coming still have more to say about the first question than about the second. It looks as if we have to take basic sentience as a given, for now, and hope we know more about how to talk about it when we know more about how we do the things we do sentiently.

What we know already is that in a brain-based theory of cognition both percieving and imagining are neural events and as such they are spatiotemporal shapes. These shapes both are the EVENT of perceiving or imagining -- they are the sentient moment -- and they are connections to other spatiotemporal shapes in the brain.

If we don't lose sight of the fact that a sentient moment is a connected spatiotemporal configuration, we are less likely to be puzzled about whether sentience does any cognitive work. We can say it this way; what we think of as cognitive work is done by structures some of which are sentience structures.

I am using the word `feeling' not in the arbitrarily limited sense of `pleasure or displeasure' to which the psychologists have often restricted it, but on the contrary to its widest possible sense, i.e. to designate anything that may be felt. In this sense it includes both sensation and emotion -- the felt responses of our sense organs to the environment, of our proprioceptive mechanisms to internal changes, and of the organism as a whole to its situation as a whole, the so-called `emotive feelings.' Langer, 1988, 16


8. configuration

Although there is something basic about sentient perception that we have to take as given even while knowing we don't know how to talk about it, we are pretty sure we know why we have it ( --are it): we are able to perceive because abilities that are its precursors helped keep our ancestors alive. It sometimes has that use for us too.

Is it equally clear what imagining is for? It is likely that imagining and perceiving as we know them are so similar in their use of more basic cognitive capabilities that they have mostly the same cognitive precursors. Maybe we are able to imagine solely because we are able to perceive. What is more likely is that we can also perceive because we can imagine. I will describe some of the functional interrelations of perceiving and imagining in a later section.

I have said that in a brain-based theory of cognition perceiving and imagining are neural events and as such are spatiotemporal shapes connected to other spatiotemporal shapes. We do not have anything like a completed neuroanatomical map, but we do know that nervous tissue can do what it does because it is organized in specific ways. The nervous system is configurational: it is a configuration of nervous tissue distributed throughout other sorts of tissue in animal bodies. Both its localization in those bodies and its connectivity across regions of those bodies are important. By means of them it supports other more transient configurations, whether of energy, motion or chemical substances -- the structured electrochemical states that build themselves within and around neural connections.

We haven't settled how best to talk about the building up and melting down of these organic forms in and around the nervous system. We are in the midst of trying out metaphors. Pribram (1991) talks about the construction of holoscapes. Churchland (1989) talks about thermodynamic settling into positions or trajectories in state space. Varela (1984, 219) talks about shifting dynamical landscapes. All of these ways of speaking have the advantage that they encourage us to think of a cognitive event in physical terms.


9. computation as consequential configuration

A more ambiguously useful metaphor is the computer metaphor. It is difficult to use the computer metaphor clearly; our understanding of mechanical computation tends to import the mentalist equivocations we fall into when we use the notion of representation, since we have been defining computation as the processing of representations.

There is a better way to understand cognition as computation, a use of the computer metaphor that by-passes the usual identification of computers with the formal systems they are meant to automate, and instead emphasizes the physical properties of computing mechanisms that allow them to derive logical consequences among others.

The nervous system's changing acts of spatial structure may be thought of as computation if we think of computation itself as systematic pattern transformation made possible by the connective design of a machine whose physical materials have stable dynamical properties. This description is of course true of computation with digital machines, but it is easier to notice with analog or parallel processors. Mechanical computation is systematic consequential connection. In a nervous system systematic consequential connection has to do with coordinating an organism both internally and externally.

Coordination must be both spatial and temporal. Nervous system activity must be relevantly simultaneous -- correctly distributed into many cooperating regions -- and relevantly sequenced, again in such a way that simultaneous activities will be timed correctly in relation to each other and in relation to events in the world. Complex spatiotemporal coordination is what mechanics and organics have in common; it is what supports the usefulness of a computational metaphor in the first place.

If imagining is sensory simulation -- as I am about to argue -- and if we think the function of sensory simulation is to set up downstream (or upstream or sideways) connections, and if we understand computation as systematic consequential connection, then we can say imagining serves computational purposes -- is a form of computation. Connectionist models of computation, which are grounded in an understanding of computation as physical pattern transformation, are extremely useful in helping us think about styles of pattern transformation the nervous system may be using (principal-components versus competitive mappings between unit layers, for instance) and thus they carry us into the specifics of understanding how cognition is embodied. But our difficulties with the notion of representation tend to follow us even into connectionist pictures, and so for the purposes of this paper I will try to integrate connectionist contributions without calling them computation.

Unsupervised learning algorithms can be classified according to the type of representation they create. In principal-components methods, the hidden units cooperate, and the representation of each input pattern is distributed across all of them. In competitive methods the hidden units compete, and the representation of the input pattern is localized in the single hidden unit that is selected. Hinton 1992, 150


10. sensory simulation

The medieval distinction between intuitive and abstractive cognition was a distinction between perceiving and imagining. Intuition, or immediate apprehension, is situational contact: a transaction between a perceiver and a perceived. Abstractive cognition is anything else, including such activities as remembering and planning, which have situational relevance but are not an instance of physical contact between an imaginer and whatever she is imagining.

When we are thinking of any sort of cognitive structure as a spatiotemporal shape of coordinative connection, the difference between perceiving and imagining will be harder to specify. We could try saying perceptual structure is relevantly responsive to the physical presence of something in the world. Many descriptions of perceptual structure mention as part of the notion of structure relevantly responsive to the presence of the observed, some form of dynamic contribution from the object. Varela (1984, 220) speaks of a perceiver convolving its own and a received structure, Pribram (1991, 4) of a dynamic matching procedure between sensory input and the brain's microprocesses, Freedman (1991, 78) of the evolution of a dynamical system toward attractors picked out by patterns of input.

Describing perceptual structure as neural structure that is relevantly responsive to the physical presence of something in the world has the advantage that it invites an extension of the notion of perceptual structure into sensory-motor and other downstream (and upstream and sideways) connections that constitute the perceiver's readiness to feel, act, remember, think, move, say something, look at something else. Perceptual structure can thus be thought to extend seamlessly outward from a sensory core.

If we define perceptual structure as having something to do with relevant response to something physically present we could say a similar structure evoked WITHOUT triggering or structuring input from something physically present is a simulational structure -- is perceptual simulation.

Where cognition is imagined as manipulation of codes by digital computers the notion of simulation does not have easy purchase, but if we think of cognition as inherently configurational consequential connection, the notion of simulative configuration makes immediate sense. If we can think of a coyote as neurally configured to bark when it sees the moon, it makes sense also to think of the coyote barking when it is neurally configured to seem to see the moon -- when it is set up, at least partially, as if it were seeing the moon.

So if we think of any form of cognition as connective configuration, and if we think of imagining as sensory simulation, then we are a small step from guessing why imagining is elaborately developed in complex-enough creatures -- it is a way of setting up neural connections in perceptual structures so that they in turn can set up neural structures of other kinds, which may themselves set up yet other connections. Imagining structure, like perceptual structure, will extend seamlessly outward into connections that constitute the imaginer's readiness to feel, act, remember, think, move, say something, look at something -- or to seem to do any of these things.

Understood as sensory simulation, imagining can be seen to have lots of kinds of uses. Dennett suggests that human neural systems use seeming to hear a sentence as externally-patched access to instruction-following routines (" Alright, now I have to turn left at the corner, and then I have to look for the Mobil sign on the south side of the street"). Damasio guesses that our brains use seeming emotional/somatic response to mark as desirable or probably harmful possibilities of action we are considering ( I felt so awful when I thought of visiting my relatives that I didn't go -- even though that awful feeling was not really a bellyache, only a cortically simulated seeming bellyache).

A perceptual system that has become sensitized to certain invariants and can extract them from the stimulus flux can also operate without the constraints of the stimulus flux. Information becomes further detached from stimulation.. The adjustment loops for looking around, looking at, scanning, and focussing are then inoperative. The visual system vizualizes. But this is still an activity of the system, not an appearance in the theatre of consciousness. Gibson, 1966, 256

All that has to be the case for this practice to have this utility iis for the preexisting access relations within the brain of an individual to be less than optimal. Dennett 1991, 195-6


11. nonsensory simulation

I will note that the category of neural simulation is larger than that of sensory simulation: many kinds of neural connection don't involve sensory tissue at all, and yet they can be simulational. Grush (1995, 37) and Jeannerod talk about motor short-loop circuits which provide efferent motor networks the feedback they would probably have had if there had been time for such feedback to arrive. Such feedback need not reach sensory areas: it may be thought of as a motor 'image' (Jeannerod, 1994, 187) without being imagined muscular sensation. A similar extension of the idea of simulation into nonsensory simulation can make sense of Freud's mysterious posit of unconscious images: if sensory neural response can be set up in the absence of its usual or proper occasion, presumably nonsensory neural response can also be set up in the absence of an appropriate trigger, and thus we can behave in the presence of a husband as if we were in the presence of a father without being aware that we are doing so. 'Image' here can be read as meaning simulatory structure, and 'unconscious' can be taken to mean nonsensory.

12. wide nets: generic cognitive structure

In (5) I said the three main pushes in a physicalist redescription of imagining are watching how we talk, getting an understanding of what imagining is for, and working out some sort of spatiotemporal picture of what is happening in a brain when we imagine. I have not been able to talk about the first two of these efforts without describing cognition as widely connected consequential configuration, so part of the work of this section is already done. When we think of a brain imagining, a widely connected neural configuration is what we have to imagine. Given the limits of our knowledge we cannot believe that we are imagining it correctly, but even a very approximate version will have the advantage that it imagines an embodied event AS embodied-- as spatiotemporal, causal, historical and embedded in the world.

Neural theorists whose work I am citing offer similar visions of the wide network as generic cognitive structure. Changeux (1989, 24-5) speaks of neuronal graphs he sees as spider's webs set up both vertically, from spine through midbrain to cortex, and horizontally, across cortical regions. Pribram (1991, xxviii) talks about temporarily stable states which are mappings of isopotential dendritic polarizations. Edelman (1989, 195) defines a category, by which he means a generic perceptual/conceptual structure, as a thalamocortical reentry system involving primary and secondary sensory areas, association cortex, and cortical appendages. Damasio (1995, 102-5) talks about strengthened systems which are global mappings, strongly related by circuit design, linking meshes of topographically organized nets in early sensory cortices with non-topographically organized but systematically interrelated convergence zones in higher-order association cortices, basal ganglia and limbic structures.

The mental object is identified as the physical state created by correlated, transient activity, both electrical and chemical, in a large population or `assembly' of neurons in several specific cortical areas. This assembly, which can be described mathematically by a neuronal graph, is discrete, closed, and autonomous, but not homogeneous. It is made up of neurons possessing different singularities laid down in the course of embryonic and postnatal development. The earmark of the mental object is thus initially determined by the mosaic (or graph) of neuronal singularites and by a state of activity in terms of the number and frequency of impulses flowing in the circuits they form. Changeux, 1985, 137-8

Human emotion is phylogenetically a high development from simpler processes, and reason is another one; human mentality is an unsurveyably complex dynamism of their interactions with each other, and with several further specialized forms of cerebral activity, implicating the whole organic substructure
. Langer, 1988, 9


13. sensory ratio

Changeux talks about the members of our old cognitive classification -- percepts, images and concepts -- as cognitive structures different not in kind but in degree of sensory involvement: a concept would be a neuronal web or graph whose sensory ratio is low, whose neural population was dispersed, multimodal or amodal. If we visualize the event of imagining as a wide net with areas of activation in many regions of the brain, we can see immediately that different kinds or moments of imagining could also have different degrees of sensory involvement. There could be fleeting hypnagogic activations that are purely and simply visual and implicate very limited areas of early visual cortex -- circuits in layer VI perhaps (Zeki 1992). Or there could be multimodal hallucinations implicating both early sensory and associational areas in occipital, temporal and parietal cortex. Or there could be a wide net with complex but nonsentient activation in frontal association cortex and a limited involvement of sensory-motor areas (someone plotting a chess move?).

Visualizing the imagining or perceiving or thinking moment as a wide net has another kind of payoff -- it is instantly obvious that we could be thinking, perceiving and imagining at the same time. This lets us catch up with Ryle, Wittgenstein and the Romantic poets, who long ago noticed that fact. An episode of seeing-as, or seeing an aspect (Wittgenstein 1953, 213, 210), or seeing my doll smile when it isn't smiling (Ryle 1949, 74), or having ideas of full sail modifying the impression of the naked masts (Coleridge 1957, 2340) can be handled by envisaging degrees or strands of simultaneous activation in various cortical regions. The sort of interpenetration which interests Mary Warnock -- imaginative presences of past perceptions enriching the present moment's experience (1976, 150 ) -- can similarly be thought of as multiplex simultaneity of activity in a wide net.

Faculty divisions rightfully founder in the face of the coordinative unities of cerebral connectivity, and in the face of the unity of the felt subset of these connections which is the experienced moment. Sensing, imagining and thinking are alike in being aspects of the function of wide nets of neural activity: they make varying use of the same or similar substructures. They are subserved by the same bases of cognitive capability.

[A concept] contains only a small sensory component or even none at all, because it is the result of neuronal activity in association areas such as the frontal lobe (where multiple sensory or motor modalities are mixed) or in a large number of areas in different regions of the brain. Changeux, 1985, 138


14. semantic topography

Where the abilities once classified as psychological faculties are seen as very loosely categorized kinds of global uses of what is in fact one organ, we are more likely to think of cognitive capability of any kind as grounded in characteristics the nervous system has as a whole, characteristics that are the functional preconditions for imagining as they are for thinking and perceiving. The most basic of these characteristics is what we could call the brain's intrinsic semantics. The wide net of activity that is the event of an imagining moment is set up across what Changeux calls a mosaic of neuronal singularities. The brain just is a little region of inherently semantic material whose topography, evolutionarily given, developmentally tuned, and individually fitted by a life-time's net-training, supplies us with a ground of semantic possibility. Activation of a neuronal graph in this landscape is an event of content discrimination (in Dennett's nicely neutral term) which is
differential -- a pattern in physical space is a position in cognitive state space.

We know that with one pattern of activity in the olfactory bulb we will smell pine and with another we will smell raspberries (see Freedman 1991). When we have isolated the microcomponents of smell identification and shown how pattern analysis by layers of units can pull them apart every time, we still don't know why THIS pattern is smelling pine and this one is smelling raspberries. We do know, generally, that it has to have something to do with the way sensory receptors map onto adjacent spaces which map onto each other. Studies of cerebral morphogenesis are giving us a clearer picture of how such spaces flower outward from initial cell clusters and retain traces of the structure of their origination. Certain kinds and levels of activation in primary and secondary sensory cortices are sensory discriminations the way certain kinds and levels of activation in motor cortices trigger muscle contractions, because of the ways they are connected to the rest of the body and by means of that body to the rest of the world.

If we imagine a brain embedded in the world in this way -- if we visualize its connectivity extending beyond an animal's skin into the world -- we are probably in a better position to begin to think about how the brain can be an intrinsically semantical piece of physical space. One of the risks of calling differential activation of some region of the brain a 'symbolic representation ' (as Crick and others do ) is that we lose sight of the fact that what is being called a symbol is in fact anchored in an organized physical space which is semantic because of the ways it in turn is embedded in the physical space of the world -- when we call it a symbol we tend to think of it as a little object whose physical position has no importance, and as soon as we do that we have to be mystified by how it can possibly refer or designate.

All behavior, all sensation, is explicable by an internal mobilization of a topologically defined set of nerve cells, a specific graph. The `geography' of this network determines the specificity of the function. Changeux, 1985, 124


15. structural retention

Where a particular spatiotemporal shape is a particular relatedness to the world, maintaining a relation will require maintaining a shape; maintaining a potential for relation will require fixing a potential shape; modifying a relation will require modifying aspects or parts of a shape; and restoring a relation will need a mechanism for reactivating a shape.
An imagining moment will be a shape that is temporarily stable and that is a particular coalition of shape-parts that were latent and have been triggered on this occasion. A brain-based picture of cognitive order has to start at this level: we have to talk about neural stabilization and recruitment, structural modification and retention, and reactivation triggering.

A pattern of neural activity has to settle -- it has to come to a decision within its space of semantic possibilities -- and then it has to hold that shape long enough to affect other shapes relevantly. Crick (1992, 157) mentions 60-70ms. as a minimum processing period for extended cell assemblies. What he calls iconic memory -- which we might experience as an afterimage -- could be effected by stabilities lasting less than a second, but what is often called working memory -- which we might experience as keeping something in mind -- would be a shape held for several seconds, or longer if it is refreshed by the use of tricks like repeating a telephone number aloud. When we are looking at the world there are stabilities given by constancy of input from objects that aren't changing much -- we could say we are neurally entrained by the object. When we are imagining we are not so entrained and we are able to hold shapes only briefly. Then neural circuits will be entirely self-stabilized, and here the imagining moment has its main value as a window onto the dynamics of neural self-organization.

If imagining is perceptual simulation -- perceptual self-structuring in the absence of perceptual input -- the nervous system will need to be able to fall into shapes it has learned in perceptual encounters: it will have to have inscribed itself with lasting structural changes. In a net, structural modification will be modification at the nodes: synaptic modification. And synaptic learning will have to be selective -- it will have to fix synaptic change so that overall shapes of activity are inscribed. Hebb long ago (1949) suggested this learning rule: synaptic connections between cell assemblies that are active together will be modified to retain those connections.


16. structural assimilation

Along with an explanation for how cognitive shapes are fixed we have to have an explanation for how they may be relevantly modified. Cognitive abilities develop; we extend abilities we already have into new circumstances. Piaget, who was impressed with the orderliness of change in cognitive ability, saw that he could posit something like Hebb's rule as a mechanism for cognitive development as well as for structural retention. He envisaged increasingly inclusive neuron-enlisting processes that he called schemas of assimilation (in Piattelli-Palmarini 1980, 92); if we have a circuit established for putting our finger in our mouth, it can assimilate nearby circuits simultaneously activated by the presence in our baby hand of other objects. Hebbian learning will then cause them to be coactivated at other times. And so on, with increasing orders of generality, so that next the putting-in circuit will enlist another destination as well as another object. The baby puts a toasted cheese sandwich into the VCR. Here we'd have to tell a story about inhibitory learning too.

To explain imagining we will also have to say something about how perception-like structure can be set up in the absence of perceived objects: we will need a mechanism for reactivation-triggering. Again, Hebb's rule can be invoked. If perceiving has had wide-spread connections into language and association areas, then those areas will have learned connections with perceptual structures. Activity in those areas will be able to seed some of the shapes we are when we perceive.

It will have to be able to do so partially and selectively, in what can be called a content-addressable way. It will have to be able to reconstruct wholes given parts, but it will also have to be able to evoke parts given other parts. If imagining recruited too much perceptual structure we would be hallucinating.

Memory means simply that if in this model system you put a pattern of firing onto a number of the neurons, then either a different pattern comes out or, in special cases, the same pattern comes out because the system regenerates it. In this ... case if you put in a small amount of that pattern it will run around and produce the whole pattern. Crick 1988, 533

In such a reciprocally acting set of systems, input triggers an operation that at any moment is largely self-determining ... Further, the larger the amount of experience stored in the systems operating in a top-down fashion, the greater the self-determination. Thus Beethoven ... Pribram, 1991, 4


17. imagining, logic and creation

The wide net embodying an imagining moment is a semantic structure inasmuch as it touches into cerebral areas connected to sensors and effectors, but it is semantic also in the discriminations it accomplishes across areas and across layers within an area. Layer-to-layer and cross-regional transformations can be thought of as accomplishing logical functions.

Physically speaking, mappings between neuronal groups are filters, amplifiers, transducers, sets of gates; logically speaking, since they accomplish a sequence of decisions within an intrinsically semantic physical material, they are performing something like induction, deduction, feature abstraction, nested generalization, higher-order description. A moment of perceptual discrimination may thus be thought of as setting up instantly and automatically a whole range of classificatory possibilities. The logical relations embodied in its extensive connections may include descriptions in terms of properties, of parts, of class inclusion, of opposition within a class. A perceptual discrimination could even be thought to reach into very high-order mappings that amount to subsumptions under physical laws that have never yet been articulated -- a physical fine-tuning we'd call scientific intuition.

I am invoking a picture of physically embodied high intuition because the fact of imagining experience that most impresses novelists and other visionaries is its creative coherence -- the depth of semantic order possible in cognitive structures which are experienced as entirely self-produced. If there are layers of perceptual structure which are discriminations of properties of so high an order that they amount to descriptions at the level of physical (or social, or cultural) law, then activation organized from this level might be experienced as creation from first principles -- creation with great semantic depth. This gives us a cognitive account of the Romantic distinction between Fancy, which was superficially associated sensory memory, and Imagination, which was principled creative synthesis (Coleridge, Biographia Literaria II, ch. XIII).

A specific image or belief is just an arbitrary projection or slice of a deeper set of data structures, and the collective coherence of such sample slices is a simple consequence of the manner in which the global information is stored at the deeper level. P.M.Churchland 1989, 109

The so-called concepts of extension, of far and near, gravity, rigidity, horizontal, and so on, are nothing but partial abstractions from a rich but unitary perception ... The parts of it he can name are called concepts, but they are not all of what he can see. Gibson 1966, 261


18. backstage connections

Although they speak of the wide net of a perceiving or imagining moment as ramifying almost anywhere in the brain, Damasio and others tend to make a critical distinction between structures in sensory areas and structures they think of as doing backstage work. Damasio calls these backstage connections dispositional representations (1994, 241-2) because they are dispositions to respond to perceptual structures or to trigger them as simulations: prefrontal cortices in particular will be suitable locations for dispositional representations, he thinks, because they are well connected to and from both sensory and nonsensory areas, as well as to and from bioregulatory old-brain structures and nuclei controlling neurotransmitter distribution -- structures Damasio thinks of as supplying motivation and feeding outward into motor and chemical response. For Damasio, imagining would thus be vitally motivated centrally-organized activation in sensory maps. Both Damasio and Edelman (1989, 195) call these activations images and seem to speak as if they exist primarily as an interface with the imaginer, while dispositional representations out of sight do the work of cognition. This curious mentalist remnant is most obvious in Crick who says, for example, 'Some of these neurons would be involved in doing computations -- trying to arrive at the best coalitions -- while others would express the results of these computations, in other words, what we see!' (1992,155).

It is one thing to say brain structures can have cognitive effect even when they are nonsentient, or to say the sentient aspects of perceptual or imagining wide nets are triggered or organized out of these nonsentient configurations. But it is something else completely to speak of the sentient parts of perceptual or simulational structures as being DISPLAY structure. The persistence of this notion in otherwise hip theorists is presumably one more instance of the representation equivocation that slips between what we see or seem to see and the means by which we see or seem to see it.

The way to say it is this: we perceive and imagine by means of neural structures parts of which are sentience structures. There is functional continuity between felt and unfelt processes. Sentience structure and cognitively effective structure will overlap but not coincide: sentience structure will be a subset of cognitively effective structure. Sentience. structure will not be some 'mid-level description' 'projected into consciousness' but will instead be working structure as actively involved as nonsentience structures in basic cognitive tasks: the structures by means of which we are seeing and imagining will ALSO be sorting and passing activation.

Signals arising in those images are relayed to several subcortical nuclei and multiple cortical regions which contain dispositions for response. Damasio 1994, 241

..accounts for the importance of preconscious processes in the construction of such elaborate phenomena as ideas, intentions, images and fantasies, and makes it not only reasonable but obvious that they are rooted in the fabric of totally unfelt activities. Langer 1988, 8

It seems obvious that the purpose of vivid visual awareness is to feed into the cortical areas concerned with the implications of what we see. Crick 1992, 157


19. subsystem relations

If we imagine the structures by means of which we imagine as wide nets. we can begin to ask more interesting questions about exactly how imaging is done. Kosslyn's Image and Brain (1995) is an attempt to work out some of the specifics of what a brain would have to do to set up and use perceptual simulation. Although Kosslyn writes in the idiom of computer functionalism, he has several suggestions about what he calls implementation. His guess is that 'imagery' involves 'a relatively small number of processing subsystems that are drawn upon in different combinations for different tasks' (396). The 'central core' of a subsystem, by which Kosslyn means an input-output mapping that performs something we can recognize as a distinct subfunction, as, for instance, color discrimination is an identifiable subfuction in a visual system, tends to be implemented in local tissue, Kosslyn guesses (31), but it may also have diffuse outlying connections. Since subprocesses feed backwards and forwards onto one another, and since participating subsystems are active concurrently and continuously, cross-regional settling can be thought of as cooperative in the same way neural settling in local networks is cooperative. What we think of as distinct subfunctions could thus overlap spatially -- they would share tissue. A single subsystem can also be multifunctional, passing activation simultaneously into a number of other subsystems. There could also be alternative routes through processing space -- the same global input-output mapping could recruit different subsystems at different times (51).

Given this picture of imagining as a fluid and multimodal confederation of simultaneously active multifunctional parts, the sorts of questions we ask about the relation of perception and imagining shift level so that we are less interested in the traditional contrast between two terms than we are in the ways they are functionally interdependent.


20. perceiving as imagining

The brain-based theories I have been describing offer variations on the notion that perceiving and imagining require an integrated theory because they are similar functions performed in what are largely the same structures. As a consequence, most brain-based theories of mind concentrate on trying to understand perception, on the assumption that an understanding of perceptual simulation will fall out of a theory of perception. Kosslyn, who has committed many years to studies of visual imaging, is now saying (1995) that a theory of visual object recognition is the necessary foundation for a theory of visual mental imaging; but he is also saying that perception requires the use of simulational structure, and that in fact our ability to imagine may derive from abilities already employed in ordinary object perception.

Kosslyn thinks our ability to seem to perceive originates in neural anticipation: a well learned perceptual sequence, which makes it possible for us to see moving objects quickly enough, runs ahead of its input. Dennett similarly invokes simulation as part of the rudimentary work of perception, as when a hunter must keep track of an animal that disappears behind a bush (1991, 191). Freyd offers experimental evidence of what she calls representational momentum -- a rapid mandatory mechanism (1988, 432) that leads people to remember interrupted actions as having progressed further than they had in fact progressed when they were interrupted. (This sort of perceptual extrapolation is not far from the sort of ordinary pattern completion capability hidden unit processors can supply to noisy input.) Perceptual tracking, priming, anticipation and perseveration can all be seen as precursors of our ability to sustain seeming to perceive independently of sustained sensor input.

In perception, if one anticipates manipulating an object, the appropriate representation in the pattern activation and spatiotopic mapping subsystems are primed. In imagery, the priming processes are so strong that a mental image is generated, and this image moves through the sequence one anticipates seeing. Kosslyn, 1994, 351


21. does imagining use perceptual tissue?

Damasio's hypothesis, like Kosslyn's, is that our experiences of imagining are the result of structures activated in topographically organized areas of early sensory cortices (98, 101). Edelman similarly guesses we will find that they result from 'reentrant engagement of perceptual categorial systems even down to primary receiving areas' and including secondary sensory areas (1988, 195). Changeux is tempted, he says, to suggest that perception and imagining 'share the same neural material base,' and constitute 'different forms or states of the same basic material infrastructure' (1989, 132-133). -- are both 'neuronal graphs recruited principally from the maps or homunculi of primary and secondary cortices' (138). Farah says 'mental images' 'rely on the same neural substrate as perceptual images generated during normal perception' (1989, 188).

The easy assumption that imagining and perception share the same mechanisms, or more specifically, that imagining operates back onto primary and/or secondary sensory cortices, is supported by evidence of a number of kinds. There is a variety of psychological testing evidence, some of it quite old. Changeux cites Perkey's 1910 experiment showing that test subjects reported they were imagining when in fact a subthreshold image was being back-projected on a screen they had been instructed to fixate. Other results (Segal and Fusella 1970) showed competition effects when subjects were asked to perform visual perception and visual imaging tasks at the same time; these competition effects were not present if perceiving and imagining tasks were performed in different sensory modalities. Kosslyn (1994, 328-9) cites evidence that there are parallels in the ease or difficulty subjects experience in tasks they perform both while perceiving and while imagining. Similar sorts of after effects and illusions are experienced in both perceiving and imagining conditions.

Martha Farah's 1988 survey of neuropsychological findings concludes that two lines of recent evidence reliably localize brain activity during visual imaging tasks to visual cortical areas. Regional cerebral bloodflow analysis and brain electrical mapping show activity in the same regions of vision cortex when subjects perform similar vision and imaging tasks. Tasks requiring imaging in another sensory mode do not activate these areas. Second, psychological testing data imply shared functional mechanisms for visual imaging and visual perception. Evidence described shows parallel sorts of deficit in perceptual and imaging tasks following cerebral damage. Someone with cortical blindness cannot visualize, while someone whose blindness is peripheral can. Selective imagery deficit IN ALL CASES, Farah says, parallels visual deficit. Acquired color blindness applies also to imaging; inability to localize objects, or to recognize them, will be found equally in perception and imaging tests; visual neglect of a single hemifield will extend to similar imaging neglect; a difficulty with face recognition will extend to face recall.

Changeux mentions studies indicating that direct stimulation of neurons in the primary visual cortex produces simple visual hallucinations while direct stimulation of neurons in the secondary visual cortex sets off more complex hallucinations of changing scenes ( H. Haecan and M. Albert, 1978, Human Neuropsychology).

Kosslyn points out that Bain in 1885 and James in 1891 were already suggesting imagining is simulative use of perceptual neural structure. Without the more detailed knowledge we are beginning to have of the actual organization of connectivity in the brain, it was not possible, however, to say much more than that when we perceive, structures in sensory areas are activated from the sensory periphery, and when we imagine they are 'centrally activated'. 'Central activation' is still to some extent a wave in the direction of terra incognita, but brain-based theorists of imagining are beginning to think about what else might have to be active in an imagining episode's wide net, both to set up a sensory simulation and to use it. Kosslyn suggests, for instance, that, in order to organize ourselves to imagine we are seeing an object rotate, we may have to simulate seeing an object as we move around it. That is, sensory simulation may at times have to be organized by means of a motor simulation which is not experienced as such.


22. imagining and awareness

We aren't always conscious although our brains are always active. Even when we are awake not all neural activity is sentient: people respond to stimuli they do not notice. We know only a little about what makes the difference. It has something to do with rate of activity, with duration and with location.

Libet reports (1991, 1753) that a minimum duration of appropriate neural activity of up to about .05s is needed for people to sense a stimulus whose intensity is low, although durations below that minimum can mediate response. And there is a trade-off between duration and stimulus intensity, within certain limits. Where stimulus intensity is greater, cerebral on-time can be less. Crick suggests awareness is tied to rate of neural activity: neurons will respond to the same things whether we are awake or asleep, but fire more strongly when we are awake and attending -- and less strongly when we are awake but responding to something else. Thus, he proposes, timed bursts of activation from the thalamus that boost firing rates throughout the whole of a wide net of activity could coordinate awareness.

I have been assuming throughout this paper that all consciousness is sensory: that only those structures are sentient which are the structures by which we perceive or seem to perceive, and that forms of consciousness we think of as nonsensory (thinking, reasoning, etc.) are conscious only to the extent that they include sensory structure. The kinds of sensing and seeming to sense involved may include proprioceptive awareness of tissue states and physical tensions, as Damasio and Langer suggest. In humans conscious states can rely heavily on seeming to hear ourselves speak, as Dennett suggests. If the thesis that consciousness IS sensory engagement turns out to be true, a theory of sensory simulation will be a key to the general theory of consciousness.

One of the questions current in philosophy of mind (for instance Jackendoff 1991) concerns the 'level of description' that is 'projected into consciousness' when we perceive or imagine. The question has a mentalist sound, but if we interpret it charitably we can take it as a question about whether all or some or only one or two of the mutually mapped layers in sensory cortices are bearers of the activity by means of which we perceive or seem to perceive. This is a fair question. Dennett has crusaded for a picture of sentience as distributed in the brain, 'events of content discrimination' being accomplished wherever they happen to happen -- color discriminations, motion discriminations, form discriminations, identity discrimination, name discriminations -- the lot. The wide net of a perceiving or imagining moment would be a standing mosaic of mini-micro-discriminations, some of which are taking place (at adequate rates and durations of activation) in sensory areas -- and those microdiscriminations would be the standing texture of conscious experience.

Feature detections or discriminations only have to be made once ... the information content fixed does not have to be sent somewhere else to be rediscriminated. Dennett 1991, 113

All varieties of perception -- indeed all varieties of thought or mental activity -- are accomplished in the brain by parallel, multitrack processes of interpretation and elaboration of sensory inputs. Dennett, 1991, 111

23. transparency of the imagining moment

If we think of cognition as shapes of activation in a brain, and if we think of ourselves as BEING those shapes of activation, we can begin to think fruitfully and interestingly about what our present moment's experience might imply about its own neural means.

People who speak in terms of neural representations must -- because of the temptations implicit in their metaphor -- continuously remind themselves that introspection of representational structures is impossible. But this is just to say that seeing is not seeing the means by which we see, and neither is seeming to see. Of course we cannot introspect the means by which we perceive or imagine. We cannot introspect at all: we can only perceive or imagine.

But when we imagine cognition as wide nets of activation, we can begin to take any perceiving or imagining moment as an immediate expression of its means. The more we learn about the brain's organization, the more transparent the cognitive moment will be. It is already temporally transparent: we know that the timing of cognitive events is precisely coordinate with neural events, since they are the same thing. And there are other quite general things we can know, for instance that when we are awake some exact neural condition prevails and when it ceases to prevail we are asleep. As we know more about neurotransmitters we will be able to say something about our neurochemical state, just as we (some of us) are already able to notice our own estrogen levels. Perceptual illusions such as subjective contours can always be taken as implying pattern-completion activity in form-discrimination areas of visual cortex. ANYTHING we are aware of would imply a certain level of neural activity in sensory cortex somewhere. When we notice the sorts of things a meditator might notice, that the sensation of thinking involves subtle forms of motor fantasy, perhaps, we could assume structures active in sensory-motor areas as we think; and that could tell us something about how our conclusions are being processed in nonsentient areas. A particularly vivid dream could imply high levels of well-organized structure. We would be able to notice sensory blends or ratios as such. We would have a theoretically supported expanded phenomenology for kinds and tones of sensation we ignore because we do not name them. Perhaps we will discover there are ways we ARE able to perceive the means by which we perceive: perhaps there is proprioception at the level of neural configuration. Perhaps we are already feeling shapes in our brains.

These sorts of possibility seem to give more joy to artists than to philosophers, but it is a kind of joy philosophers could learn. Strawson in his 1971 paper on the notions of perceiving and imagining in Kant and Wittgenstein describes an 'actual occurrent perception' infused with the thought of other past or possible perceptions of the same object'-- a moment that were it seen as a wide net would be understood as implying the coactive presence of simulational and perceptual structure.

... non-actual perceptions are in a sense represented in, alive in, the present perception. The actual occurrent perception ... is, as it were, soaked with, or animated by, or infused with -- the metaphors are a choix -- the thought of other past or possible perceptions of the. same object. Strawson 1971, 88-89

Experience clearly gives no clue as to the means by which it is organized. Lashley 1956, 4

[A midlevel description] may, as it were, give the phenomenological mind the cue to produce experiences of blue at the right times and experiences of red at the right times. But that is not the same as producing the experiences themselves. Jackendoff, 1989, 18

... more like the flow of a river in which patterns emerge and disappear, than a static landscape ... contents aren't contained anywhere but are revealed only by the dynamics: form and content are thus inextricably connected and can't ever be separated. Kelso 1995, 5

[There is no way to isolate the properties] presented in consciousness from the brain's multiple reactions to its discriminations, because there is no such additional presentation process. Dennett 1991, 393

Rising from a background of general bodily feeling and a texture of emotive tensions so closely woven that the separate strands of process in it are not distinct, but compose a `mental state' ... more specifically articulated acts [are] felt as envisagements, intentions, cogitations, insights, decisions ... Langer 1988,12

24. imagining and metaphor

We know we use metaphors; we know, further, that everything we are able to do we are doing by neural means. In my reading a metaphor, like any simulational structure, is a temporary configuration, a little net within the larger net of the moment's cognitive activity. It distributes activity in many directions; it is a bridge, a switching structure. It will use sentient tissue, when it does, because that structure already has working, not-wrong, connections with action, language and emotion.

It would be useful to have a more exact sense of how we use a metaphor. We are thinking about one thing in terms borrowed from our knowledge of another thing. Can we translate this into neural terms by saying that when we evoke neural configurations relevant to two different objects we are superimposing two physical structures and thereby setting up a processor that convolves their connections in some way? Evoking two nets simultaneously could create what amounts to a filter, as certain connections are strengthened and others cancel out. The convolved structure that results would distribute activity in ways relevant to both contexts, perhaps in unusual ways. The moment of metaphor-experience would be the result of the parts of that mixed structure that touch into sensory tissue; we'd experience ourselves, as we do, as imagining-cum-thinking.

If in fact this is how we use metaphor, helping ourselves to a clearer picture of a brain in the act of picture-thinking could help us figure out how philosophers go wrong. Consider the metaphor used by this philosopher:

There is an imagination; it is a faculty or power;
specifically it is a faculty for internal representation; these
representations are image-like; therefore they share a
certain character with external images; in particular, like
material images they represent absent objects as present;
they do so by means of resemblance.
(Brann 1991, 5)

What sort of cognitive landscape could be responsible for this description? --Not that it is wrong. Everything said can be construed in a way that makes it make sense. 'There is an imagination': we are able to imagine. 'It is a faculty or power': ditto. 'Specifically it is a faculty for inner representations': one, we can seem to perceive when we are not in fact perceiving, and two, we are able to do so by means of neural structures in our heads. 'These representations are image-like': the experience of imagining is (in some ways) like the experience of look at a picture. 'They share a certain character with external images ... they represent absent objects as present': when we imagine we merely seem to see. 'They do so by means of resemblance': to the extent that the structure we are when we imagine is like the structure we are when we perceive, we find the two experiences similar and respond similarly within them.

Brann's classically mentalist description of imagining is not so much mistaken as awkward, uninformative, and misleading. When we talk about inner images that resemble objects we cannot help importing the structure of our dealings with external images; thus we automatically think of ourselves as scanning them, storing them, comparing them with the original they depict. We think of them as located in their own space, which must be inner since it is not outer. And we think of ourselves, when we 'see' them, as separate from them, looking at them across a little distance. None of this do we intend to do: it simply comes with using a picture metaphor for imagining. We are Cartesian materialists not by intention but by metaphor.

If, along with thinking of every cognitive moment as a physical configuration that we ARE and not as a 'representation' we somehow 'are aware of', we keep in mind the fact that every theorist is such a configuration in the moment of writing theory, we may be able to spot the simulational structure that is misdistributing parts of what is not-wrong basic intuition. We could apply brain theory to creating better brain theory, and we could be clearer about the ways transitional theory IS transitional and needs to be teased apart into its useful and nonuseful threads.