The last post on feelings generated some excellent conversations. In a couple of them, it was pointed out that my description of feelings put a lot of work on the concept of imagination, and that maybe I should expand on that topic a bit.
In their excellent book on animal consciousness, The Ancient Origins of Consciousness, Todd Feinberg and Jon Mallatt identify important behavioral markers of what they call “affect consciousness”, which is consciousness of one’s primal reactions to sensory stimuli (aka feelings). The top two markers were non-reflexive operant learning based on valenced results, and showing indications of value based cost/benefit trade-off decision making.
In thinking about what kind of information processing is necessary to meet these markers, I realized that a crucial component is the ability to engage in action-sensory scenario simulations, to essentially predict probable results based on decisions that might be taken.
In their book, Feinberg and Mallatt had pointed on that the rise of high resolution eyes was pointless without the concurrent rise of mental image maps. Having one without the other would have consumed valuable energy with no benefit, something unlikely to be naturally selected for. Along these same lines, it seemed clear to me that the rise of conscious affects was pointless without this action-scenario simulation capability. Affects, essentially feelings, are pointless from an evolutionary perspective unless they provide some value, and the value they provide seems to be as a crucial input into these action-scenario simulations.
I did a post or two on these simulations before realizing that I was talking about imagination, the ability to form images and scenarios in the mind that are not currently present. We usually use the word “imagination” in an expansive manner, such as trying to imagine what the world might be like in 100 years. But this type of imagination seems like the same capability I use to decide what my next word might be while typing, or what the best route to the movie theater might be, or in the case of a fish, what might happen if it attempts to steal a piece of food from a predator.
Of course, fish imagination is far more limited than mammalian imagination. The aquatic environment often only selects for being able to predict things a few seconds into the future, whereas for a land animal, being able to foresee minutes into the future is a very useful advantage. And for a primate swinging between the trees and navigating the dynamics of a social group, being able to see substantially farther into the future is an even stronger advantage.
But imagination, in this context, is more than just trying to predict the future. Memory can be divided into a number of broad types. One of the most ancient is semantic memory, that is memory of individual facts. But we are often vitally concerned with a far more sophisticated type, narrative memory, memory of a sequence of events.
However, extensive psychological research demonstrates that narrative memory is not a recording that we recall. It’s a re-creation using individual semantic facts, a reconstruction, a simulation, of what might have happened. It’s why human memory is so unreliable, particularly for events long since past.
But if narrative memory is a simulation, then it’s basically the same capability as the one used for simulation of possible futures. In other words, when we remember past events, we’re essentially imagining those past events. Imagination is our ability to mentally time travel both into the past and future.
As I described in the feelings post, the need for the simulations seem to arise when our reflexive reactions aren’t consistent. (I didn’t discuss it in the post, but there’s also a middle ground for habitual reactions, which unlike the more hard coded lower level reflexes, are learned but generally automatic responses, such as what we do when driving to work while daydreaming.)
But each individual simulation itself needs to be judged. How are they judged? I think the result of each simulation is sent down to the reflexive regions, where they are reacted to. Generally these reactions aren’t as strong as a real time one, but they are forceful enough for us to make decisions.
So, as someone pointed out to me in a conversation on another blog, everything above the mid-brain region could be seen as an elaboration of what happens there. The reflexes are in charge. Imagination, which essentially is also reasoning, is an elaboration on instinctual reflexes. David Hume was right when he said:
Reason is, and ought only to be the slave of the passions, and can never pretend to any other office than to serve and obey them.
When Hume said this, he wasn’t arguing that we are hopelessly emotional creatures unable to succeed at reasoning. He was saying that the reason we engage in logical thinking is due to some emotional impetus. Without emotion, reason is simply an empty logic engine without purpose and no care as to what that logic reveals.
Okay, so all this abstract discussion is interesting, but I’m a guy who likes to get to the nuts and bolts. How and where in the brain does this take place? Unfortunately the “how” of it isn’t really understood, except perhaps in the broadest strokes. (AI researchers are attempting to add imagination to their systems, but it’s reportedly a long slog.) So, if this seems lacking in detail, it’s because, as far as I’ve been able to determine in my reading, those details aren’t known yet. (Or at least not widely known.)
What I refer to as “reflexes” tend to happen in the lower brain regions such as the mid-brain, lower brainstem, and surrounding circuitry. Many neurobiologists often refer to these as “survival circuits” to make a distinction between the reflexes on the spinal cord, which cannot be overridden, and the ones in the brain, which can, albeit sometimes only with great effort. (I use the word “reflex” to emphasize that these are not conscious impulses.)
Going higher up, we have a number of structures which link cortical regions to the lower level circuitry. These are sometimes referred to as the limbic system, including the notorious amygdala, which is often identified in the popular press as the originator of fear, but in reality is more of a linkage system linking memories to particular fears. Since there are multiple pathways, when the amygdala is damaged, the number of fears that a person feels is diminished, but not eliminated.
Of note at this level are the basal ganglia, a region involved in habitual movement decisions. This can be viewed as an in-between state between the (relatively) hard coded reflexes and considered actions. Habitual actions are learned, but within the moment are generally automatic, unless overridden.
And then we get to the neocortex. (Some of you may notice I’m skipping some other structures. Nothing judgmental, just trying to keep this relatively simple.) Broadly speaking, the back of the cerebrum processes sensory input, and the front handles movement and planning.
The sensory regions, such as the visual cortex, receive signals and form neural firing patterns. As the patterns rise through regional layers, the neurons become progressively more selective in what triggers them. The lowest are triggered by basic visual properties, but the higher ones are only triggered on certain shapes, movements, or other higher level qualities. Eventually we get to layers that are only triggered on faces, or even a particular face, or other very specific concepts. (Google “Jennifer Aniston neuron” for an interesting example.)
Importantly, when the signals trigger associative concepts, the firing of these concepts cause retro-activations back down through the layers. The predictive processing theory of perception holds that what we perceive is actually more about the retro-activation than the initial activation. Put another way, it’s more about what we expect to see than what is coming in, although what is coming in provides an ongoing error correction. (Hold on to the retro-activation concept for a bit. It’s important.)
Imagination is orchestrated in the prefrontal cortex at the very front of the brain. Note that “orchestrated” doesn’t mean that imagination wholly happens there, only that the prefrontal cortex coordinates it. Remember that sensory imagery is processed in the perceiving regions at the back of the brain.
So, if I ask you to imagine a black bear climbing a tree, the image that forms in your mind is coordinated by your prefrontal cortex. But it outsources the actual imagery to the back of the brain. The images of the black bear, the tree, and of the bear using its claws to scale the tree, are formed in your visual cortex regions, through the retro-activation mechanism.
Of course, a retro-activated image isn’t as vivid as actually seeing a bear climbing a tree since you don’t have the incoming sensory signal to error correct against. You’re forming that image based on your semantic memories, your existing associations about trees, bears, and climbing. (If you need help imagining this, and don’t know not to try to escape from a black bear by climbing a tree, check out this article on black bears.)
What the pfc (prefrontal cortex) does have are pointers, essentially the neural addresses of these images and concepts. This is the “working memory” area that is often discussed in the popular press. And the pfc is where the causal linkages between the images are processed. But the hard work of the images themselves remain where they’re processed when we perceive them.
So imagination is something that takes place throughout the neocortex and, well, throughout most of the brain since the simulations are judged by the reflexive survival circuits in the sub-cortical regions. Given how difficult this capability is to engineer, I don’t think we should be too surprised.
So that’s my understanding of imagination and how it ties in with perception and affective feelings. Have a question? Or think I’m hopelessly off base? As always, I’d love to find out what I might be missing.