The problems with the Chinese room argument

Posted on November 10, 2019 by

In 1950, Alan Turing published a seminal paper on machine intelligence (which is available online).  Turing ponders whether machines can think.  However, he pretty much immediately abandons this initial question as hopelessly metaphysical and replaces it with another question that can be approached scientifically: can a machine ever convince us that it’s thinking?

Turing posits a test, a variation of something called the Imitation Game.  The idea is that people interact with a system through a chat interface.  (Teletypes in Turing’s day; chat windows in modern systems.)  If people can’t tell whether they are talking with a machine or another person, then that machine passes the test.

Turing doesn’t stipulate a time limit for the test or any qualifications for the people participating in the conversation, although in a throwaway remark, he predicts that by the year 2000 there will exist a system that could fool 30% of participants after five minutes of conversation, a standard many have fixated on.  This is a pretty weak version of the test, yet no system has managed to pass it.

(There was a claim a few years ago that a chatbot had passed, but it turned out to depend on a clever description of the person that was supposedly on the other end, a foreign teen with a shaky grasp of English, which most people think invalidated the claim.)

We’re nowhere near being able to build a system that can pass a robust version of the test with at least an hour of conversation that fools at least 50% of a large sample of human participants.

I think Turing’s overall point is the philosophical problem of other minds.  We only ever have access to our own consciousness.  Although physical or systematic similarities may give us clues, we can ultimately only infer the existence of other minds by the behavior of the systems in question.  The Turing test is essentially a recognition of this fact.

The argument most commonly cited in opposition to the idea of the Turing test is a philosophical thought experiment put forth by John Searle in 1980: the Chinese room argument.  (The original paper is also available online).

Searle imagines himself sealed in a room with a slit for questions in Chinese to be submitted on paper.  Searle, who does not know Chinese, has a set of instructions for taking the symbols he receives and, using pencil and paper, producing answers in Chinese.  He can follow the instructions to the letter and produce the answers, which he slides back out the slit.

Searle’s point is that the Chinese room may pass the Turing test.  It appears to understand and can respond to Chinese questions.  But Searle himself doesn’t understand a word of Chinese, and, he argues,  neither does anything else in the room.  It appears to be a case where we have an entity that can pass the Turing test, but which doesn’t have any real understanding.

The takeaway from this argument is supposed to be that Searle is doing the same thing a computer processor does, receiving and manipulating symbols, but with no understanding of what is happening.  Therefore the Turing test is not valid, and computationalism overall is wrong.

There are a number of common criticisms of this argument, which Searle responds to in the paper, one of which I’ll get to in a bit.  But what I consider the most damaging criticism is rarely discussed, that the scenario described is, if not impossible in principle, utterly infeasible.

We’re asked to suppose that Searle will do everything a computational system that can pass the Turing test will do.  But no one really imagines him doing that.  Generally we end up imagining some procedure with maybe a few dozen, or perhaps even a few hundred steps.  It might take Searle a while to respond, but there’s nothing too out of bounds about it.

Except that we need to consider what a system that can pass a robust Turing test needs to be able to do.  A brain has billions of neurons that can spike dozens to hundreds of times per second, and communication throughout the brain tends to be recurrent and ongoing.  Which is to say, that a brain receiving a question, parsing it, considering it and responding to it, will engage in at least hundreds of billions of events, that is, hundreds of billions of instructions.  A machine passing the Turing test may not do it exactly this way, but we should expect similar sophistication.

And Searle is going to do this by hand?  Let’s suppose that he’s particularly productive and can manually perform one instruction per second.  If he takes no bathroom, meal, or sleep breaks, he should have his first billion instructions performed in around 30 years.  Responding to any kind of reasonably complex question would take centuries, if not millenia.

Maybe, since Searle is a fairly complex system in his own right, we can provide higher level instructions?  Doing so, we might be able to reduce the number of steps by a factor of 10 or maybe even 100.  But even with that move, the response will be years to decades in coming.  And making this move increases the amount of human cognition involved, which I think compromises the intuition of the thought experiment.

We can make the thought experiment more practical by the expedient of giving Searle…a computer.  Even a mobile phone today operates at tens of thousands of MIPS, that is, tens of billions of instructions per second.  But of course, then we’re right back to where we started, and the intuitive appeal of the thought experiment is gone.

Okay, you might be thinking, but by introducing all this practicality, am I not failing to take this philosophical thought experiment seriously?  I’d argue that I am taking it seriously, more seriously in fact than its proponents.  But, in the spirit of philosophical argument, I’ll bracket those practicalities for a moment.

The other response to the argument I think remains strong is the first one Searle addresses in the paper, the system response.  The idea is that while Searle may not understand Chinese, the overall system of the room, including him and the instructions, do.  If the room can respond intelligently in Chinese, including to unplanned questions about the house and village that it grew up in China, which sports teams it was a fan of, which schools it went to, restaurants it ate at, etc, then at some point we should consider that buried in that system is an entity that actually thinks it did grow up in China, or at least one that can conceptualize itself doing so.

Searle’s response (done with disdain, but then the whole paper has a polemical feel to it) is to simply posit that he memorizes all the instructions and performs them mentally.  With this modification of the scenario, Searle still doesn’t understand Chinese and, he argues, the system reply is invalidated.

Okay, I know I said I would bracket the practicalities, but if the initial scenario was infeasible, this one is simply ridiculous enough that it should be self refuting.  Searle’s going to memorize the hundreds of billions of instructions necessary to provide convincing answers?

But, bracketing that issue again, nothing has changed.  The system still understands Chinese even if the parts of Searle following the instructions doesn’t.  If Searle is somehow superhuman enough to memorize and mentally follow the code of the Chinese system, then he’s arguably superhuman enough to hold another thinking entity in his head.

And a counter argument here is to consider how I, as a native English speaker, understand English.  If someone were to sufficiently damage Wernicke’s area in my brain, it would destroy my ability to comprehend English (or any other language).  In other words, the rest of my brain doesn’t understand English any more than the non-instruction part of Searle understands Chinese.  It’s only with the whole system, with all the necessary functional components, that I can understand English.  What’s true for me is also true for the room, and for Searle’s memorized version of it.

Searle addresses a number of other responses, which I’m not going to get into, because this post is already too long, and I think the points above are sufficient to dismiss the argument.

If Searle had restricted himself to addressing the possibility of a simple system passing the weak version of the Turing test, pointing out that such a system would be missing the mental representations necessary for true understanding, and the utter inability of computer technology c. 1980 to hold those representations, he might have been on somewhat firmer ground.  But he pushes a much stronger thesis, that mental content in computation is impossible, even in principle.  How does he know this?  He feels it’s the obvious takeaway from the thought experiment.

Any meaning in the computer system, Searle argues, comes from human users and designers.  What he either doesn’t understand or can’t accept, is that the same thing is true for a brain.  There’s nothing meaningful, in and of itself, in the firing of individual neurons, or even in subsystems like the amygdala or visual cortex.  The signals in these systems only get their meaning from evolution and by the relation of that content to the environment, which for a brain includes its body.

A computer system gets its meaning from its designers and environment, including its human users, but the principle is the same, particularly if we set that computer as the control system in a robotic body.  Yes, human brains include representations about itself, but so do most computer systems.  All you have to do is pull up Task Manager on a Windows system to see representations in the system about itself.

So I don’t think the Chinese room makes its case.  It attempts to demonstrate the infeasibility of computationalism with a contrived example that is itself far more obviously infeasible, and responds to one of the strongest criticisms against it by ramping that infeasibility to absurd levels.  The best that might be said for it is it clarifies the intuitions of some anti-computationalists.  The worst is that by demonstrating the need to resort to such absurd counter-examples, it arguably strengthens what it attacks.

Unless of course I’m missing something?  Are there weaknesses in Turing’s argument or strengths in Searle’s that I’m missing?

Posted in Mind and AI | Tagged , , , , , | 80 Comments

The difficulty of isolating evidence for the location of consciousness

Posted on November 2, 2019 by

In the ongoing debate in neuroscience between those who see consciousness being in the back part of the brain, among the sensory processing regions, or in the front of the brain, in the cognitive action planning regions, there are issues confounding the evidence.  Most experiments testing for conscious perception depend on self report from the test subjects, but this causes a problem since the frontal lobes are necessary for any method of self report (speech production, pressing a button, etc), so when those frontal lobes light up in brain scans in correlation with conscious perceptions, the possibility exists that they only light up due to the self report requirement.

So, an experimental protocol was developed: the no-report paradigm.  One group of subjects are given a stimulus and asked to report if they consciously perceive it while their brains are being scanned.  Another group are given the same stimulus that led the first group to report conscious awareness, but the second group is not required to self report, also while being scanned.  The scans of the groups are compared to see if the frontal lobes still light up in the second group.  Generally, although there is variation, the frontal lobes do still light up, implicating frontal regions in conscious perception.

However, Ned Block, a philosopher who thinks it likely that phenomenal consciousness “overflows” the self report from access consciousness, sees an issue that he describes in a paper in the journal, Trends in Cognitive Sciences.  (Warning: paywall)  Block points out that potential confounds remain, because we can’t rule out that the test subject isn’t thinking about reporting their perception, or cognitively processing the perception in some manner, causing the frontal lobes to light up for reasons other than the conscious perception itself.

Block points out that the real fundamental distinction here is between those who see cognition (in the frontal lobes) as necessary for consciousness versus those who see perceptual processing (in the back regions) as sufficient.  Global workspace and higher order thought theories are cognitive accounts, while integrated information and local recurrent loop theories are more sensory oriented.

Block argues that the no-report paradigm needs to be replaced with a no-cognition paradigm, or to avoid begging the question against cognitive accounts, a “no-post-perceptual cognition” paradigm.  But how can cognition be eliminated from subjects who have perceptions?  Short of selectively anesthetizing the frontal lobes (which would be invasive, risky, and unlikely to get past IRBs), is this even possible?

Block focuses on a study of binocular rivalry, by Jan Brascamp and colleagues, as a possible solution.  Binocular rivalry is the phenomenon that, when the eyes are shown very different images, conscious visual perception alternates between the two images, rather than blending them together.  (Blending can happen, but only if the images are similar.)  The goal of Brascamp’s study is to determine whether the selection between the rival images happens in the back or front of the brain.

To do this, the study constructs rival images of random dots such that, although they are different enough to lead to binocular rivalry (the dots in one image move left vs moving right in the other image), they are similar enough that the subject’s attention isn’t called to the switching between the images and so can’t report it.

For subjects who aren’t required to report what they’re seeing, brain scans show variations correlated with the image switching in the back of the brain, but not in the front.  In other words, the study shows that the selection of which image to momentarily “win” in the binocular rivalry happens in the back of the brain.

Block sees the methodology here as an example of the “no-post-perceptual cognition” paradigm, and the specific results as indicating that the frontal lobes aren’t necessarily involved in conscious perception of the images.  He focuses on the fact that subjects could, if queried, identify whether the dots were moving left or right, indicating that they were conscious of the specific image at the moment.

I think there are problems with this interpretation.  By Block’s own description, the subjects didn’t notice and couldn’t self report the oscillations between the rival images, so we shouldn’t expect to see correlated changes in the frontal lobes for those changes.  The subjects may have become conscious of some details in the images when asked to report, but when they weren’t asked to report, it seems more likely they were only conscious of an overall “gist” of what was there, a gist that worked for both images, and so didn’t need to oscillate with them.

The Brascamp et al. study is hard core functional neuroscience, aimed at narrowing the location of a specific function in the brain.  They succeed at establishing that the selection happens in the back of the brain.  But I don’t think a “frontalist” (as Block labels them) should be concerned about this.  A pre-conscious selection happening in the back of the brain doesn’t really seem to challenge their view.

And Brascamp et al. actually seem to come to a different conclusion than Block.  From the final paragraph in their discussion section:

A parsimonious conceptualization of these results frames awareness of sensory input as intimately related to the planning of motor actions, regardless of whether those actions are, in fact, executed. In this view a perceptual change of which the observer is aware might be one that alters candidate motor plans or sensorimotor contingencies. This view also marries the present evidence against a driving role of fronto-parietal regions in perceptual switches to the notion that these regions do play a central role in visual awareness when viewing a conflicting or ambiguous stimulus, a switch in perception may arise within the visual system, but noticing the change may rely on brain regions dedicated to behavioral responses.

So, while the study succeeded in its aims, I can’t see that that the results mean what Block takes them to mean, or that the methodology accomplishes the no-post-perceptual cognition paradigm he’s looking for.  That doesn’t necessarily mean that sensory consciousness isn’t a back of the brain phenomenon.  It just means getting evidence for it is very tricky.

This front vs back debate is a major issue in the neuroscience of consciousness.  One I’m hoping that Templeton contest does succeed in shedding some light on.  Myself, I suspect the frontalists are right, but wouldn’t be surprised if it’s a mix, with maybe sensory consciousness in the back, but emotional and introspective consciousness in the front, with our overall experience being a conjunction of all of them.

What do you think?  Is consciousness a cognitive phenomenon?  Or is perceptual awareness independent of cognition?  Or in a system where the components evolved to work closely together, is this even a well posed question?

Posted in Zeitgeist | Tagged , , | 64 Comments

Integrated information theory

Posted on October 29, 2019 by

Cover of the book The Feeling of Life ItselfI think most of you know I’m not a fan of integrated information theory (IIT).  However, it is a theory proposed by scientists, and I’ve always had a mildly guilty conscience over not having read about it other than through articles and papers.  Some years ago, I tried to read Giuilio Tononi’s book, PHI: A Voyage from the Brain to the Soul, but was repelled by its parable format and low information density, and so never finished it.  So when Christof Koch’s new book, The Feeling of Life Itself, was announced, and that it would be an exploration of IIT, I decided I needed to read it.

Koch starts off by defining consciousness as experience, “the feeling of life itself.”  He muses that the challenge of defining it this way is that it’s only meaningful to other conscious entities.

He then discusses the properties of experience, properties that eventually end up being axioms of the theory.

  1. Experience exists for itself, without need for anything external, such as an observer.
  2. It is structured, that is, it has distinctions, being composed of many internal phenomenal distinctions.
  3. It’s informative, distinct in the way it is, contains a great deal of detail, and is bound together in certain ways.
  4. It’s integrated, irreducible to its independent components.
  5. It’s definite in content and spatiotemporal grain, and is unmistakable.

These then map to postulates of the theory.

  1. Intrinsic Existence: the set of physical elements must specify a set of “differences that make a difference” to the set itself.
  2. Composition: since any experience is structured, this structure must be reflected in the mechanisms that compose the system specifying the experience.
  3. Information: a mechanism contributes to experience only if it specified “differences that make a difference” within the system itself.  A system in its current state generates information to the extent that it specifies the state of a system that could be its possible cause in the past and its effect in the future.
  4. Integrated: the cause-effect structure specified by the system must be unified and irreducible, that is, the system can’t be reduced to independent non-interacting components without losing something essential.
  5. Exclusion: only the set of elements that is maximally irreducible exists for itself, rather than any of its supersets or subsets.

All of this feeds into the “central identity of IIT”, which I’ll quote directly from the book.

The central identity of IIT, a metaphysical statement, makes a strong ontological claim. Not that Φmax merely correlates with experience. Nor the stronger claim that a maximally irreducible cause-effect structure is a necessary and sufficient condition for any one experience. Rather, IIT asserts that any experience is identical to the irreducible, causal interaction of the interdependent physical mechanism that make up the Whole. It is an identity relationship—every facet of any experience maps completely onto the associated maximally irreducible cause-effect structure with nothing left over on either side.

Koch, Christof. The Feeling of Life Itself (The MIT Press) . The MIT Press. Kindle Edition.

All of this factors into the calculation of Φ (pronounced “phi”), a value which indicates the extent to which a system meets all the postulates.  However, as noted in the postulates, there can be Φ values for subsets and supersets of the system.  What we’re interested in is Φmax, the combination of elements that produce the maximum amount of Φ.  According to the Exclusion postulate, only this particular combination is conscious.

The Exclusion postulate allows IIT to avoid talking about multiple consciousnesses within one brain, or of group consciousnesses.  Although it doesn’t rule out scenarios where splitting or combining systems results in new consciousnesses, such as what happens with split-brain patients, or what might happen if two people’s brains were somehow integrated together.

Not all of the brain is necessarily included in its Φmax, but a particular subset.  Koch thinks this is a region he calls the posterior cortical hot zone, including regions in the parietal, temporal, and occipital lobes.  In essence, it’s the overall sensory cortex, the sensorium, as opposed to the action cortex, or motorium at the front of the brain, which is why that Templeton contest between IIT and global workspace theories (GWT) is focused on whether consciousness is more associated with the back or front of the brain.

Koch discusses the evolution of consciousness.  He sees it going back to the reptiles, when the sensory cortex first started to develop.  (Somewhere around the rise of reptiles, or mammals and birds, seems to be where most biologists see consciousness arising, excluding fish, amphibians, and most invertebrates, although as always, a lot depends on the definition of consciousnss being considered.)

Koch in his earlier book, Consciousness: Confessions of a Romantic Reductionist, evinced a comfort level with panpsychism.  In the disccusion of IIT in that book, he implied that IIT and panpsychism were compatible.  But in this book, I got the feeling that he now views IIT more as an alternative to panpsychism, one which resolves some of panpschism’s issues, such as the combination problem.

As noted above, I’m not a fan of IIT, and I can’t say that this book helped much.  All the axioms and postulates make it feel more like philosophy than science.  It continues to  feel very abstract and disconnected from actual neuroscience.  Some of the axioms, such as structure and information, seem vague and redundant to me.  (The book adds examples, but I didn’t find them to help much.)   And others, such as the exclusion principle, seem arbitrary, included to save appearances.

The intrinsic existence one seems to imply metacognitive self awareness, but the theory simply asumes that it emerges somehow from integration, ignoring the actual neuroscience of the regions in the brain associated with introspection.  The postulate also ends up attibuting self awareness to all animals going back to reptiles, despite the lack of any empirical support.

IIT also posits that the feeling of all this emerges from the integration, again ignoring all the neuroscience on affects and survival circuits.  Bringing in all that neuroscience inescapably leads us to the front of the brain, which Koch rules out as having a role in consciousness.

And Scott Aaronson’s classic takedown of the theory remains in my mind.  Koch mentions Aaronson’s criticism, but like Tononi, doubles down and accepts that the arbitrary systems with trivially high Φ that Aaronson envisages are in fact conscious.  If the theory’s designations of consciousness aren’t going to match up with our ability to detect it, how scientific is it really?

But I think my biggest issue with IIT is it inherently attempts to explain the ghost in the machine, particularly how it’s generated.  Most of the other theories I find plausible simply dismiss the idea of the ghost, I think rightly so.  There’s no evidence for a ghost, either spiritual, electromagnetic, or any other variety.  The evidence we have is of the brain and how it functions.

I’ll be happy to go back to IIT if it manages to rack up empirical support.  Until then, it seems like a dead end.

To be clear, I do think integration is crucial, just not in the specific way IIT envisages it.  There are many integration regions in the brain, regions which are themselves integrated with each other.  But Antonio Damasio’s convergence-divergence zones and convergence-divergence regions seem to model this in a much more grounded manner than IIT.

What do you think?  Am I too skeptical of IIT?  Are there virtues of the theory that I’m missing?

Posted in Mind and AI | Tagged , , , , , , | 101 Comments

Angst over mini-brains

Posted on October 26, 2019 by

Organoids, the small pieces of neural tissue grown from stem cells, dubbed “mini-brains”, and used in research, have been getting a lot of attention lately.  Apparently a few neuroscientists are concerned that the organoids might be sentient, and suffering as experiments are performed on them.  There’s growing concern that implanting human mini-brains in animal brains could lead to them becoming self aware.  One guy is even worried that the mini-brains might eventually outsmart us and take over the world or something.

We don’t understand how consciousness is generated, the argument goes, so we can’t be completely sure these lumps of tissue aren’t conscious.  However, the vast majority of neuroscientists aren’t particularly worried, and I think for good reasons.  The probability that organoids are conscious is vanishingly small.  And there are real people, who are unquestionably sentient and suffering, who could be helped by the research.

Still, people are worried, so I think it’s worth considering what the organoids don’t have.  First, consider that, at least currently, they’re typically a few millimeters in diameter and have, at most, 1-2 million neurons in them, which puts them in cockroach territory in terms of nervous system size.

These neurons are not connected to any kind of sensory organs, so there’s no chance that sensory representations are being built.  There are some with retinal cells that respond to light, but we’re talking about very primitive lower level functionality typically seen in the types of worms that very few people regard as conscious.

The neurons are grown in an artificial environment, without the typical support systems, such as glia, or the chemical signalling necessary to regulate gene expression, so the probability that they’ve wired any kind of survival circuitry, which is the source of the impulses that eventually become affective feelings, such as pain, hunger, fear, etc, is infinitessimal.

There’s just not enough there, and what’s there isn’t organized in the right way to process sensory information, have preferences about the states of affairs, or do any of the things we normally associate with consciousness, even if you hold to first order theories.  If you hold to global workspace, higher order thought, or other similar types of theories, there definitely isn’t the necessary organization.

There is a growing concern as these organoids become larger and more developed, but I personally think even if an entire cortex was grown in isolation, it wouldn’t be conscious, at least not without the supporting subcortical infrastructure, such as a thalamus, amygdala, basal ganglia, hippocampus, midbrain, reticular formation, etc, nor would any of these structures, in isolation, be conscious.  Although I’ll admit that the closer to an actual brain the grown structure gets, the less confident I’d feel about that conclusion.

Of course, there are theories of consciousness which might lead to people thinking these mini-brains have it.  For instance, Christof Koch noted that the organoids would have a phi value, which under integrated information theory gives them some degree of consciousness.  But whatever experience this might entail remains very hypothetical, and as noted above, has to be balanced against the definite sentience of suffering patients.

Overall, it seems to me, there are more pressing problems in the world to address than whether tiny clumps of neurons are suffering.  But maybe I’m missing something?

Posted in Zeitgeist | Tagged , , , , | 6 Comments

Michael Graziano on mind uploading

Posted on October 20, 2019 by

Michael Graziano has an article at The Guardian, which feels like an excerpt from his new book, exploring what might happen if we can upload minds:

Imagine that a person’s brain could be scanned in great detail and recreated in a computer simulation. The person’s mind and memories, emotions and personality would be duplicated. In effect, a new and equally valid version of that person would now exist, in a potentially immortal, digital form. This futuristic possibility is called mind uploading. The science of the brain and of consciousness increasingly suggests that mind uploading is possible – there are no laws of physics to prevent it. The technology is likely to be far in our future; it may be centuries before the details are fully worked out – and yet given how much interest and effort is already directed towards that goal, mind uploading seems inevitable. Of course we can’t be certain how it might affect our culture but as the technology of simulation and artificial neural networks shapes up, we can guess what that mind uploading future might be like.

Graziano goes on to discuss how this capability might affect society.  He explores an awkward conversation between the original version of a person and their uploaded version, and posits a society that sees those living in the physical world as being in a sort of larval stage that they would all eventually graduate from into the virtual world of their uploaded elders.

Mind uploading is one of those concepts that a lot of people tend to dismiss out of hand.  Responses seem to vary between it being too hopelessly complicated for us to ever accomplish, to it being impossible, even in principle.  People who have no problem accepting the possibility of faster than light travel, time travel, or many other scientifically dubious propositions, draw the line at mind uploading, even though the physics for mind uploading are far more feasible than those other options.

That’s not to say that mind uploading should be taken as a given.  It is possible that there may eventually turn out to be something that makes it impossible.

For example, I’m currently reading Christof Koch’s new book, The Feeling of Life Itself, in which Koch explores the integrated information theory (IIT) of consciousness.  A big part of IIT is positing that the physical causal structure of the system is crucial.  As far as IIT is concerned, mind uploading is pointless because, even if the information processing is reproduced, if the physical causal structure isn’t, the resulting system won’t be conscious.

I think Koch too quickly dismisses the idea of it being sufficient to reproduce the causal structure at a particular level of organization.  But if he’s right, mind uploading becomes far more difficult.  Although even in that scenario, the possibility of neuromorphic hardware, computer hardware engineered to be physically similar to a nervous system, including physical neurons, synapses, etc, may still eventually make it possible.

Even if nueromorphic hardware isn’t required in principle, it might turn out to be required in practice.  With Moore’s Law sputtering, the computing power to simulate a human brain may never be practical with the traditional von Neumann computer architecture.  A whole brain emulation might be conscious using the standard serialized architecture, but unable to run at anything like the speed of an organic brain.  It might take a neuromorphic architecture, or at least a similarly massively parallel one, to make running a mind in realtime feasible.

However, all of these considerations strike me as engineering difficulties that can eventually be overcome.  Brains exist in nature, and unless anyone finds something magical about them, there’s no reason in principle their operation won’t eventually be reproducible technologically.

Although this may be several centuries in the future.  I do think there’s good reasons to be skeptical of singularity enthusiast / alarmist predictions that it will happen in a few years.  Our knowledge of the brain and mind still have a long way to go before we’ll be able to produce a system with human level intelligence, much less reproduce a particular one.

On the awkward conversation that Graziano envisions between the original and uploaded person, with the original in despair about being the obsolete version, I think the solution would be to simply have mind backups made periodically, but not run until the original person dies.  That should avoid a lot of the existential angst of that conversation.

That’s assuming that there isn’t an ability to share memories between the copies, with maybe the original receiving them through a brain implant of some type.  I think being able to remember being the virtual you would make being the mortal physical version a lot easier to bear.  The architecture of the brain may prevent such sharing from ever being feasible; if so, then the non-executing backups seem the way to go.

I don’t know whether mind uploading will ever be possible, but in a universe ruled by general relativity, not to mention the conservation of energy, it seems like the only plausible way humans may ever be able to go to the stars in person.  If it does turn out for some reason to be impossible, then humanity might be confined to this solar system, with the universe belonging to our AI progeny.

What do you think?  Is mind uploading impossible?  If so, why?  Or is it possible and I’m too pessimistic of it happening in our lifetimes?   Are there reasons to think the singularity is near?

Posted in Zeitgeist | Tagged , , , | 204 Comments

A competition between integration and workspace

Posted on October 17, 2019 by

Back in March, I did a post on a proposed Templeton Foundation project to test major scientific theories of consciousness.  The idea was to start with a head to head competition between the integration information theory (IIT) and global workspace theory (GWT).  Apparently that project got funded and, according to a Science Magazine article, there are now active plans to move forward with it.

The first two contenders are the global workspace theory (GWT), championed by Stanislas Dehaene of the Collège de France in Paris, and the integrated information theory (IIT), proposed by Giulio Tononi of the University of Wisconsin in Madison. The GWT says the brain’s prefrontal cortex, which controls higher order cognitive processes like decision-making, acts as a central computer that collects and prioritizes information from sensory input. It then broadcasts the information to other parts of the brain that carry out tasks. Dehaene thinks this selection process is what we perceive as consciousness. By contrast, the IIT proposes that consciousness arises from the interconnectedness of brain networks. The more neurons interact with one another, the more a being feels conscious—even without sensory input. IIT proponents suspect this process occurs in the back of the brain, where neurons connect in a gridlike structure.

To test the schemes, six labs will run experiments with a total of more than 500 participants, costing the foundation $5 million. The labs, in the United States, Germany, the United Kingdom, and China, will use three techniques to record brain activity as volunteers perform consciousness-related tasks: functional magnetic resonance imaging, electroencephalography, and electrocorticography (a form of EEG done during brain surgery, in which electrodes are placed directly on the brain). In one experiment, researchers will measure the brain’s response when a person becomes aware of an image. The GWT predicts the front of the brain will suddenly become active, whereas the IIT says the back of the brain will be consistently active.

Tononi and Dehaene have agreed to parameters for the experiments and have registered their predictions. To avoid conflicts of interest, the scientists will neither collect nor interpret the data. If the results appear to disprove one theory, each has agreed to admit he was wrong—at least to some extent.

The whole thing has a bit of a publicity stunt feel to it.  As I noted back in March, both of these theories make differing philosophical assumptions about what consciousness fundamentally is, and the authors of both theories used empirical data, as it existed at the time, when formulating their theory.   So I’m not expecting the results to be overwhelmingly conclusive.  (Although it’d be good to be proven wrong on this.)

Lobes of the brain
Image credit: BruceBlaus via Wikipedia

What might be interesting is the front of the brain vs back of the brain thing.  I’ve noted this debate before.  Some scientists, notably people like Tononi and Christof Koch, see consciousness as concentrated in the back part of the brain, in the sensory processing regions including the temporal and parietal lobes.  Others, such as Dehaene, Joseph LeDoux, and Hakwan Lau, think we don’t become conscious of something until it reaches the prefrontal cortex.

This also has relevance on the distinction between first order and higher order theories, that is, between theories that hold that the representations and processing in sensory regions are conscious ones, versus theories that hold that further “higher order” processing in the prefrontal cortex is necessary for us to be conscious of them.

Part of the difficulty is that scientists depend on subject self report to know when those subjects are conscious of something.  However, self report requires the frontal lobes.  There are protocols to minimize the confounding role of self report, such as comparing brain scans of people who see something and report being conscious of it with people who see the same thing but without the requirement to report it.  But a first order advocate can always insist that any remaining frontal activations are superfluous, that all that’s needed for actual consciousness is the posterior activity.

My own money is that the frontal regions are important, perhaps crucial.  But this is complicated.  It’s possible for sensory information from the back part of the brain to trigger sub-cortical activity, such as habitual or reflexive action, without frontal lobe involvement.  It’s even possible to remember what happened during that behavior and consciously retrieve it later, giving us the impression we were conscious of the event during the event, even if we weren’t.

But if we insist that consciousness must include emotional feelings, then I think the frontal lobes become unavoidable.  The survival circuit activations that drive these feelings happen in subcortical regions in the front part of the brain, which have excitatory connections to the prefrontal cortex.  Of course, you could insist that the felt emotions lie in those subcortical circuits rather than the cortex, but severing the connections between those circuits and the prefrontal cortex (like what reportedly used to happen with lobotomies) typically results in deadened emotions.

And all of this is aside from the fact that the introspection machinery is in the very front part of the prefrontal cortex (the frontal poles).  Are we conscious of it if we can’t introspect it?

As I said, complicated.  A lot of this will depend on the assumptions and definitions the experimenters are using.

Still, I’m curious on exactly what they plan to do to test the back versus front paradigms.  If they do figure out a way to conclusively isolate conscious perception with one or the other, it might answer a lot of questions.  And if they do plan to eventually move on to testing theories like local recurrent processing or higher order thought theories, this work might provide a head start.

What do you think?  Am I being too pessimistic on whether these experiments will validate or falsify IIT or GWT?  Or are these theories all hopelessly underdetermined, and we’ll still be arguing over them months after the experimental results are published?

Posted in Zeitgeist | Tagged , , , | 50 Comments

The problems with post-empirical science

Posted on October 9, 2019 by

Jim Baggott has a pretty good piece at Aeon on the problems with post-empirical science.  I’ve highlighted Baggott’s views before.  Along with others like Sabine Hossenfelder and Peter Woit, he calls attention to a serious issue in physics, the rising acceptance of theories that show little promise of being testable in the foreseeable future.  In many ways, this piece is a reaction to Sean Carroll’s new book, Something Deeply Hidden, which champions the Many Worlds Interpretation of quantum physics.

Rhetoric in this debate often gets pretty heated, but Baggott is relatively restrained.  After describing examples of physicists who express certitude in the existence of things like the multiverse, he wraps up his piece with this:

Perhaps we should begin with a small first step. Let’s acknowledge that theoretical physicists are perfectly entitled to believe, write and say whatever they want, within reason. But is it asking too much that they make their assertions with some honesty? Instead of ‘the multiverse exists’ and ‘it might be true’, is it really so difficult to say something like ‘the multiverse has some philosophical attractions, but it is highly speculative and controversial, and there is no evidence for it’? I appreciate that such caveats get lost or become mangled when transferred into a popular media obsessed with sensation, but this would then be a failure of journalism or science writing, rather than a failure of scientific integrity.

Many physicists who discuss things like multiverses, when read at length, are careful to provide these kinds of caveats.  But that can be hard to remember when reading an entire book on multiverse concepts.

I do think scientists need the freedom to speculate in these types of realms.  They shouldn’t necessarily be constrained by what can be tested today.  It’s always possible someone will find a way to these these propositions in the future.

For example, when Albert Einstein and co-authors wrote about the EPR paradox, many condemned it as hopeless metaphysical navel-gazing speculation, but decades later John Bell figured out a way to test it.  Our knowledge of entanglement, along with related concepts like decoherence, was built on this early purely theoretical speculation.

On the other hand, science has credibility because it produces reliable knowledge.  It seems like there should be a clear divide between the well tested theories that make up that reliable knowledge, or in many cases the well tested aspects of a theory, and the speculative stuff.

I don’t think it’s fair to call this stuff “non-science” or “pseudoscience.”  It’s often formulated by working scientists using the same logic and mathematics used to derive well tested theories.  It’s compatible with known empirical data.  Unlike actual pseudosciences, like astrology, cryptozoology, or parapsychology, which ignore the fact that they were long ago falsified, speculative science remains possible, at least in principle.

Maybe we could call the reliable stuff “empirical science” or “settled science” and the speculative stuff, well, “speculative science”?

(Baggott himself in his book, Farewell to Reality, used the phrase “Authorized Version of Reality” for reliable knowledge, which I thought was a terrible name, seeming to imply that there was a council somewhere authorizing theories.  Thankfully, he appears to have moved past that phrase.)

In the end, it doesn’t really matter which labels we use, so long as the distinction is made.

What do you think?  Does theoretical speculation deserve the name “science”?  If not, then were Einstein and his co-authors doing science or something else when they published their EPR paradox paper?  Does it matter how far it is from empirical verification?  Or how many assumptions are being made beyond reliable knowledge?  Where do we draw the line?

Posted in Zeitgeist | Tagged , , , , | 67 Comments

Recommendation: Memory

Posted on October 2, 2019 by

Cover to the book: MemoryI’ve noted before that I think science fiction author Linda Nagata is an underrated talent.  She crafts mind bendingly imaginative tales.  I’ve recommend several of her books, all in The Nanotech Succession series, which she has recently started adding to again with her latest book: Edges.  (Technically the newer books are part of a new series she calls The Inverted Frontier, but they’re continuations of the earlier Nanotech books.)

I haven’t read much of her stuff outside of the Nanotech / Inverted series mainly because I’m primarily a space opera and/or far future fiction fan, and her other stuff doesn’t seem to fit that mold.  But she wrote a book, called Memory, which originally was meant to be a standalone novel.  However she recently revealed that the book has now been retconned into the Nanotech universe.

She even linked it to an enigmatic scene in Edges where the villain is explaining the massive megastructures appearing on the ship’s long range scopes of a star light years away, including a ring shaped world in orbit of a sun.  This world, Nagata revealed in a blog post, is the setting of Memory.  In addition, while her upcoming book Silver will be a direct sequel to Edges, it will also serve as a sequel to Memory.

She was careful to stipulate that Memory is not required reading for Silver, but I’d already been tempted by the book’s description and decided I wanted to read it.

The story takes place on the ring shaped world with the land on the outer rim of the ring.  (The outer part threw me at first, since I expected the habitable part to be on the inner rim, with rotation providing artificial gravity.  There is gravity, but apparently it’s provided in some other manner.)  Another megastructure, “The Bow of Heaven” is visible in the sky.

The ring does rotate, providing night and day cycles.  According to story lore, this world was created by a goddess.  But a dark god made war with the goddess and the world was broken.  The goddess now reportedly sleeps and dreams.  (If you’ve read Edges, you’ll have a little insight into the “goddess” and “dark god.”)

An ever present reality in this world is the silver, a cloud of substance that arises at night or whenever the sun is obscured.  When the silver comes, it frequently destroys whatever it touches, including people.  Sometimes the silver recreates structures it consumed long ago, such as ancient cities, although the new structures are often made of different material from the originals.  But the silver never returns people, although it does sometimes return artificial intelligence devices.

So people must avoid the silver.  They can often do so by retreating at night to high places since the silver often stays in lower lying areas.  Or they can shelter in temples, which are built around pits that produce small robots that are able to keep the silver back.  However, neither of these strategies completely guarantee safety from the silver, as sometimes it surges and covers high places, or overwhelms temple defenses.

In addition, people in this world are essentially reincarnated after they die.  They generally don’t remember their past lives, but do appear to have innate “talents” which are skills acquired in past lives.  They also can only physically mate with one person in all the world, their reincarnated spouse, which they must somehow find.  There are matchmakers which can help people find their one lover, but many have to wander the world for years looking for theirs, and some never do find them, at least in that life.

The protagonist, Jubilee, as a child, watches her brother, Jolly, be taken by the silver.  Years later, when sitting on the wall of her mother’s temple, as the silver is forming for that night, a man walks out of the silver, an incomprehensible event.  The man arrogantly demands to know where her brother Jolly is, and attempts to draw Jubilee into the silver.  She escapes, but shortly thereafter she receives a message from Jolly, her lost brother.  Somehow he has returned from the silver, but is in a far away land, and fleeing the man from the silver.

Nagata is careful to craft the story in such a way that this is unmistakably science fiction, but it often feels like fantasy, or perhaps what used to be called planetary romance, complete with an epic quest across several lands.  The limited perspective of the characters enhances this feel.

I enjoyed this book immensely, but I’m actually glad I didn’t read it before now.  While the story was very engaging, I wasn’t satisfied by the ending.  Too much remained unexplained about the world and setting.  I got the feeling that the book was meant to be the first in a series, the rest of which never materialized.  (It came out in 2003.)

Fortunately, Nagata is now working on her new book, Silver, which should provide opportunities to answer many of the questions.  Some of them were already answered by the events in Edges.  One thing reading Memory does do is clarify the stakes at the end of Edges.

So, mind expanding fiction that is highly recommended!

Posted in Science Fiction | Tagged , , | 15 Comments

A standard model of consciousness?

Posted on September 29, 2019 by

I’ve often noted that I find more consilience than disagreement between the empirically grounded theories of consciousness.  They seem to be looking at the problem at differing levels of organization, and together they may present a growing scientific consensus about how the mind works.

In particular, a few weeks ago, when discussing higher order theories, I made the observation that another theory historically discussed on this blog, the attention schema theory, could be considered a higher order representation.  It’s gratifying to know that my conjecture wasn’t hopelessly off base, as Michael Graziano, the author of the attention schema theory, along with coauthors, has the same idea.

In a new paper (which unfortunately appears to be paywalled), they take a shot at reconciling various theories and philosophies of consciousness, and reach a striking conclusion:  Toward a standard model of consciousness: Reconciling the attention schema, global workspace, higher-order thought, and illusionist theories:

Here we examine how people’s understanding of consciousness may have been shaped by an implicit theory of mind. This social cognition approach may help to make sense of an apparent divide between the physically incoherent consciousness we think we have and the complex, rich, but mechanistic consciousness we may actually have. We suggest this approach helps reconcile some of the current cognitive neuroscience theories of consciousness. We argue that a single, coherent explanation of consciousness is available and has been for some time, encompassing the views of many researchers, but is not yet recognized. It is obscured partly by terminological differences, and partly because researchers view isolated pieces of it as rival theories. It may be time to recognize that a deeper, coherent pool of ideas, a kind of standard model, is available to explain multiple layers of consciousness and how they relate to specific networks within the brain.

As a quick reminder, global workspace theories (GWT) argue that consciousness results from mental content making into a global workspace, often thought to range across the fronto-parietal network.  Higher order theories (HOT) posit that consciousness involves secondary or higher order thought or representations of first order sensory representations or other more primal processing, with the higher order representations though to be in the prefrontal cortex, although possibly in other regions as well.  The attention schema theory (AST) posits that awareness is a model, a schema, of the messy emergent multilevel process of attention.

Before attempting a reconciliation, the authors make a distinction between two views of consciousness, which they label i-consciousness and m-consciousness.  i-consciousness is the information processing view of consciousness, how information is selected, enhanced, and used.  m-consciousness is the mysterious experiential essence version.  (This distinction seems similar, but not exactly equivalent, to Ned Block’s distinction between access consciousness and phenomenal consciousness.)

The theories noted above all focus primarily on i-consciousness, often to the frustration of those concerned about m-consciousness.  However, the authors argue that our impression of m-consciousness comes from an internal model the brain uses to track its own processing, a model that evolved to be effective rather than accurate, meaning that what it presents is a simplified and cartoonish picture of that processing, one that because of the simplifications, produces something that seems magical and incompatible with i-consciousness, but that nevertheless is a component of it.  This description of m-consciousness resonates with illusionist theories.

For the authors, the internal m-consciousness model is the attention schema.  However, while they think the attention schema uniquely tracks subjective experience, they admit that the brain probably produces numerous models of different aspects of its processing.  All of these models could be considered the higher order representations of HOT.

They also put forth the attention schema as a bridge between HOT and GWT.  The global workspace could be seen as the highest level of attention processing, which would make the attention schema essentially a higher order representation of the global workspace.

So we have a multilevel attention mechanism, which culminates in the global workspace, which is modeled by the attention schema, making it a higher order representation of the workspace.  The attention schema, along with all the other higher order representations, produces m-consciousness, a simplified cartoonish model that is effective but not accurate, the contents of which we could describe as an illusion.  All of this would make up i-consciousness.

My take is that this is a compelling view, although perhaps a bit too slanted toward the AST.  For example, I think emotional feelings are higher order representations of lower level reflexive survival circuits, which seem just as central to subjective experience as attention.  All of this, to me, strengthens HOT as the more fundamental view of what’s happening.  (Which since HOT is a collection of theories, means we’re still far from a final theory, if we indeed ever get to one final one.)

But the convergence of these theories, if they are in fact converging, is starting to look like the rough outlines of a standard model, a collection of theories that together provide an account of the mind.  Only time and additional research will tell if it actually is.

Unless of course, I’m missing something?

h/t Neuroskeptic

Posted in Mind and AI | Tagged , , , , | 129 Comments

Keith Frankish on the consciousness illusion

Posted on September 26, 2019 by

Along the lines of last night’s post, Keith Frankish has an article at Aeon describing and defending the illusionist viewpoint, that phenomenal consciousness is an illusion.  It’s an excellent introduction for anyone who isn’t familiar with the basic argument.

As noted before, I think the illusionists are right about the reality, but I’m not sure using the word “illusion” is productive. We could just as easily say that yes, phenomenal consciousness exists *subjectively* but not objectively, and this is how that subjective experience is constructed. There is some value in using stark language to get people’s attention, but it also frequently gets their summary dismissal.

I’m also not entirely sure it’s all in the introspection mechanisms.  Phenomenal qualities seem useful in discriminating between different objects, and the affect lacing the brain weaves in also clues the deliberation engine on how to regard those objects.  It seems likely that our introspective representations of these perceptual representations are value added rather than entirely constructive.  Thinking the latter implies a lot of processing overload for introspection, which doesn’t necessarily feel adaptive to me.

What I definitely think is an illusion, however, is the notion that qualia exist as something above and beyond the neural processing in the brain, that it “arises” in some fashion from that processing.  A physicalist might talk in those terms in a metaphorical manner, but what they usually mean is that it is the processing.

Anyway, Frankish’s piece is worth checking out, particularly if you think illusionism is prima facie false.

Posted in Zeitgeist | Tagged , , , | 188 Comments