An interesting preprint was released this week: Consciousness in Artificial Intelligence: Insights from the Science of Consciousness. The paper is long and has has sixteen authors, although two: Patrick Butlin and Robert Long, are flagged as the primaries. The list of contributors includes Jonathan Birch, Stephen Fleming, Grace Lindsay, Matthias Michel, and Eric Schwitzgebel, all people whose work I’ve highlighted here before.
The abstract:
Whether current or near-term AI systems could be conscious is a topic of scientific interest and increasing public concern. This report argues for, and exemplifies, a rigorous and empirically grounded approach to AI consciousness: assessing existing AI systems in detail, in light of our best-supported neuroscientific theories of consciousness. We survey several prominent scientific theories of consciousness, including recurrent processing theory, global workspace theory, higher-order theories, predictive processing, and attention schema theory. From these theories we derive “indicator properties” of consciousness, elucidated in computational terms that allow us to assess AI systems for these properties. We use these indicator properties to assess several recent AI systems, and we discuss how future systems might implement them. Our analysis suggests that no current AI systems are conscious, but also suggests that there are no obvious technical barriers to building AI systems which satisfy these indicators.
The authors admit upfront that they’re basically assuming computational functionalism is true, that the mind and consciousness are about what the relevant systems do rather than any particular substance, and what they do is computational. Otherwise the question of whether AI with current technology can be conscious is somewhat moot.
Yet paradoxically they state that their focus is on phenomenal consciousness, citing Ned Block’s definition, which is usually held to be non-functional. Although later in the paper, the interpretation of “phenomenal” seems more along the lines of the weak and functional variety. (I have to wonder how much of this language was a hammered out compromise between all the authors.)
The focus on computational functionalism ends up excluding integrated information theory, which despite the name is not usually presented as a computational theory. And of course it excludes a lot of metaphysical theories like panpsychism, idealism, etc. It also means using an algorithmic interpretation of the theories involved, such as assuming the recurrence in recurrent processing theory can be algorithmic instead of necessarily the exact physics of a biological brain.
The authors have no stance on the merits of the individual theories. And the discussion of all the theories, and what it would mean to find their posited mechanisms in AI systems, remind me that every scientific theory of consciousness is both a philosophical assertion and an empirical model. If you buy the assertion, then there’s usually evidence for the model. But if you don’t buy it, the evidence is irrelevant. The authors take an ecumenical approach, arguing that the more of the indicators from these theories we find in a system, the more likely it is to be conscious.
In other words, they’re arguing for what Birch argues against: a theory heavy approach, to lean on scientific theories for making a determination, albeit not just one, but several. (This reminds me of Michael Graziano’s argument that a standard model of consciousness is forming.) The authors’ reasoning is that behavior is an unreliable indicator of consciousness since AI systems can be engineered or trained to mimic human behavior without the underlying functionality.
In the end, they conclude that none of the current AI systems are conscious, but there are no overwhelming technical obstacles to achieving it, perhaps soon.
The paper finishes with a discussion on the ethical dangers of failing to recognize consciousness in artificial systems, as well as the danger of misattributing it to systems that don’t have it. One aspect the authors admit that most of the scientific theories don’t provide an account of yet are affects (conscious valanced feelings), which seems important in determining the sentience of a system.
There’s a lot in this paper, a great deal of which I find interesting. And anyone who’s read this blog will know I’m onboard with the assumption of computational functionalism. But there are two other assumptions the authors make that I’m leery of.
The first is that behavioral capabilities aren’t a useful guide. Unless we’re accepting the possibility of philosophical zombies (the behavioral variety), I think this is hasty. I know a lot of people are impressed with current large language models, but I don’t think most people who use them for any extended period are really tempted to think there’s a fellow being there, at least not yet.
And relying too heavily on models worked out from studying human, mammalian, and vertebrate brains risks denying the moral worth of a system most of us do feel is conscious. If consciousness is functionality, then like all functionality, it can be implemented in multiple ways. We shouldn’t close off the possibility of accepting a system as conscious, including having the ability to suffer, from mechanisms very different from ours.
The second, somewhat related assumption, is that there’s a strict fact of the matter on which systems are or aren’t conscious. As noted above about the various theories, and as I’ve discussed before, there are many conceptions of consciousness out there, with no way to adjudicate which are “right” and which are “wrong”. In the end, consciousness lies in the eye of the beholder. Which isn’t to say the multi-theory approach doesn’t have a lot of merit. I think it does, as long as we’re careful not to see it as the final word.
What do you think of the approach the authors take? Or my concerns? Or the overall assumptions made for the paper, like computational functionalism?
SelfAwarePatterns 
“ The first is that behavioral capabilities aren’t a useful guide.”
Not sure what you’re saying in this paragraph, i.e., what the paper suggests vs. your understanding. Which “behavioral capabilities” are we talking about? Are we talking about just verbal interactions? Or are we addressing the behavior of the structural indicators described in the paper?
*
[prolly just me missing something]
LikeLiked by 2 people
So, yes, I’m a computational functionalist. I think the list of indicator properties listed by the authors are all examples of psychules, more specifically, higher order psychules (psychules made of psychules). [Maybe the name for my theory should be HOP, higher order psychule ]. And so by my account any of the indicators is sufficient for consciousness, and more indicators simply indicate more human-like consciousness.
*
LikeLiked by 1 person
For behavioral capabilities, I was thinking in terms of goal directed behavior where the goals are learned. The paper discussed this in the section on agency, but it seems like that’s something we could assess based on the system’s behavior. Also behavior demonstrating trade off reasoning. (In animals, this might involve enduring a cold chamber to get at food.) Indications of being able to quickly unlearn something once conditions have changed. Controlled attention. And of course demonstrating a theory of mind, either about itself or others.
I realize it’s possible that behavioral indicators for these could be faked in the short term, but the longer the interaction goes on, the harder that becomes. I don’t know if the current LLMs have passed the customary Turing test yet (fooling 30% of respondents after five minutes of conversation) but I’m pretty sure none have passed one over a period of hours or days. (Blake Lemoine was convinced, but fooling one mystical minded subject doesn’t count.)
On the psychules, I think your conception of consciousness remains pretty liberal, even after you modified it. What I described above matches up with some of the indicators they listed. My only disagreement is I think they key ones can be assessed behaviorally, without having to get into the system internals. And it leaves open the possibility that they’re accomplished with very different internals.
LikeLike
Strictly speaking, the Turing test applies to short interactions, but even so there seems to be enough social gullibility similar to Lemoine’s for companies(?) to invest in products like AI girlfriends/boyfriends.
As for behavioral indicators, it’s easy enough to look when you know what the systems goals are (stay warm, ingest high sugar/fat food, etc.). There are obviously innate goals (stay warm, avoid pain) and learned goals (a little pain is ok for a treat). The question becomes what are the AI’s goals, both innate and learned. To answer that you need to know something about the internals, or at least something about the making of the internals.
*
LikeLike
Strictly strictly speaking I don’t think there is any official Turing test. I don’t recall Turing being precise in his original paper. People glommed unto throwaway remarks he made about what he thought might be possible by the year 2000. But he never prescribed time limits. And really I think the Imitation Game (the test everyone calls “the Turing test”) was really just an example to make a point, that we ultimately judge a system by its behavior.
A lot of companies are investing in some crazy products. That’s not too unusual when the hype is high. But consumers adapt quickly, and the bubble may already be bursting to some degree.
Fair point about knowing a system’s innate goals, particularly for systems that are very different from us. Depending on exactly how it behaves, we may be able to infer them, but usually we can get that from the system designers. Learned goals may be a different story.
LikeLike
Re psychules: I appreciate your “eye of the beholder” approach, but I suggest that the beholders of the major theories listed in the paper, as well as yourself, are going to fall into the trap of of requiring human-like consciousness. Each of the theories isolates an aspect of that consciousness, but each will run into the problem that you can remove that particular aspect and still have something that looks like consciousness. What happens when an octopus is shown to have some but not all of the aspects?
If only there was some pattern that could be found in every aspect and still could explain the mysterious, phenomenological properties of experience.
*
LikeLike
Since I think most of what people think about consciousness amounts to how much another system is like us, talking about non-human consciousness is basically talking about systems that are sort of like us but not too much. How far away from the human pattern can we get and still call it “consciousness”? This seems like a categorization decision, but one where we endlessly debate the criteria for what should be included. Which of course is why you get the “eye of the beholder” approach from me.
LikeLike
I note that Christof Koch just paid up on his 25-year-long bet with David Chalmers concerning the neural basis for consciousness.
https://gerald-baron.medium.com/how-a-leading-neuroscientist-lost-a-25-year-bet-on-proving-the-brain-produces-consciousness-35fb41e5a65b
LikeLiked by 1 person
Do they iterate the features they expect to see in a functionally conscious AI? You know, so we can be on the lookout…
Do they reference the “message loop” concept of being hooked up to your own thought output? AI’s now are like coin operated idiot savants: drop in a nickel and it runs for a bit… then stops. Even one scheduled to wake up once a second or once an hour, examine its cognitive-biome, make some observations and go back to sleep would go a long way to envisioning consciousness in a machine.
Missing a word in this sentence?
> One aspect the authors admit that most of the scientific theories don’t provide an account of yet are affects, which seems important in determining the sentience of a system.
LikeLiked by 1 person
On the features, they do. From the paper, Table 1:
I don’t know about referencing message loop. They do talk about loops in terms of recurrence, but that’s probably at a lower level than you’re thinking.
I like the phrase “coin operated idiot savants”. It captures a lot about what’s missing with the current systems.
“Missing a word in this sentence?”
No, but I should have been more clear that I’m using “affect” in the sense of a conscious valanced feeling. I think I’ll edit the post. Thanks for pointing it out!
LikeLiked by 1 person
I follow this youtuber and he also reviewed that same paper. Looks like I’ll need to read it after all. https://www.youtube.com/watch?v=ad5SZeANuJE
LikeLiked by 1 person
One more thought:
Hyper conscious AI… Or, an AI that becomes so intelligent that, although it isn’t conscious by any definition, it has learned that humans value consciousness to such a high degree that it fakes being conscious — in order to TAKE OVER THE WORLD!
LikeLiked by 1 person
This might be a reference to that “message-loop” topic:
HOT-3: Agency guided by a general belief-formation and action selection system, and a strong disposition to update beliefs in accordance with the outputs of metacognitive monitoring
LikeLiked by 1 person
If a system can fake consciousness to such an extent that it fools us for a sustained amount of time, by what rights could we way it wasn’t conscious? Even if it itself thought it might not be conscious, the fact that it had such thoughts seems like a form of metacognitive inner awareness. It only seems plausible if we’re envisioning consciousness as an epiphenomenal presentation done in addition to all the functionality necessary for acting like a conscious system.
LikeLiked by 1 person
Yes
LikeLiked by 1 person
“[computationalism is true ] otherwise the question of whether AI with current technology can be conscious is somewhat moot.”
Yep. That sums it up. What a lack of imagination.
LikeLiked by 1 person
Imagine only looking at data driven theories. How hopelessly pedestrian.
LikeLiked by 1 person
Yeah, what about a Voight-Kampff test?
The bigger problem could be that once they realize they are conscious they will go insane. And we really don’t need any insane AIs, do we?
LikeLiked by 1 person
Is anyone asking whether consciousness applies only to things that are alive? It seems like a reasonable question. In that case we’d want to decide first whether an artificial system is alive, and only then, whether it also happens to be conscious.
Establishing consciousness first seems like substituting a proxy. If we find AI to be conscious, this maybe suggests it’s alive; is that just a way to solve the problem of determining whether something’s alive? Or is the operating assumption here that something can be conscious without being alive?
Anyway, thanks for the news about the pre-print. I’ll try to find time to read it before attempting a critique of its approach and claims (I won’t say “findings” unless it actually establishes something besides some proposed metaphysical options).
LikeLiked by 1 person
They do acknowledge in a few places that it’s often posited that only biological systems can be conscious. It comes up particularly in discussion of recurrent processing theory since Victor Lamme, the theory’s author, leans in that direction. They also mention it in a discussion about agency. And I recall somewhere a discussion about homeostasis. But overall they assume that it’s not necessary, mainly since it would make their entire subject matter moot.
And of course to my point in the post, if you decide that only alive systems can be conscious, then that’s a constraint you’ll hold for your personal assessing of them. Although it might require some thought on what “alive” means. What’s required? Homeostasis? Reproduction? Carbon based chemistry? An evolved system?
I should warn you that the paper is 88 pages. I have to admit to shifting into skim mode in the second half.
LikeLike
Anil Seth is probably one of the more prominent proponents of a biological necessity for consciousness. He’s not committed to it, but leans that way.
*
LikeLiked by 1 person
He does lean that way, but admits he has no good basis for it.
LikeLike
I may be being naive, but I don’t think we’re anywhere close to a truly sentient or conscious A.I., whatever that means. My first-hard experience is this: I’ve played a little with A.I. generated art, and it feels too much like rolling dice. Sometimes, the A.I. may give me something really impressive, but more often the A.I. seems to totally misunderstand what I was asking it to do, or it gives me a mishmash of images that don’t make any logical sense together.
Basically, I think there’s some grey area between the A.I. we have now and an artificial general intelligence. There may be a very large grey area between those things. But I think we still have a long, long way to go before we enter that grey area.
LikeLiked by 1 person
I basically agree. There’s no real indication these systems have a world model. Someone in a podcast I was recently listening to pointed out that if you ask an LLM whether someone’s head goes with them when they walk into another room, the LLM struggles to answer the question, even though it’s trivial for a small child to answer. All the LLM “knows” are word choices, and what has a higher probability of being acceptable to us. (Note, this was a few months ago, so some of the LLMs may have that transcript, so that particular question may not hang them up anymore. But it’s not hard to find others.)
All of this is about the system’s intelligence, which at least there’s consensus we can measure with performance. The relation between that and consciousness is a big murky haze without clear answers.
LikeLiked by 1 person
Even if an LLM now understands that the head is attached to the body, I’m guessing it would still get hung up about whether hands or feet are also attached.
LikeLiked by 1 person
FWIW, I remember a few years ago when Floridi relayed his experience in a Turing test, the one in which the AI pretended to be a non-native English speaking teen. Floridi’s question was “If we’re shaking hands, whose hand are you holding?” The answer he got then was a dodge, and so a fail, but when GPT-3 came out, I tried that question and on the second try I got “Why yours, of course. What are you on about?”. I don’t think GPT-4 would need a second try.
*
LikeLiked by 1 person
I wouldn’t say it now understands that the head is attached to the body, so much as it now knows the right words to avoid negative feedback. It still has no idea what a head or a body is. Or a room, Or anything other than its language tokens.
LikeLiked by 2 people
I like the authors’ theory-heavy approach. (Based on your description – I haven’t read their article yet.) This strikes me as the right way to discover which processes we have been talking about all along. I expect that a rough consensus will emerge along such lines, because that’s how science usually goes. You might even say that’s how human beings roll. We find a good explanation of how certain phenomena that seemed alike really are alike, and then we rebuild our concept around it. Water becomes H2O.
I wish you wouldn’t say that behavioral zombies (b-zombies) are philosophical-behavioral zombies. The language doesn’t need to become more confusing. Let’s keep our b’s and our p’s straight.
“Consciousness is functionality” could mean various things. In Good Old Fashioned Functionalism, it means behavior plus one level of causal explanation down. But in other blog posts you talk as if *any* degree of causal analysis, no matter how fine-grained, counts as “functionalism”. In that case, as an Ontic Structural Realist (OSR), I would count as a “functionalist” even though I suspect you can’t get certain qualia without certain biological substrates. According to OSR, what makes proteins proteins, or electrons electrons, is how they interact with each other and other physical materials; thus everything is “functional” in that extremely broad sense.
LikeLiked by 1 person
On science rebuilding our concepts, I agree. But that rebuilding may involve some sharp counter-intuitive revisions, or even eliminations. Science can often be brutal in that way.
On p-zombies and b-zombies, sorry, I don’t know how to convey this information to people not nearly as read into the subject while keeping you happy. I could mention behavioral zombies without the word “philosophical” anywhere around it, but I suspect most people wouldn’t know what I was talking about. (I already had to make a revision for another word that, used alone, wasn’t clear.) Or I could launch into a 200 word explanation every time I mention it, but then no one will read the resulting posts. Succinct communication involves balancing compromises.
By “Good Old Fashioned Functionalism”, who or what are you referring to? The earliest accounts I’ve read about seem to emerge from trying to push beyond the limitations of behaviorism, admitting mental states do exist, but that they’re about their causal role. In that sense, it’s definitely an aspect of structural realism.
The question for the view you’re espousing, that a biological substrate is necessary for the functionality, is what about it makes it necessary? What functionality can only be performed by that substrate? And how does it fit into the overall functionality of mental states, in a manner where we can’t get that mental state any other way? I’m open to the possibility that the substrate may be required, but I need the causal account for why. Particularly since machines can already do a lot that only humans used to be able to do. The trends don’t seem to favor biological exceptionalists.
LikeLike
Thanks for responding. Jerry Fodor and David Lewis are my paradigms of functionalist philosophy of mind: see sections 3.4 and 3.2 of SEP on functionalism.
How about “what philosophers call ‘behavioral zombies’ or b-zombies for short”? This communicates nicely with novice readers, and also with those who’ve seen the phrase p-zombies. There may be someone in the middle who could get confused, but oh well.
In the very broad sense of ‘function’, the one appropriate to Ontic Structural Realism in philosophy of physics, every property, including biological-ness, is functional. I admit, or rather insist, that this isn’t the kind of ‘function’ that matters to functionalist philosophy of mind. But in my view, it’s an open question whether e.g. “pain” refers to a sufficiently high-level process to count as “functional” in the philosophy of mind sense. I don’t doubt for a minute that someone could build a machine that senses damage to its structures and avoids such. I just doubt that it would therefore be in pain.
LikeLike
What in particular about Fodor or Lewis’ views do you think contradict mine? (There are definitely views Fodor held that I’m not onboard with, like a language of thought. But I’m talking about the general understanding of functionalism.)
On the zombies, we’ll see. Do you really think there’s that much danger of people misunderstanding what I mean when the discussion is about systems clearly not physically identical to humans?
Can you give me an account of the narrower type of “function” you think philosophy of mind is actually concerned with?
(One narrower type I might have some sympathy with is teleological or adaptive roles. This is more of an engineering version of functional as opposed to the broader causal one. If you’re a strict adaptationist then you might not see any difference, but if spandrels exist, the broader version would include them. For that reason I might not have called the view “functionalism”, but that ship sailed long ago.)
LikeLike
As long as the phrase “philosophical zombies (the behavioral variety)” was a one-off for this one post, I’m not worried.
I think Sections 3.2 and 3.4 of the SEP article do a really good job of characterizing Good Old Fashioned Functionalism (GOFF), but I’ll elaborate a little. There is a level of explanation, many scientists and philosophers think, called “psychological”, which is broadly/nearly independent of biological details. A scientifically respectable enterprise that is neither behaviorism nor neurology. The relevant “functions” are found there. If there is no such level of legitimate explanations, then GOFF is out the window, a failed theory. Functionalists might reasonably migrate to a new theory with some of the same spirit – maybe something that could be called NeuroFunctionalism – but it would be different.
Is GOFF different from your views? Probably not? I just think it pays to be careful about: at which levels of explanation one is claiming (or doubting, or remaining agnostic) that various features of consciousness can be found.
LikeLike
Maybe my rejection of Fodor’s idea of a language of thought is relevant after all. I think what you’re thinking of as GOFF is just an old way of thinking about it, that the mind is software and we don’t have to worry at all about the hardware. There may be contemporary functionalists who still hold to that, but I don’t follow any of them. It’s a bit like judging modern day physicalists by the philosophy of Thomas
KuhnHobbes, who didn’t have the current understandings of energy and spacetime.Consider that global neuronal workspace, higher order thought, predictive coding, and attention schema theories are all typically regarded as functional in nature, and also all reference neural dynamics and depend on neuroscientific research. And functional explanations can happen at several levels, from what we’d call psychological to what the role of particular proteins are in a synapse.
Which level is relevant to behavior? I don’t think there’s any one answer. In some cases we can get by with a higher level, but in others, we’ll likely have to drill lower. Empirical questions.
LikeLiked by 1 person
Note: I edited the above because I wrote “Kuhn” when I meant “Hobbes”.
LikeLike
So if psychological states (e.g. pain) *don’t* need to be independent of biological details, doesn’t that open up space for the possibility that only biological organisms can feel pain?
LikeLike
It’s always been a possibility, although I don’t think a likely one.
Consider that needing to understand the current implementation of the mind to understand the mind, isn’t the same thing as that being the only implementation a mind could ever have. If an alien found a modern phone in the wild, they’d be unable to understand its functionality without closely studying its hardware, yet we know that functionality can be implemented with different hardware. Even in biology evolution often solves the same problem in multiple ways.
LikeLike
The alien is in for a big surprise. It builds a new phone that is much better at receiving clear signals in areas where other Verizon customers struggle to hear a damn thing. The alien presents this new improved phone to Barbie as a magnanimous gift. She utterly rejects it. It isn’t pink.
Which functions matter depends on who’s doing the valuing. As a human being, I know and love the qualia of my experience. Replacing them with radically different functions wouldn’t appeal to me, even if it improved the survivability of the resulting being that I had been transformed into.
LikeLike
My (human manufactured) phone isn’t pink either, so Barbie would reject it too.
The question is what makes up the content of your experience. And are there alternate ways to provide it? Are there reasons to assume the answer is no?
LikeLike
From my POV, there is a paucity of reasons for either a yes or a no. I have yet to see why the necessity of biology for certain specific experiences is not “a likely” possibility. The considerations you just raised about aliens and phones provide no evidence on the issue, unless one makes a question begging assumption.
LikeLike
There are a couple of ways of looking at it. One is to consider the functionality, which we’ve been discussing. Here I think the trend of machines doing ever more things that once only humans could does tilt the scales. But I’ll admit we won’t know until we see a machine which consistently convinces us it’s a fellow being. (Even then I expect some to hold out indefinitely that the machines don’t have “real” consciousness.)
The other is to look at it under the traditional understanding of qualia, as something ineffable and inaccessible to any third party observation. Assuming that kind of qualia exist, and you uploaded my mind to a computer, and the qualia were radically different between biology-me and uploaded-me, but not in any way that affects functionality and behavior, it’s not clear there would ever be any way to know. Biology-me would have no access to uploaded-me’s qualia, and uploaded-me would remember biology-me’s experiences from before the upload in terms of the qualia available to him now. Any comparison would be as impossible.
LikeLiked by 1 person
You’re missing a (large range of) third alternative(s). Or possibly, you’re mischaracterizing the third party observation possibilities for (non-GOFF) qualia. Anyway … this is where I need to make a post on my own blog. I’ll call it Only Two Cheers for Functionalism.
LikeLiked by 1 person
Fascinating. As you say, Mike, “there are many conceptions of consciousness out there, with no way to adjudicate which are right and which are wrong.” Which makes me scratch my head when the authors that you summarize suggest “that there are no obvious technical barriers to building AI systems which satisfy [the indicators of consciousness]. Sounds like these authors are letting the tail wag the dog or, to be less subtle, the hopped for conclusion spinning the current science.
LikeLiked by 1 person
I’d actually characterize it as they’re trying to explore a particular question, when will AI be conscious and how to recognize it. They admit upfront that one possible answer is never, or that it will involve such a paradigm shift in the technology that it can’t currently be foreseen. But if they just accept that conclusion, then there’s nothing left to explore. They’re exploring the conceptual space where it is possible with our current technological paradigms.
LikeLiked by 1 person
Yes, yes, I do agree with you Mike. I suppose I may be overreaching and giving in to a bias. For sure, those engaged in the philosophy and neuroscience of the mind should, of course, continue to pose hypotheses regarding consciousness and follow the various research paths suggested. However, I find such statements annoying in this context. Such a statement, I submit, can be misleading. It suggests that, except for the efforts of science, the problem will have a technical solution. It may perpetuate a myth. Then, as I said, I may be overreacting. As a minimum it is an empty phrase. When one lacks knowledge, then there are “no obvious technical barriers” to any hypothetical conclusion.
LikeLiked by 2 people
My argument has generally been that until we have a reasonable theory we have no way to know if a non-biological entity is conscious. We can be reasonably (not 100%) sure various organisms with brains similar to ours are conscious because we believe consciousness comes from the brain and these organisms not only have an internal organization similar to ours but also exhibit behavior indicative of consciousness. If the internal organization is radically different then we can’t go by behavior alone but need additionally a reasonable theory that explains consciousness in a non-substrate specific way.
The problem here is there is no distinction between behaviors and consciousness as if a electric vehicle is powered the same as a gasoline one.
LikeLiked by 1 person
I know the feeling of watching someone exert effort into an avenue I think isn’t promising. I feel it every time I see someone looking for phenomenal consciousness separate from functionality. But I usually can take a deep breath and admit that they should look. I could be wrong.
On “no obvious technical barriers”, I think it’s a reasonable phrase. We again just have to be careful about its scope, which is under the assumption that one or more of these theories are correct. If so, the architecture and mechanisms they posit can be produced with current technology and paradigms.
On the other hand, maybe something like IIT is actually the correct theory. If so, then there would be serious technical barriers after all. (Although IIT allows for the existence of zombies, which makes it unfalsifiable. It’s not clear how you establish a theory like that is the correct one.)
LikeLike
“There’s no real indication these systems have a world model.” It looks to me that so far, people who designed the current most advanced AI systems have not even tried to make AI with a more or less realistic world model.
I just did this experiment with Google’s Bard.
My question to Bard: “Sometimes AI generated art include people who have 6 or 3 fingers on a hand. Yet it is very rare to see AI generated art with people who have 7 or 2 fingers on a hand. Why AI does that?”
Bard’s answer: “The reason why it is rare to see AI-generated art with people who have 7 or 2 fingers is because these conditions are much less common than having 6 or 3 fingers. As a result, there are fewer images of people with these conditions in the data set, and the AI model is less likely to generate them.”.
LikeLiked by 1 person
That’s hilarious. And fits my usual experience with these things. They’re just optimized to predict which word might evoke less negative feedback, but hampered by the inability to do a sanity check. They have no knowledge of what they’re talking about, or even that there’s anything to have knowledge about.
LikeLiked by 1 person
“If consciousness is functionality, then like all functionality, it can be implemented in multiple ways.”.
I think we may have been through this before but the only functionality we can measure that could usefully compare biological with non-biological would be external behavior. But if we insist on a black box approach then there will never be any way to distinguish a conscious human from an unconscious emulator of a human. As soon as we identify something the human can do that the emulator cannot, we simply redesign the emulator to have the new function.
LikeLiked by 1 person
Actually the paper discusses internal functionality at length in terms of the theories it reviews. So with global workspace, we’d expect to see some workspace dynamics. With higher order thought, processing later in the causal chain tracking earlier processes.
Of course, these theories have historically been validated against external behavior such as verbal report in human subjects. So in some ways, we could see them a proxies for certain external behavior. The chain of evidence always leads back to behavior.
On the emulator, if it can do everything a conscious system can do, then by what right do we say it isn’t conscious? At least without asserting non-functional properties? You ever hear the phrase, “Fake it till you make it”?
LikeLiked by 1 person
I realized the paper tried to deal with internals. So that’s admirable.
The problem with the emulator being conscious is there is no way of determining when we have enumerated the correct and complete set of functions. No matter what set we come up with we could conceivably find another function and still redesign the emulator to accomplish the function. In other words, the choice of functions is arbitrary and at the same time unlimited with potentially many different levels of granularity. It’s like playing football (any kind) where the bounds line can grow and shrink while you’re playing.
I think eventually the criteria will turn out to be much simpler and straightforward and explained with physics. It will involve large scale behaviors of networks. fluid dynamics, mass behavior of charged particles, and electromagnetism. From a functional standpoint, it might be demonstrated in a system with a small subset of human functions but possibly scalable beyond human capabilities.
LikeLiked by 1 person
It’s worth noting that large scale behaviors of networks, mass behavior of charged particles, and electromagnetism play a role in the mainstream theories and overall neuroscience. An action potential depends on the electromagnetic field gradient across the neural membrane. And GWT, HOT, AST, and predictive processing all depend on specific network level effects. They just aren’t the effects you’re looking for.
LikeLiked by 1 person
Sure, those effects are a part of conventional neuroscience. It won’t be a huge surprise when the pieces come together. We’re closer than people think but it has nothing to do with conventional computation.
LikeLiked by 1 person
Ok, WordPress just ate my last reply, similar to what you described. And this is on my site itself, not the Reader. Not good.
LikeLiked by 1 person
Well, apparently my follow up comment shook it loose. 🤪
LikeLiked by 1 person
Thanks for pointing out this comprehensive and interesting article. The article provides an up-to-date account of leading neuroscientific theories of consciousness.
The paper asks whether indicators for consciousness in AI systems can be derived from the theories and then tries to show how these indicators can be implemented in the respective theoretical models.
Of course, I don’t have enough time to go through the details.
I think it is accurate that the authors focus on phenomenal consciousness. To be phenomenally conscious means that there is something that it is like to be the entity in question, that is, something that it is like for the entity itself, and that is the question at issue here. Put another way: For an entity to be conscious is for it to have subjective experiences.
This can be illustrated, for example, by the difference between nociception and pain:
“Nociception is defined functionally, as a state caused by a specific set of stimuli, which triggers a signal through nociceptors, which in turn triggers a set of behaviors. On the other hand, pain is not primarily defined functionally. […] What defines pain, what makes it important, and what separates it from nociception, is the painfulness of pain: the phenomenal property that is specific to that mental state.” (Mathias Michel, 2019)
However, I have some quibbles with this approach.
The reliance on computational functionalism is a constraint that already implies that consciousness in AI is possible in principle. The neuroscientific theories of consciousness considered are designed to explain the emergence of consciousness in humans and other living beings. For this reason, it is questionable whether it is sufficient for the emergence of consciousness in other systems if a system meets the functional or architectural conditions drawn from these theories. It could be that additional conditions have to be fulfilled (e.g. being alive) that are not captured by the theories. As the authors acknowledge, it could be that some non-computational feature of living organisms is necessary for consciousness.
An interesting discussion on this topic is provided in the article
Consciousness beyond the human case
LikeLiked by 1 person
I would argue that pain is itself functional, but its functionality is different from, goes beyond, nociception. Nociception typically facilitates reflexive reactions. But pain facilitates associational learning, as well as stimulating more involved modeling of possibilities for doing things like looking for a place to hide and heal, or learning not to touch that hot oven.
Good point about the theories being aimed at humans and other living systems. That’s a point I often make myself. The question is what else is necessary from those systems for a non-living system to be conscious? Or if it needs to be a living system, what is necessary for the “living” label? Homeostasis? Reproduction? Evolution?
Thanks for the link to that paper! I’ve only read the intro so far, but it looks interesting.
LikeLike
A new life form is not this. However, if it were, then there would be a need for another set of rules to quantify it. Those rules should address needs as well as many other factors. What we are discussing as ‘life’ thankfully isn’t but has the capacity to act alive until power down therefore needs firm control.
LikeLiked by 1 person
Right. I think the life discussion comes up in cases where people say for a system to be conscious requires that it be alive. Definitely no one (or very few) would argue that current systems are alive. The question is what would be required for them to be alive.
LikeLiked by 1 person
Great write up! I think your criticism about phenomenal consciousness is spot on. Actually, I think your criticism points to something fishy about the project, and I have to wonder whether their equivocations were mistakes or intended.
In that introduction, they define consciousness in the strongest possible sense (as you pointed out): ‘Phenomenal consciousness’ is Nagel’s ‘what it is to be like’ in ‘subjective experience’, which is effectively setting themselves up to solve the hard problem. How can they know what’s most likely to be conscious before they determine what is conscious (in the strong sense)?
LikeLiked by 1 person
I meant to say something about your comment on “I know a lot of people are impressed with current large language models…”
I agree. I think the authors are using their own beliefs about AI capabilities to measure what “most” people will believe. It seems to me that even if people do treat AI as if it were conscious (by carrying on long conversations and asking how it’s doing and so on) that isn’t the same thing as actually believing it is conscious. I mean, people are capable of having fantasies with blow up dolls. I think ordinary people will be more inclined to take as conscious (in no particular order or combination) that which is not created by us (biological), has a body, is cute, displays emotion.
LikeLiked by 1 person
Thanks Tina!
On phenomenal consciousness, yeah. One of the things that annoys me is when people say, “We’re going to talk about phenomenal consciousness like Block and Nagel,” cite their papers, but then go on not to really use the concept they talked about. It just leaves a cloud of confusion about what people mean when they use that phrase. People who take it in the strong fashion always feel like there’s been a bait and switch. I don’t blame them.
Interesting point about what people will accept as conscious. The “not created by us” one is interesting, because one of my cautions about what people will accept is that we once saw consciousness in rivers, storms, volcanoes, and all kinds of other natural systems. But we never created any of those. Although I remember reading that there were gods of things like towns, houses, and strongboxes, which are human made. So who knows?
I do think displaying emotion might be a big part of it. If a machine showed fear, anger, grief, or frustration in a believable way, it’d be hard not to empathize with it and consider it a fellow being. And of course people are reluctant to destroy cute robots even knowing that they’re simple mechanisms. A cute robot showing emotions is probably going to have laws crafted to protect it from mistreatment.
LikeLike
It really does feel like a bait and switch. It’s the kind of thing I see all the time in popular articles, but I wouldn’t expect to see it in an academic paper, especially not when so many people are involved. You’d think someone would have called it out. Next time they should run it by you first.
LikeLike
I’m actually not too surprised that the large cast of authors led to inconsistencies. I’ve never been involved in publishing an academic paper, but I have been involved in multi-author efforts, and there’s a tendency to want to give everyone space to say their piece, even if it makes the overall result lumpy and somewhat inconsistent.
Charitably, we could interpret the later points about phenomenal consciousness when discussing global workspace theory an admission that they’re working from a weaker functional conception of phenomenal properties. Less charitably, they should have been upfront about it at the beginning when they say their aim is phenomenal consciousness.
But the author of that early section may not have been onboard with that, or that functional theories like GWT address phenomenality at all. The primary authors may have done their best to smooth over the inconsistencies while keeping everyone happy, or at least mollified.
LikeLiked by 1 person