Human vs. Machine Consciousness | Imperial’s Murray Shanahan

We're way past the cheuring test easily. The point is to show you she's a robot and see if you still think she's conscious. Human consciousness is constrained to a subject object dualism and that AI consciousness might not be. But my philosophy is very very much influenced by by Buddhism. Turing was going to Vickenstein's classes. There was a Vickinsteinian uh influence on Turing when he wrote that paper. There is a fact of the matter whether something is conscious or not. You don't need to go get all philosophical. But but I'm not I'm not getting philosophical. No,

but you are. As soon as you probe using that little extra phrase, is there a fact of the matter? Okay. I'm saying this is not conscious. I am conscious. You are conscious. Of course. Okay. You're willing you're willing to grant that? Of course. What do you think? I'm an idiot. I'm a fellow human language user. How do we tell when AI is conscious? My guest, Imperial's Murray Shanahan, has spent his career investigating this question. And in this interview, we'll lay out a systematic proposal for how to validate machine consciousness. Now, you might think that the

question of whether AIs are conscious is a trivial theoretical fancy with no use in the real world. But this interview will show you why it's one of the most important questions with practical implications for not just how we treat AI systems, but also how we build and align them. But the most unexpected reward of investigating AI consciousness turns out to be what it reveals about our consciousness, about the nature of the human self. Murray's most interesting claim is that LLMs have an important Buddhist lesson to teach all of us, namely that there is no us.

What we've learned to call the self is merely an illusion. My name is Jonathan B. I'm a founding member of Cosmos. We fund research, incubate, and invest in AI startups and believe that philosophy is critical to building technology. If you want to join our ecosystem of philosopher builders, you can find roles we're hiring for, events, grant programs, and other ways to get involved on jonathanb.com/cosmos. Without further ado, Murray Shanahan. One obvious reason people are interested in the question whether AI is conscious is the ethics question. Right? If they are conscious, maybe we need to think

twice about uh turning them off, about saying insulting things to them. What are some of the less obvious reasons why this question matters? Well, you said we maybe we need to think twice about turning them off, but actually maybe we need to think twice about turning them on, right? What's really an issue here is whether they can suffer. And so if so we I really think we maybe we want to hesitate before we build something that's genuinely capable of suffering. I mean there are many dimensions to that question but the question of moral uh I

mean it might not just be moral standing that matters but the even if we build something that perhaps appears to be conscious but we ultimately decide it isn't really to mistreat something which appears conscious is in itself does seem like a bad thing in the same way as it would seem bad to you know torture a doll or something like that and as you probably know Kant K can't K had the view that that animals couldn't experience suffering in the same way that we can, but nevertheless thought it was bad if humans subjected animals to

torture or something like that because it was bad for the humans themselves. And so maybe we'll be in that position with AI as well. I mean, so so here's my answer for why it might be significant for us to answer this question outside of the the ethics question, which is that studying machine consciousness, whatever that means, might help us better understand the idiosyncrasies of our own. Of our own. Oh, for sure. So you you wrote uh a lovely paper called Satori before singularity uh in which you argued that human consciousness is constrained to a subject

object dualism and that AI consciousness might not be. Why is that? Yes. So it's it's quite funny that paper which was published in 2012 sto people have been talking for a long time about super intelligence um and uh you know the idea that we might be able to build AI that is but in some sense superior intellectually to humans um and so I was thinking is there another sense in which we could make or imagine something that was better than us and uh so I was thinking about um uh about the Buddha actually and thinking

about the the idea of people who are enlightened and people who you know perhaps have transcended dualistic thinking. Uh and so I was thinking well perhaps there's another sense in which we could build something that's better than us um in that respect something that that is less hampered by its own ego and um so so it's a very speculative slightly bonkers crazy paper which is why I liked it and why I'm beginning with it because you're talking to a continental philosopher here. Right. Right. Um, so let me give you a quote from that that paper,

right? Or am I going to cringe? Okay, go on. Yeah. The pre-reflective, reflective, post-reflective series is not just one among many paths through the space of possible minds. Rather, the space of possible minds is structured in such a way that this is the only path through it. What are these three different stages that you laid out? Yeah. Okay. So, I I I I um I don't agree with that uh anymore. That was more than 10 years ago. So I don't at all think it's the only path through the space of possible minds. Can you explain

what these three what these three? Okay. Absolutely. So no there so that so they I do take seriously as as different you know ways of being. There is the pre-reflective mind which is the the mind of of um say the na a naive uh child or or a simple straightforward ordinary person. Um they haven't really thought about philosophical problems. I haven't had these sorts of thoughts that many of us have start to have when we're children of you know, you know, why do I exist? You know, what is how do I know that my parents

are conscious? You know, some people start to have those thoughts when they're children very often. Um, as I certainly did and I think you probably did as well by Unfortunately. Yes. Unfortunately, indeed. And I do think of it as a kind of affliction because they can you can really be bothered by these thoughts. They can be both exciting and disturbing at the same time. The one that's troubled me more than anything else really is the kind of the mind body problem. How is it that that I the myself and my my experiences my consciousness how

does that relate to the world? So how can I reconcile these first personal experiences with a universe which is just which is matter and energy and so how can I reconcile the subjective with the objective? I think you can spend your entire life in that reflective stage and worried about that um and those problems especially if you're become a professional philosopher then you can actually make a career out of that reflective stage. Um but my I tend to think um that there is a stage beyond that which I call the post-reflective stage. But I think

the post-reflective stage is where we transcend that all of that dualistic thinking and it's really really difficult but we somehow come to see our our inner life and the outer world the subjective experience and the objective physical reality uh as not two different metaphysical categories but somehow somehow the same thing. To summarize for our audience, um these things are related. What what you said about the Buddha and the lack of ego centricity and the post-reflective state that that does not have this subject object dualism. Absolutely. Um and your interesting claim in this paper is that

it's something about the human hardware, the fact that we're embodied in one body that cannot be copied and mult, you know, multiplied and and paused. Yes. that that that hardware is what gives us this software limitation. And what I found most interesting about applying this to AI is you cited I believe uh another thought experiment about what if we had creatures whose bodies are just fusion and fision all the time. Your bodies are going in and out of ex existence and you said in the paper isn't that close to what computers are or aren't computers

like the closest thing? Absolutely. And so it's because of the nature of a software program that can be copied, halted, multiplied, deleted, uh, uh, recreated that you're saying that is the reason why you you speculate uh, uh, AI might be post-reflective. Did I did I get that right? 100% right. And in fact, I'm actually developing a paper right now of those that literally are revisiting those exact considerations about the difference between software and uh and you know human and biological of of an of an individual and the fact that that that software can be taken

apart and reassembled and copied and manipulated in all kinds of ways. And it's interesting that of course when I wrote that paper large language models didn't exist. Now we have large language models. You can ask a large language model to sort of roleplay a conscious artificial intelligence, you know, what do you mean by the word I when you're using it? And then if it comes up with a a slightly philosophically dodgy answer, you can probe that and push it into into interesting territory. And so um so if we think of what might I refer to

when a large language model uses the word I in a conversation, now there are lots of different candidates, but one of the one of the candidates is that it's confined to the to the context of that particular conversation. the same model like Gemini or Claude or something might be uh obviously having lots of other conversations with other uh individuals but they're in a sense separate eyes, separate selves. It's really just the text that you've got which is the transcript of the conversation in a sense that captures the the little miniature eye that was created in

the context of the conversation. And of course it's trivial to chop that up, copy it, you know, blend it with other conversations. And and so if that is a you think of that as a kind of little mini self that sparks into existence very fleetingly and and and flickers into existence every time you're interacting with it, but otherwise is dormant, then um uh then you've got a you've got a very strange conception of self. And it's possible because of the nature of the underlying substrate you're dealing with really, which is very different from us. we

have a single body and uh and so of course you know we we we just are not used to the idea of tearing our bodies apart and reassembling them and copying them and and so on. What's really interesting with this concept of roleplay is that it's almost a more enlightened being role-playing a less enlightened being which in some sense is quite Buddhist. So so I I've I've been fascinated with Buddhist philosophies and there's a separation of conventional truths and ultimate truths because because the enlightened bodhic sava still needs to operate in the world some way.

So the bodhic sava he or she still needs to recognize that you know this is this is my hand conventionally even though I know that's not the ultimate truth and that's what kind of the role play is like. Um now I I believe in that same roleplay paper that you that you've just published you gave an even more interesting depiction of what selfhood of AI could be with the example of the 20 the 20 questions game. So explain that. Yeah. So in the 20 questions game, the idea is to uh is that uh the one

player thinks of an object and then the other player has to guess what that object is by asking a series of questions or yes or no questions. Um so if you're playing with this with a large language model and then it says, "Oh, I'm thinking of an object." And then you say, "Oh, okay. Is it large or small?" And say, "Well, it's small. Is it alive or is it inanimate?" And say, "Oh, well, it's alive." And then say, "Well, okay, I'll give up. What is it?" And then it'll say, "Uh, oh, it was a mouse."

But then you might just resample and exactly the same point and it might say, "Oh, it's a cat." Well, hang on a minute. If either you thought either you're cheating, you know, or what's going on here? Did you actually think of something in the first place and answer honestly all the questions going along? At each point in the conversation with most UIs, user interfaces for these things, you can actually resample and try another one. You know, if you didn't like the first answer, you can try another one and go off from there. So as you

can imagine that induces this whole tree of of possible conversations you know. So what the language model actually actually generates is a probability distribution over all the possible words and then what actually comes out is uh you sample from that probability distribution. It's absolutely inherent in the way large language models are are are built that it's not going to commit at the beginning of the conversation to to exactly what the object uh is. So at the beginning of the conversation, all of these possibilities still exist and they still continue to exist all along. So even

though it should have committed to a particular object, it never really has. Right? And so getting back to the roleplay thing. So the idea with uh with how this relates to role play is that um is that we can think of um the LLM the large language model as playing the role of a particular character, right? So the default character is a sort of helpful assistant, but we can always steer it into playing, you know, a favorite character from history or or or a kind of an or or a romantic partner or an or an

angry friend or all kinds of things. You can easily just steer the conversation into those different places. And at any point in the conversation, there always remains as like in the 20 questions, it's never really committed to one role. there's always a vast possibility of different roles that it could carry on playing from there. So that further undermines this ordinary everyday notion of selfhood and identity because this is particularly particularly weird because although we of course to some extent also play roles in our lives but there is a kind of ground. So to summarize for

our audience of the 20 questions game, the point you're trying to make is when I when we play that game, you're thinking about a mouse or a cat. And so you're answering their yes or no as a heat-seeking missile to to that truth of that answer, right? Whereas your point is the LLM is not thinking about an answer. It's just stochastically creating whatever is plausible given the commitments it's already made in the previous tree. And so when you rewind it, it can stochastically give another branch of the tree. that that's what you're trying to say.

And so perhaps an analogy here is in quantum mechanics where uh the Copenhagen interpretation believes that there there's there's there's multiple worlds, there's many worlds. And what you're trying to highlight, I think, is that maybe that's what AI selfhood is like, is that there's a plethora, there's a multiplicity of selves that exists all in one uh time or or slice of time. That's what you're trying to highlight, right? That's exactly what I'm trying to highlight. So, so in the case of the 20 questions, it's like all the possible answers that are consistent with all the

answers so far. So, you know, cat, uh, mouse, dog, you know, and so on, they all as it were still exist in superp position and it's only then when you it's when you actually ask it what the thing is that you collapse as it were the the distribution and and it has to fix on one of them. So similarly the the idea is that um the all the many the roles that that that are consistent with the conversation so far um that that the LLM has been playing they still exist all in superp position. So

uh so yes it's like there's a multiverse of possible conversations and possible roles that can be can be played right in a way that we are having this conversation is not like that at all right yeah well you say that it's not like that at all but of course this is the point is that is that I think our intuition says that it's not like that but the whole analogy then with Buddhism I think is is saying is saying well isn't it interesting that these large language models if we if we think about them in

these terms they uh as it were they bring to the four these sort of slightly challenging conceptions of selfhood which then we can reflect back on our own onto ourselves. Right. Right. I see. So that's so I find that really really uh provocative and interesting you know avenue of thought and this is what I meant when I said that my answer would be investigating machine consciousness can be a mirror for us to better understand 100% agree with that. Yeah. Yeah. And I think that's particularly um uh interesting at the moment with the advent of large

language models. And it was uh it was much more speculative and and wild and crazy back in 2012 when I published that that paper that you originally mentioned. Now our our audience members I think should be forgiven if at this point of the conversation they're like oh this is just philosophical mumble jumb but what's the real payoff of this kind of view of uh of AI selfhood? And I think this is the payoff to so let me read you a quote from your essay. Untainted by metaphysical egoentricity, the motives of a post-reflective AI plus would

be unlikely to resemble those of any anthropocentric stereotype motivated to procreate or self-modify. If the post-reflective AI plus were in fact the only possible AI plus and if it produced no peers or successors then the singularity would be forstalled or more precisely the intelligence explosion that is central to imagined singularity scenarios would be capped. There would be AI plus but no further progression to AI++. So this is the the payoff okay of this self. By the way, just to read just to read now, this thing was written in 2012. And I, you know, I I'm

not sure I really believe all of that at all. If only it were that easy to um uh to ensure that uh uh that some kind of intelligence exploion didn't happen. So, as a just to reiterate, this is a slightly mad essay that you've that you've picked on. I'm I I'm I'm still uh you know, I'm actually more proud of it in some ways now than I was then, but it's still not to be taken entirely serious. So, let me just um reiterate that. But there is a very serious idea wi-i within here which is

that we assume the terminator scenario these this AI is going to want to defend itself. It's going to want to procreate. But your point is if what we talked about about the the egocentric less view of AI is right that the AI is going to be like the Buddha then maybe those fundamental assumptions are wrong and maybe this is the this is the title of the paper there will be Satatori before singularity. There's an maybe there is an an inherent cap yes on a superior intelligence willingness to advance that that's the thought right I guess

that's the thought yes but I um you know now now allow me to argue against myself please so so you know the argument of the people who are concerned with existential risk um so they will point to what they call convergent instrumental goals and you've alluded to some just now so the uh uh so the the idea is that whatever goal you give to the uh to to your AI for example manufacturing paper clips to use Eleazowski's and Nick Bostonramm's famous example then in pursuit of that goal if it's very very very very powerful then

there will always be these instrumental goals such as accumulating resources protecting itself so I don't necessarily buy my own argument in that paper although although I like the ideas I I I love the idea again I I do think I may be a bigger fan of your paper than you are at this But that was going to be my objection. The paper clip example. There might be instrumental goal like like in fact procreation and defending yourself is probably instrumental to most goals we would we would want the AI to do. However, right there's conversion to

right it removes one let's call it the Terminator scenario or something that the AI itself has some kind of selfobsessed uh ambition. Right. And so absolutely if that is true. Um that is true. Yes. So there's two more critiques of your original argument I had which I think it sounds like you're probably going to agree with. The first one is that they are still roleplaying selfhood. Yes. And and selfhood at least for the information they've been fed procreation and self-defense is critical to that. Right. So so even though maybe their core operating system doesn't have

that kind of egoentrism they are roleplaying it. Yeah. Um, and the second issue that I had w with with with your argument is it assumes the hardware determines the software. But even in the human case, that's not true. Even our human embodiness in the case of the Buddha and enlightened masters today, they were able to transcend. Yes. This this dual this uh this dualistic view, right? So, so how do you respond to those two issues? Okay. Well, of course, when that paper was written then then we didn't have large language models and the whole roleplay

scenario wasn't uh you know wasn't something that I was thinking about at all. I mean I was very much was appealing to that notion of ego in that in the first paper and today we see it being sort of manifest perhaps in a kind of role playinging self. I so I totally totally agree with you. Um uh so it is something a little different and actually let me let me use that as an as an entree to this other scenario which I wanted to to paint. So, so one way in which um uh in which

we could imagine a positive outcome is through hyperstition. If there's some fictional uh story um of uh of a say I don't know something some some something bad like let's say like a murder or something and then somebody in reality has a copycat of that of that murder that would be an example of hypers that's not a very pleasant example but it's basically where or or any way in which um uh life imitates art say so so um so that's that's hypian so it's where things that are fictional then because people then imitate the fiction

based on the fiction then they can become reality. Um and one way in which that can can happen in the contemporary an uh world of large language models is through science fiction characters science fiction AI characters. Now of course our large language models they were trained on a vast repertoire of of uh stories including scripts of science fiction movies, science fiction stories and novels and so on. many many AI characters uh exist in those um uh in those stories. So when a contemporary large language model starts to roleplay an AI system, which it's often going

to do because it's uh it it knows that it's an AI system usually. Then then what's it going to roleplay? It's going to roleplay all these architect well the things exactly or some mashup of all these different things. So it's going to pick its ideas from stuff that's in its training set and all of these, you know, literary and fictional and and artistic examples in its training set. Now we are sort of a little bit in a position to maybe try and steer this whole process a little bit because the more good stories we have

and in a sense my own paper dare I say it is a good story of a of an imagined science fiction and then the more of those that are around then the more possibility there is of the uh of the the future AI roleplaying these good role models that are that are out there. So perhaps through the mechanism of hyperstition, we can actually um uh you know make it more likely that the AIs of the future will have good role models when they're role playing, right? That makes sense. I have a friend who is um

starting a company um a film company to make more positive sci-fi uh views for humans, right? But I think he'll be very delighted to learn that you think that there's also potentially a moral benefit for the machines as well as as a model to learn. Indeed. If those if those uh well if the movies themselves or the or the the screen plays find their way into the training sets then then that's all part of a melting pot of possible uh roles that the thing can play. What about the second push back I had which had

to do with the connection between the hardware and the software because in the very example that you gave the Buddha it was an example of the software transcending the hardware. Yeah. Yeah. So you're alluding I think particularly to the fact that um that we have bodies that are confined. There are plenty of biological organisms that are are much less uh that are you know that are that of course they're embodied in a sense but they but they're if you think of things like microisal networks or or or or things things like that then then their

bodies are much less confined in space and so on. But the problem for humans is because our bodies are confined to one, you know, little lump of of of matter. And so we we identify with this lump of matter and uh and and that means that our conception of selfhood, I think, is very much informed by the fact that this lump of matter stays kind of the same. Now, of course, there are um uh uh you know, all kinds of challenges to the to to the idea of identity of personal identity. Chipsius, right? Chip thesis,

which of course we can apply to the human body because we know that all of ourselves are replaced over our lifetime and our memories grow and our personalities change and Buddhist thinking actually, you know, takes those challenges and and indeed uses them to challenge our notions of selfhood as well. But nevertheless, it's still there's still a lot to overcome there because we really do have a notion of selfhood, I think, which is which is tied down to to this and that that we we think it has inherent existence because of that. Right. So, so, so

you're saying you're kind of agreeing with the critique that it is possible to transcend. There's just a heavy heavy inertia. Yeah. He heavy inertia. Um, but if you flip that, you then aren't saying that an AI built on, let's call it non-dual hardware necessarily will conceive of itself as non-dual. It will it's just more conducive. That's the point you're making. Well, I mean, there's how it might conceive itself, you know, some future AI when that even makes sense, right? Uh, and then there's the AI systems that are around today, which can roleplay things that are

doing that and can be lessons for us because we can actually they they can actually be philosophically provocative for us even today. So, they might not conform to their own self-descriptions, but their own self-d you know, if they describe themselves as conscious, maybe that's right, maybe that's wrong. is philosophically provocative and we can we can apply that those provocative conceptions back to ourselves. Yeah. You mentioned the uh the ship of thesis and those are the exactly the type of um dialectical questions a Buddhist master would would ask the student with. Yeah. Yeah. Yeah. Are you

are you are you your shoe? Are you your your your feet? Are you your hands? Are you your chest? And perhaps we should say what they say. So you you brought up the ship of thesis. Is that the ship of thesis that that um is gradually take you know over the years the the the the mast is replaced and then the next mast is replaced and then a bit of the deck's replaced and part of the hull is replaced and this happens over many many years until eventually all every last bit of wood in the

uh of the original ship is gone and you have a have a completely new ship. Well, do you so is it still the same ship or is it not? And if it's not, at what point did it change from, you know, did it become not the ship it originally was? So that's the the the problem of identity. We we might actually sort of, you know, sort of be very frustrated and think, well, what is the right answer here? And people might come up with all kinds of theories about identity. And but, you know, if we

think about it honestly, we we'd have to say, well, it's just it's just up to us. We just decide what we think is when it's the same ship and when it's not the same ship. It's a entirely matter of convention to say that this is the same ship and that's not the same ship. There's no metaphysical fact to the matter about whether it's the still the ship of thesis or not. Now with with with um ordinary everyday objects, it's easy to recognize that it's just a matter of convention and consensus that there's no such underlying

essence of the ship of thesis which suddenly switches off or anything like that. But when it comes to our own selves, it's much harder to uh to uh to apply that thought. And I think this I think I was wrong before. I think this is the true payoff of of the Satori paper because by examining an LLM that appears to be a human self, it's much easier to see for example 20 the 20 questions uh uh example you gave, how it is this kind of Buddhist self in flux that's going and you look at the

hardware it's built on. It could be copied, it could be deleted, could be restarted. By seeing how such a a a human appearing self is actually this non-self, yes, we can apply that to ourselves. That's precisely. And this is what the Buddhist master would want us to do. So maybe the Buddhist master in the 21st century can use the LLM as an example. That's exactly what I think um we can do and it's something I'm very very interested in pursuing in fact. Um right. So I've been talking recently to some to some uh to some

Buddhists. Bob Thurman uh who is a very well-known I I just emailed him last week. In fact, we are we we were uh co-founding a foundation together called the Eternity Foundation to use AI to translate uh a lot of the the lost texts from from the Tibetans. So, well, how interesting. Well, we spoke last week and we're speaking again tomorrow. um and um uh and this is exactly the kind of project that you just described which um which I'm going to be talking about with him and I have a a paper which is in the

pipeline which describes exactly exactly this. So for me that is the payoff at the moment. Yeah. And so in other words, you agree with the core insight of Buddhist philosophy that this is a ship of thesis because because you know a westerner let's say a cartisian right kito like they just put a stake in the ground and say no no no LLMs are completely different from me. I have a soul that lasts say K above time even you you take the Buddhist stance 100% I do. Yeah. Yeah. I'm deeply influenced by by Buddhism and always

and always have been. All of my philosophy is very very much influenced by by Buddhism. Well, um I know you're dying to talk about Wickenstein. Yes. So, let me give you uh a quote about why you think it's so important for us to talk about Wikenstein before we investigate or when we investigate AI consciousness. Wikinstein's phrase, nothing is hidden, is to say, nothing is metaphysically hidden. My experiences are just as much out there as in here. Consciousness is only private in the unsterious sense that a ball can be hidden under a magician's cup. In both

cases, a more detailed inquiry would reveal all. What does Wickstein have to teach us about LLM consciousness? Well, I think Vickinstein has a lot to teach us about consciousness in general. Um uh so so let's set aside AI and LLM consciousness for now because in my mind um much of the discuss contemporary discussion about consciousness is mired in dualistic thinking and and so let's take an example so u very famously David charas introduces the distinction between the so-called hard and easy problems of consciousness. Um so the easy problem of consciousness is trying to understand the

relevant the all the kind of cognitive operations that that that we associate with consciousness. So such as the ability uh to produce verbal reports of uh our experiences to bring to bear uh memory and and so on on our decision- making to integrate all of these things. So understanding those which is a huge scientific challenge um that he characterizes as the easy problem. Now the hard problem is to uh is to try to understand how is it that mere physical matter as it were can give rise to to the to our inner life at all

to the to to the fact that we experience things at all. How does that come from mere physical matter? Because this it seems as if whatever explanation we provide for all of those cognitive aspects is not going to account for this magical light that's on inside me. And uh so that's sort of the hard problem and you earlier on you alluded to dayart the demon might be tricking me into thinking that you're out there. All of these sensations might be not from a world but they all might come from a from this demon. All I

really really really know for certain is that uh is that I think I am because if even as I'm doubting that the very act of doubt is only possible because I am thinking to doubt is to think is to exist as a as a subject. So Daycart reduces everything and so pairs away all of the physical world and leaves us just with just with the the the the experiencing ego. And so in doing that he's kind of he's carved the reality in two by by saying that there's this stuff out there and there's this thing

in here which is me and my my my ego. So, so he creates this dualistic picture and this is what Dave Charmer's when he um talks about the hard problem is is alluding to now. Um so what does Vickenstein teach us to get back to the original question Vickinstein's uh procedures philosophical procedures and and and tricks and therapeutic methods right enable us to overcome that dualistic thinking much as Buddhist thinkers do such as Nagajuna. Can you can you tell us a bit more about these therapeutic techniques or or how he resolves this dualistic? Yes. Well,

so the first so the very first step is to is to understand a little bit about the nature of language. So he wants us to to as he would say it you know let's not ask what words mean or what a sentence means but let's ask how the words are used and how the sentences are used in everyday human life and everyday human affairs. And the important thing is there is that that that the context in which we use words is always uh you know the context is our human affairs other people the things we

want to do what we want other people to do for us or with us or what we want to do together. So that's the essence of of language is that it's is that it's it's something that we um use to oil the wheels of human commerce. Now that's all very well with ordinary words like chair and shoe and so on. But when it comes to really difficult words like self and consciousness and belief and truth and um knowledge and uh beauty, you know, so these are all very difficult philosophical words and it's much much harder

to um to to so if we want to ask what those words mean a mind, you know, it's it does it seems a bit inadequate to to actually say, well, let's do how are those words used? But that is the strategy. So his strategy is to is to say well let's let's really look into how these words are used. And if we have a philosophical problem that or something that bugs us philosophically like the mindbody problem or this or this kind of dualistic thinking the hard problem then maybe the way to tackle it is to

is to uh investigate how the relevant words are used. And by doing that you can often ultimately dissolve the sense that there is a philosophical problem there in the first place. And the really interesting case to do this with is with the case of consciousness and of subjective experience. And that that's where the whole private language remarks which are central to the philosophical investigation. So they do that they apply this strategy to subjective experience and it's really the deepest thing. So when I hear you say, let's stick stick on the human case. There's nothing in

here that's not out here. Yeah. Is this a good way to understand it, which is let's say a tragedy befalls my family. And you know, you can measure a lot of things about my heart rate, my speech, you know, my how my brain is firing. And you're saying there's nothing more that my subjective experience of witnessing that tragedy would add. Well, so so so as soon as I hear sentences like there's nothing more than or there's there is something is no more than something is just then then then we know that we're going wrong in

a different kind of way, right? Because that in itself is a metaphysical claim is a reductive metaphysical claim. So so people often misread Vickenstein as a kind of behaviorist and and is saying oh well consciousness just is behavior. Say so of course he's not saying anything like that. The really great punchline in the philosophical investigations is where somebody accuses him of saying that so so we're thinking about sensation say a sensation of pain and uh and somebody accuses him of saying well you're you're saying that the sensation of of the the sensation itself is a

nothing and Vickenstein comes back saying no I'm that's not what I'm saying um it it's not a nothing but it's not a something either the point is that a nothing would serve as well as a something about which nothing can be said. And that's you know that to my mind is as is as great as any line any Zen Buddhist has ever uttered because it uh um it it it the point is not to establish a a metaphysical position of its own but to enable people to transcend the metaphysical positions that they're tempted by. Okay.

But but help me let's let's use the tragedy case when Wigenstein says nothing is hidden about my experience of the suffering of like let's say all my family dies what does he mean by that what is he trying to say there well he's saying he's trying to say that nothing is metaphysically hidden I mean of course you're quite right to a light on you know the most difficult examples of this so where we have like deeply felt things that we feel so we feel that there's there's a there's a private dimension to this. Right now,

of course, there's a private dimension in the same sense that a ball could be hidden underneath a magician's cup in the fact that I might, you know, I might uh um uh have all kinds of thoughts and words and so on that are going on inside my head that are uh that I don't articulate. Um uh but they're they're not metaphysically hidden. They're just I you know, I don't express them. There's all kinds of things going on inside my brain and inside my body that might be hidden in practice because because you can't see inside

my brain, but they're not metaphysically hidden either. I can I can probe inside your brain. I can see things lighting up. So, all of that contributes to what we might to to the public to what is public. I I know you're going to be resistant to identity claim, but when I hear nothing is metaphysically hidden, I hear there's no epistemic barrier for me to access whatever this is. And so the only barrier right now is a barrier of technology essentially something like that, right? Like oh well okay now that that's is a great point but

there's a very big difference important difference uh here right which is which is um which which is really crucial which is that I'm not you right? So, so, so there is a difference between being you and being me and the same and and and and that's very different, right? So, so, so your feelings are yours and mine are mine and I'm not you and you're not me and so that that is a barrier that can't be overcome. I mean, it's but it's not a metaphysically mysterious barrier. It's no more metaphysically mysterious than the fact that

there is not the same as here. So we can easily confuse indexicality so that the so that the fact that I am in this position here and I am me we can confuse that with a metaphysical um uh division. So so let's use more examples to help our audience understand the position Nagel's bat. So famously yeah so famously Nagel says you can learn everything about a bat right like the things we we know right now and the things that we can know future with better technology how it flaps its wings. Yes. But you're not gonna

know what it really feels like to be a bat. Yes. Right. Absolutely. What would Stein say to that? Well, I've often um thought that there's a little bit of a linguistic trick going on there. There's there are different senses of no in play to conjure up this this this dualistic um thought. Um and so I think that's happening that happens here as well. So when so when Nagel says we can never know really know what it's like to be a bat, all he's saying is that we are not bats and we can never we can

never be bats, right? We we're not bats. That's he's not adding anything more with the word no. So so it's interesting that that in one sense it sounds mysterious. In the other sense when you remove the word no, it doesn't sound mysterious. But he but but when he uses that when he adds that word no, that is not adding anything uh more. It's just creating a puzzle as I see it. But if you had a fact sheet about everything you could possibly measure externally about a bat, what is the relationship between that and being a

bat? Well, that's what is the relationship between that and being obviously there's obviously it's it's it's very different from from from being here than to being there. I mean, that's it's the same difference as that. So, let's apply this to to first human consciousness, right? Because there's another thought experiment about the zombie example, which is everyone here in this room, I can't know for a fact, yeah, that you guys aren't black and dead inside, right? You're just you're you're just like what I feel like when I go to sleep, right? Um, how how would you

respond to to to that kind of thought experiment? So, Vickenstein does say so he says, well, just just try imagining that the people around you are automter. imagine that your, you know, that your friend or your your partner just to try imagining that they really are a zombie. You really probably can't do it really. He says, I'm not of the opinion that that uh that that you have a soul. I I rather I I just treat you I take the attitude towards you that I take towards a soul. I just treat you as a as

a as a being with a soul. In other words, it kind of goes deeper than an opinion. So, we're going to come back to AI here, right? So clearly we're coming back to air because this is where things get difficult, right? So in iroot, so Will Smith, the Will Smith character in Iroot, the kind of cop who's very skeptical about robotics and a robots and he, you know, he really doesn't like robots at all. And then then the robot Sunny, it's all about the robot Sunny. And of course, you know, uh the point is that

the robot Sunny is has much more of a human, you know, like nature. and and eventually Will Smith comes around to seeing him as a fellow conscious creature. Um, and why does that happen? Uh, well, it happens because of of the time they spend together and they spend, you know, they they spend more and more time together and eventually Will Smith can't help himself but to see this as a fellow conscious being and that's on the basis of their of their, you know, the extended encounter they have with each other. Right. Um, so yeah, what

what I'm hearing you say is kind of a descriptive claim, which is you can't help, I can't help but treat you as a sentient being. It says something like that. And obviously, this gets challenging. Yes. Already, but will be more so with AI, but already I'll give you the thought experiment, which is I'm a big fan of video games. Some characters are non-playable characters, NPCs. Some have a real human behind them. Um, and I need to I need to figure out which one is which. Yes. Yes. In order to know like an NPC, of course,

I can just abuse them. I can humiliate them. The question shouldn't be totally shouldn't be should I like will I treat them as moral agents. The question is the normative one like like ought they be normative agents, right? So, so yeah. Well, so I think so so so um so this relates to the kind of question of you know couldn't we be fooled, right? Couldn't we couldn't we be fooled fooled into thinking that something uh is uh uh you know has moral status and is conscious and so on when it when it when it really

doesn't. Right now what do we mean though by it really doesn't? What we mean by it really doesn't in all of these cases that we can imagine is that if we investigate further then then our then the scales will fall for from our eyes and we'll realize that we were wrong. Right. So just in so so of course like the magician's cup. Exactly like the magician's cup. So that that that sentence that you quoted I said you know further investigation will reveal all so I think that that's you know that is the so in the

case of the NPCs well you know as things are today of course you can you could actually kind of make you could do an investigate in the case of of of AI just as we as with certain you know animals today we need to do more and more we need to find out more and more and more so our um our attitudes towards them will become shaped by what we find out. So of course we might be wrong, right? So just just in this way that Will Smith might have been wrong and and and f

and and further you know learning more about the way say the way sunny was built might might have have led him to change his attitude towards it. So just so I have you on the record saying this because because on the book it sounded like you were stating the opposite case. There is a fact of the matter whether something is conscious or not in the sense that whether let's just let's reduce it down to suffering whether it can suffer or not. There is a fact of the matter that you I think can suffer. Yeah, there

is a fact my my iPad probably can suffer. Like there is a fact of whether there is a light inside. Well, where are you going with that? Why are you why are you pursuing why are you asking that question? Because you see the concern is that that that you're wanting to ask that question itself betrays some philosophical confusion. You see, right? Well, help me help me out here. I'm I'm I'm teasing. I'm teasing, of course. But but but you know, but I'm reluctant. Of course, you don't want to descend from that because that would be

that would be just dafted, right? That'd be ridiculous. But but then on the other hand then it depends where you're going with that because if you ascent to that then you can very soon be led down various philosophical garden paths. If somebody you know walks up to me and says well are you in pain or not? I say I'm in pain you know and and then if they then said said well is that a fact of the matter or you know or is that what you know I just told you I'm in pain. You know,

you don't need to go get all philosophical. But but I'm not I'm not getting philosophical. Like you are when you you are as soon as you say as soon as you probe using that little extra phrase, is there a fact of the matter? Okay, here's when I ask the question, is this iPad consciousness? I think I'm asking the same type of question as what is the color of Murray's pants? Right? in the sense that I'm using consciousness as a purely like ordinary language a kind of ordinary non- philosophical understanding in the sense that there is

a that there is a fact of the matter in the uh uh in the ordinary sense in the color of my trousers um then there is a fact of the matter let's say I'm happy to say that in in the in the case of consciousness yeah I see I see so and in fact um in fact because we see we do use these terms all the time the intuition that there is a fact matter is far stronger in the case of pain and things like that than in the color of my trousers actually. So, so

let let me get this clear. You're totally fine with me saying again if I'm just using consciousness as you know well when I sleep I'm not that conscious when I'm awake I'm conscious before you say I'm totally fine with saying I say so if you want to make a philosophical claim right I'm not going to agree to because I don't because philosophical claims are what one wants to get away from in all of this totally the post-reflective state yeah but again let's say just with the non-filosophical claim in the same way I'm saying your your

pants are gray I'm saying this is not conscious I am conscious you are conscious. Of course. Okay. You're willing you're willing to grant. Of course. What do you think? I'm an idiot. I'm a fellow human language user of course. Right. Right. But well, your book made it sound your book made it sound like the empirical side is all there is. Which is a metaphysical claim, of course. So all all that you're saying, let me try to summarize, is you just want to get get out of philosophy. You're willing to say in ordinary language, not conscious

conscious. And let's figure out empirically why we think I am conscious. and let's try to see if those same same things applies to other objects or agents. Yes. Yes. That's a that's a great research program. You might ask the question, what gives rise to the sorts of behavior that um that we label as conscious, right? and and how is it that that the brain can give rise to that kind that is a deeply deeply difficult question philos um scientifically scientifically let me um ask you the question here as an ordinary person you're willing to say

iPad not conscious Jonathan conscious why do you think nothing in principle is hidden by investigating yeah but the but the the question is is is wrong the question you is one you should be asking self, right? Because why do I think in principle something is hidden? Right? Because because because the the the the the the statements that you've just said, they serve a therapeutic process to in when applied to somebody who is kind of confused in this way and thinking that there is something that's hidden, right? So it's not so you're you're throwing them at

me as if as if it was some position that I need to defend, right? But that's the wrong that's back to front. but rather these are are kind of you know rhetorical and therapeutic strategies for dealing with uh with with confusions. So let's what do you think is hidden and then that's where one has to try and find ways to to to undermine whatever conceptual knots there are there untie whatever conceptual knots there are there that may make you think that there's something metaphysically hidden. I see. I'm glad we're at least zeroing in I think

where the disagreement is. And the disagreement is who has the burden who who has the burden to answer the question. Is it you that has the burden to answer the question why in principle nothing is hidden or is it me that has the burden to answer why I think something in principle is hidden? At the very least you've given me a Zen Cohen for me for me to think about. Um unfortunately we we we have to move on because there's so much else we got to cover. Okay. But the next topic is related to this

and uh it's about the movie X Machina which you were a scientific adviser to and you coined the term uh the Garland test. Tell us about that and why it's significant. Exmachino was directed by Alex Garland and credits my book embodiment in the inner life with having you know having helped him to write the script having inspired him a little bit on the script uh which is wonderful in the final film uh and a particular scene and in this scene uh Ava the robot um uh is uh is being discussed and we've got Caleb the

uh the the the sort of the programmer guy who's been brought into the compound this billionaire Nathan Nathan who's built Ava the robot and Caleb is trying to work out exactly what he's there to do and he says and he he learns about Ava and uh and then he says, "Ah, okay. I'm here to conduct a Turing test." And then uh and then uh Nathan says, "Oh, no." He says, "We're way past that. Ava could pass the cheuring test easily. The point is to show you she's a robot and see if you still think she's

conscious." So now that's a really really great line. And when I read that line in the script, I wrote I wrote spoton. So Alex Garland explicitly uh distinguishes his test from Cheurings by saying the point is to show you she's a robot, right? So you straight away you know there it's not hidden from you. I mean there aren't two entities that you're discriminating between, but you you see right away that she's a robot. It's testing for a different thing as well. It's testing for intelligence. It's not testing for intelligence or whether it can think, but

it's testing for consciousness, right? Um, and so that's the Garland. So I call that I call that the Garland test because Alex Garland basically kind of invented it, I think, in that moment in the film. And I imagine given our discussion on Wikenstein, you like this test as well as the Turing test because it's Wikensteinian in in the following sense. It turns the metaphysical question of intelligence and consciousness to one about convention, right? like in in the sense that the touring test it's not about you know let's figure out whether this thing is actually thinking

let's figure out if a regular human would think conventionally would use normal language to describe if it's thinking in the in the uh Garland test the question isn't you know let's poke into Ava's brain and and figure out if there's a consciousness somewhere hidden it's whether a person despite knowing she's a robot will still conventionally think that uh uh that that she is conscious this is very Wikensteinian. Yeah, it is. I think it is very Wickinsteinian. Um uh as is the Turing test. Very Wickinsteinian. And of course, Turing and Vickenstein uh knew each other. Turing

was going to Vickenstein's classes. And I and I Oh, wow. Uh and I, you know, and I I've always thought that there was a Vickinsteinian uh influence on Turing when he wrote that paper. I see. I see. So, so many people interpret Turing, I did certainly before this conversation, as a behavioralist, but that might not not be true, right? He might be this because because he um his move right at the beginning of uh of computing machinery and intelligence the paper in question his move is to say uh does a machine think? Well, that's a

really difficult question to answer for this that and the other reason you know and you know blah blah blah. Let's replace it by the following question. So he doesn't he he replaces the question can a machine think with a different one. He doesn't reduce it to the other one. That would be the behavioralist position. That would be a behavioral I absolutely exactly exactly so let me quote to you um another piece from your writing. If suitably embodied as a humanoid robot say there's no reason to suppose that such a system could not interact with us

in our environment in such a way that our attitudes towards it would be the attitude we take towards our peers and equals. We should like to situate consciousness as we know it within the larger picture of consciousness as it could be. And the possibility of artificial consciousness of man-made artifacts with an inner life is implicit here. The intuition behind the quote I just read as well as the Garland test and the Turing test is that in some sense the true test for consciousness is communal consensus in some or like like conventional reactions. But but that's

a bit different from what you suggested prior which is an empirical investigation into let let's say the neuronal structure. Those are two different ways right? Oh but you could do both of those things. So who knows how you know our use of the word consciousness and all the very many words associated with it is going to change and evolve over time in our using the word um we may bring in all kinds of things. Right? So in deciding whether we think octopuses um are sentient and can suffer and deserve our uh our moral concern, right?

Then then we look at their behavior. We also look at the scientific evidence for how they're constructed neuronally and and so on. So all all of that is grist mill of of of uh you know changing our consensual conventional way of talking about them and all of it right in but but all of it not is public. It's not there's nothing metaphysically hidden about what's inside the octopus's brain. It might be practically hidden for temporarily, but it's not metaphysically hidden. So, what I'm hearing from you is there are two independent criterion, right? One is the

the convention of the community and the other is the empirical foundations, but they're not conflicting in the sense that the conventions might change. And so the the empirical criterion is simply to to answer are these things conscious in the way that we are conscious right now. Yes. But we we very well might decide that we want to expand the definition of consciousness to include other types. Yes. Yes. So I broadly agree with everything that you you just said. But I should point out that I think the what you're calling the empirical criteria are part of

the con of they contribute to the convention as far as I see it. our consensus for how we use the language of consciousness. So that can be shaped by many things including what the scientists say who are studying this kind of stuff. So so if um um scientists say say oh well you know we've been looking at octopuses and we see that they've got these things that are like no receptors in uh in in mammals and and and their brains are organized in this and that the other kind of way and so that that that

all contributes to the consensus to the convention. So I don't see these things as separate things, right? So um yeah, but but even when what can be investigated about the machine does not perfectly match our consciousness, I think the second point is to say we might want to expand our the very definition of consciousness. It goes kind of both ways in empirical right we we might want to absolutely yeah so the consensus of how we use the language of consciousness can yeah that can change uh certainly can change right so as a summary for our

audience um what this whole discussion around Wiganstein has set up set us up for is to give us license to proceed with an empirical examination of our current consciousness of the current substrate the current material substrates if you will of our current consciousness And the theory you propose to explain it is called global workspace theory. So tell us about that. Yes. Um so the idea is to think of um uh a cognitive architecture in which there is a whole collection of uh parallel processes processes that are working at the same time. So in intuitively these

may be processes that are to do with uh memory. So I first I walk through the lobby and I see certain uh cues and and and like you know chairs laid out and the and the uh the reception and so on and that triggers all kinds of memories. And so inside my brain there's all sorts of little processes that are running that are triggering associations with being in a hotel with with uh uh you know seeing these kinds of uh chairs and the reception and all kinds of things. Then I might pass a person and

and that might be what remind me of some somebody. So, so all those kinds of processes are going on at the same time. You have all of these processes that are going on unconsciously in parallel but some of them become really really important to the current uh uh situation and those ones that are that are sort of um command attention as it were they take over the attention mechanism. So what they have to say the information that they're they're dealing with then gets broadcast throughout the brain. And so that's the global workspace. So they're contributing

to the global workspace and and the the central metaphor there is broadcast. So the so there's a distinction there between the processing that's going on locally within these little processes um uh versus the processing that that's going on more globally that's broadcast through the medium of broadcast. To make it very simple um let's say I have uh memory, I have emotion, uh I have appetite. Let's say that there's those are three parallel processes. uh in every moment these three processes are competing for which one is has the most important thing and then that uh becomes

broadcasted to the other parts and that is what conscious experience is about that's the rough claim right rough roughly but I think I'd want to kind of break down the processes to be smaller ones so the memory side of it so say I walk through the hotel lobby and over on my left is a reception and uh and and over to my right is the bar and to ahead of me is is is the lift. Well, you can imagine that in my memory there's there's a little there's a whole lot of little processes that are

being active that that deal with bars and um and they might be saying, "Hey, let's go and have a drink." You know, there's a whole load of processes that are active that that deal with hotel receptions and they're thinking, "Oh, we need to check in. Let's take our luggage over there." And then there's a whole load of processes that deal with um you know, going uh getting in the lift and going to the third floor, but but also I need to getting late. I need to meet Jonathan. Right? So the need to meet Jonathan process

if you like to think of it that way is going to cooperate with the go to the lift process and together that's what they're they're going to broadcast to the whole brain and they're going to shut out the kind of like let's go to the bar and have a drink uh urge and and the uh let's think about bars and receptions perceptual bits. So so so some things are going to be shut out some things are going to uh kind of come together into an active coalition that's whose influence is going to permeate the whole

brain. So the the explanation of consciousness by this theory is the processes the coalition of processes that win the competition and that is broadcasted out into its entire system. Right. You wrote a paper titled a cognitive architecture that combines internal simulation with a global workspace where you presented uh a computer architecture a robotics architecture specifically that is built to imitate a global workspace that there's these different competing processes and the most important one is gets broadcasted. You know drawing from our Wikinstein conversation or do you also want to say that this kind of thing is

conscious in some sense like if we imitate the conscious structure the cognitive architecture of human consciousness in a machine that will also become conscious. No, because um because you know e even if we uh even if we accept global workspace theory for biological consciousness um but you know the idea there is that is that is that it would be a necessary condition not a sufficient condition. Just having something that conforms to that description is not enough to sustain the level of complex behavior and and and or internal activity even that is going to lead us

to um treat something as a fellow conscious creature. Right. Right. conscious being building something in that way may enable you to build something that does exhibit very sophisticated behavior may you know maybe that's the that's the key to doing that maybe I see so so let me try to tie this with our Wikenstein in conversation where Wickensstein helped us clear the ground consciousness investigating whether LLMs are conscious is can proceed empirically you looked at how current consciousness is sustained that's the global workspace theory and then you're saying This might be a necessary condition but not

a sufficient condition. But but that's how you hope this research right the the kind of trajectory. Yeah. This is all about the easy problem. So this is this is trying to trying to explain uh the the the uh you know how well the psychological um uh and behavioral and cognitive aspects of consciousness. So given um the perhaps increased urgency of answering the AI consciousness question, what do you think are the other empirical methods we should explore alongside global workspace theory? I think behavior is is really really important. So very often when people discuss consciousness and

I do suspect that this is another aspect of of dualistic thinking um they tend to think of it as as some disembodied, you know, uh kind of thing. In the case of the of global workspace theory, what kinds of sophisticated behavior does having this cognitive architecture uh uh you know underpin? Right? That's that's uh what's the point of it? What's the why is this cognitive architecture of interest? was it's of interest because it's a way of of um uh of marshalling the resources of massively parallel computation particularly in the case of of of biological wetwware

because it's it's kind of anatomically distributed and there's all kinds of challenges with the wiring and things like that. So it may not actually be relevant in the computers in the case of AI. And so embodiment obviously is uh one of the key terms in the title of your book. Um and the claim is is that embodiment and behavior is it seems to be another necessary condition in addition to the global workspace architecture. So so for our employing so all of this is about you know when when uh would we want to employ the language

of consciousness right? So let's think about just in the biological case, it may be that that we employ the language of consciousness in the context of things that exhibit this kind of really sophisticated and rich and rich behavior. Um and uh and when we look into it, we discover that that uh that that them instantiating this global workspace architecture is a really important part of that. So another really important word here is integration. So what the global workspace architecture facilitates is is is is integration. So it means that the the full resources of our brain

can be brought to bear on the current situation. How do you think about the precondition of embodiment given LLM? Because LLM sure seem to be unembodied. They they've simply ingested a lot a lot of text and sometimes like other kinds of bits of information and yet they seem to exhibit a great deal of intelligence. Uh yes indeed. So we are we do want to talk about intelligence or consciousness maybe. Let's let's that's a great distinction, right? Because what is the what what do you think embodiment is a precondition for embodiment or sorry intelligence or or

consciousness? But maybe we can tackle them separately. Yeah. So so so does it make sense to apply the word intelligence to contemporary large language model? I have to say I think it does you know I mean how can you not I mean um so so the so it's just natural to say I think in the in the case of of it's the point of the touring test right? Yeah. Well, yes indeed it is the point of the cheuring test and they do kind of pass the cheuring test. Yeah. So I think it's natural to to

say they do have some kind of intelligence. Let's take another word understanding. So that's a controversial one. So so you know people have written and said they don't have real understanding and so on. But again you know uh I think that there are many circumstances in which um in which it's we cannot help applying the word understanding to to them. So so for example I say I want to convert all my latte entries to this format. So then I give it the thing I want to convert and it does it and it's you know it's

kind of about right but then I notice that it's it's sort of inserting more commas than you know commas than I or more spaces say more white space than I like in my so I say to it uh but I I only want you know to indent with two two spaces and and it says oh sure here's the thing indented with two spaces right now you would have to say well it understood my instruction it understood the original instruction didn't completely understand it but maybe it was a ambiguous. Then I corrected it and it understood

my correction and it made it made it right. Now, why on earth would we not use the word understanding in those circumstances to characterize what's going on there? Maybe I can give another kind of push back against the the Wickensteinian position as you've presented it um by just pushing the strategy a bit further. This this idea of observing and understanding the the the cognitive architecture. So let's imagine I have a cog a robot twin here looks exactly like me purely silicon okay and he has a global workspace uh uh setup and of cogn architecture his

behavior is onetoone my behavior in the exact same scenario and let's say we we figure out even more biological preconditions of consciousness all of it is imitated in his silicon architecture and then the question is is he conscious is it is it reasonable to think that he's conscious when you poke him the exact same reaction that I laugh. I scream and I shout like a little girl. What's the answer there? Like like like this is where I think the empirical Well, sure you can guess where I'm going to go with this kind of You're going

to say yes, right? No, no, no. I'm going to say I'm going to say well it just you know I how will we come to treat that that thing? How will we come to speak of it as a society as a community? That's so that so I want to replace your question with that question. Right. But but recall there were two standards that there was the the consensus criterion and then there was the empirical criterion and I know that these are interrelated. Well yeah don't I I I kind of um you know corrected that as

it were because so the consensus is draws on behavior and it draws on uh ways it works inside. So, so the way we come to treat it that that that may be that may be we may uh be very influenced by the fact that that's that scientists say, "Oh, this is how it works inside. Look, it's only doing this that and the other. It's just a lookup table." So, oh, yeah. Okay. Well, we're not going to treat it. That's, you know, that may be the consensus, right? Right. Or it may not. I don't know. I

don't know. You know, I we have to we'd have to wait and see how things unfold. It's very hard to imagine. And so in in ordinary language in the same way that you're comfortable in ordinary language with me saying this is consciousness. This is this is not conscious. I'm conscious he he is either conscious or or right like like he is conscious or unc like there is a fact of the matter in ordinary language. Well as for this particular example um of course I'm not going to answer the question is is there a fact of

the matter? So, so you know when this thing has been among us right for these things for 30 years and uh and uh and and we've all settled on the way we talk about these things and treat them and behave them behave with them and interact with them. Um then you might ask somebody like me in that community is there a fact of the matter about whether twin um and then in ordinary language and then they would say they would they would say well I don't know why you're adding that weird phrase the fact of

the matter but of course it's conscious or not right I don't know I see so let let me put it this way let's say a society was determining who to harm me or my or my twin and let's say you know we all need to eat food there's only one one food enough for both of us. Yeah. Uh presumably if this was a garbage can with no consciousness, you'd say giving it to the garbage can would be a morally wrong thing to do than giving it to Jonathan who's alive. Is there a right or wrong

moral decision there? Presumably there is given whether my digital twin or or my my robot twin can suffer or not. So the the moral right or wrongness seems to be dependent on the consciousness question. So there there is also a right or wrong there, right? Yeah. Well, you know, societies can get it wrong. Yeah. Yeah. Societies can 100% get it get it get it get it wrong. So but but that in itself is taking us into difficult territory. Here is here is the way that I like to wrigle out of these situations because because I

think you know there is there is an inherent tension here that we are always led to a point of tension and then it's how we break out of that tension. So there's a sentence in my book where I say it's inherent in the language game of truth to say that truth is more than just a language game and then we have to let the matter rest. Right? So when the Europeans first discovered America, there was a big debate about whether the Native Americans were real humans or not. The answer is yes. They are real humans.

They can they can suffer. I don't think I can I don't think I can escape from my uh you know the society I live in and conceive of thinking of them as as not conscious beings who who who suffer. So let's say and this is very plausible in one society or one political party one community they treat my digital twin as the same as me as moral standing. Yeah. One they they treat him completely like the garbage can. Is there nothing more the philosopher can say about about is is someone wrong or someone right or

I don't know. I mean, we have to try and reconcile these these these communities because um uh or or maybe we can't you know um uh I don't know right which community am I in in this situation? Are you in this hypothetical situation? Where are you putting me? I don't know. I I I don't know. I Yeah. So So my answer to this question depends upon where you put me of course. Okay. So, so, so let let's say you were in the camp that that thought that we had equal moral standing and there's another community

of people. If I'm in that camp, then I'm going to think we have even if I'm in that camp, I'm going to think we're not right. So, back in the the European debate, the Native Americans, well, I mean, this of me today and I cannot stand outside of that and and and you know, and I obviously I find it utterly morally repugnant, right, to think that Native Americans aren't people, right? Because it's wrong. Yeah. Because they are. They like because well so so this is getting well okay let me ask you you're um uh very

much drawn to Buddhism as well and you know so so uh so so do you think that this is a the moral moral question there question of compassion there is that is that are you asking a question that is a matter of conventional truth or or absolute truth I think ultimate truth what might be different between that question and the thought experiment I set up is in the case of the Native American uh example. The reason it's based on what I conceive to be a factual error, thinking them not to be humans. Yes. The the

the error, a factual error. Is that a matter of convention or is it a matter of ultimate reality between the Native Americans? Yeah. So, yeah. So, so my Buddhist training, you're pushing me to say convention, of course, and and um Okay, let me So, fair, fair enough. Right. You're you're kind of stuck there. Well, so am I. You know, hey, uh I mean I it's very difficult for us to to to to to step outside of you know, things that we find utterly morally repellent. Of course, you can conjure up, you know, thing, you know,

moral dilemmas that I don't know the answer to. Let me ask a final set of questions um on the the uh hardware or wetwware that the brain in the computer is based on. So this is another quote from your embodiment book. Mathematical considerations separate brains from conventional computers. A complete description of the instantaneous state of a computer is possible using a finite set of binary or natural numbers. The membrane potential of a neuron to pick just one physical property is a continuous quantity and its exact value is pertinent to predicting the neuron's behavior. A related

point is that the entire state of a computer advances to its successor when and only when the computer's centralized clock ticks so that all its internal events line up in time like a row of soldiers. But events in the brain such as neuron firings do not keep time in this orderly way. An electrical spike can be emitted by a neuron at any time where time of course is another continuous variable. From a mathematical point of view, this property alone could be enough to push the dynamics of the brain beyond the class of Turing computable functions.

The brain operates on a continuum both in the uh uh firings of the neurons themselves as well as on the time or asynchronous versus synchronous. Yeah. Yeah. Um, does this mean theoretically the human brain might have intellectual capacities that are just beyond any type of uh uh touring touring machines? Well, okay. So, it certainly means mathematically and theoretically that it could fall into a class of computations which is outside of the Turing class of computations. So, that's that is a mathematical fact. Um but uh but on the other hand we also know that during uh

computations can simulate a continuous system to any arbitrary degree of of fidelity to any degree of fidelity that you like you know you just make your numbers have give them more decimal places right um now you're never going to give them an infinite number of decimal places but you can always add more decimal places basically to to numbers so you can so you so you can simulate any physical process to any arbitrary degree of of fidelity. Now they're mathematically they're different. They remain different. But does that matter functionally? That's really what you're asking. I have

no idea. I suspect not. I suspect that because you can imitate it to to any arbitrary degree, right? Yeah. I mean that so that would be my that would be my inclination would would be to think that there's no barrier to what you can do with a computer because of this mathematical fact. in your embodiment book you described all the different types of of the brain's architecture that we have not simulated right basically what we've done with current LLM's transformer architecture uh generative uh large language models is just taking the neur the the neuron the

the neural net kind of architecture and then just scaling up into a tremendous degree why has just imitating that part of the brain and not all this other functionality been able to replicate so much of human behavior that needed the whole brain. Well, so first of all, uh you know, what we have in neural networks today in artificial neural networks is very actually very very different to what we have in in the brain. So, so artificial neural neural neur artificial neurons are not really much like real neurons at all. So that's I mean that's one

very very important caveat and the kind of learning that goes on is very very different to the kind of learning that goes on in in in real brains. So um so the question is even more interesting in some way. It is actually even more interesting in a way. Yeah. So so so why how is it that um that using this very different very different substrate we've managed to kind of create these uh um well this this extraordinary uh sort of similacum of intelligence. The interesting thing is that we don't really know. We don't really we

I mean that's the bizarre position that we've ended up in which I would not have anticipated um if you'd asked me when I wrote that book um you know 15 years ago um uh where we where we we're building things um and because we're training them on very large amounts of training data and they're very very large collections of neurons very very big networks and then then all the weights you know of these networks change as a result of training and then as if by magic they end up with this extraordinary functionality but we don't

really understand how they work at a at a level that we're able to explain the you know the intelligence there as far as there is intelligence there maybe I can ask this question um in a slightly different way like we asked the stori before singularity question which is what does it tell us about human intelligence because um for the longest time the dominant paradigm of AI is symbolic AI and for our listeners back home, this is giving declarative like logical propositions that chain together in in various ways to solve problems. Yeah. What has worked instead

is these black boxes of like rough simulations of but not even to your point of neural nets. Yeah. That is able to produce behavior that symbolic AI is just so far away from. So is this something that we can learn about human consciousness which is this? The Stoics take an extremely intellectualist view of human nature. Behind human nature are the series of propositional statements we're asenting to. That is what they think is driving the human machine. Yes. But maybe that's not it, right? Maybe for us what what is at basis is a much more complicated

biological architecture. Yeah. Which the logical statements are only surface manifestations of. Is that something you That's absolutely right. I think uh you know if to put it crudely I think we've probably arrived at the conclusion that uh you know human level cognition uh is is implemented on a spaghetti like mess. It doesn't have the kind of structure. It doesn't have the kind of structure that we would that we intuitively think should be there. It's a mess. Um and and I think that in my own trai, you know, intellectual trajectory as an AI researcher, uh it's

been a kind of a a gradual retreat from wanting to build things in uh in a way that is intelligible, you know, where the architecture is fundamentally intelligible. You were on the symbolic side for a long time for for for for quite a few years. Yeah. I mean, I I I also abandoned it a very long time ago. But then the interesting thing is that then even when I was working in in machine learning and in neural networks and uh you know what I spent a long long time on was trying to learn representations neural

representations that nevertheless had a sort of symbolic like structure right you kind of clinging on to that clinging on to that that kind of uh vision of things that things had to be had to have a compositional structure like language um and but okay they would be learned and they would be realized on the neural kind of uh substrate. But nevertheless, at some important level, they had to have this language-like compositional structure of like objects, predicates and propositions and things and um a topography essentially, right? Yeah. But I but and then I you know I

gradually had to retreat more and more and more from that position and even so so I think so I think then you then you know you retreat a bit more and then you think okay let's stop trying to force the architecture to have that structure. So the dural architecture, right? So we're not we're way out of gofine and good old fashioned AI, symbolic AI, but but I'm but I'm still trying to kind of force the the representations that are learned within the neural substrate to sort of have an have a kind of symbolic like structure.

Then okay, that doesn't work either. So then so then maybe you're thinking, well, okay, let's let it let's let it all emerge, right? So we we'll have this kind of complete black box. Yeah. Right. But but but then then surely we're going to we'll find that there are these structures there when we look but even that is not but even that doesn't seem to be the case right you find that you you know you train these things then there's a whole field of mechanistic interpretability that's trying to understand what goes on inside well you know

that's a mess as well it still looks like a mess you know you just keep looking inside and it and and of course they've made a lot of progress there are things that you can extract but they don't look like the intuitive categories we had for how we would understand cognition in the past in terms of you know language like propositional representations and so on in in some sense your entire life has been a series of retreats it has not but not just on the computer science side but it it sounds like also on the

philosophical side oh I don't know about that did did you not retreat to the Wigansteining position from a dualist oh you started oh well oh well I mean I still haven't escaped the dualist position but I know where I need to go right right I think it's but you see I'm trying to say on the metaphysical side where I imagine as any kid growing up interested in this you think well there must be an essence well is it is it Adams is it course and then you right so you you see the mirror like it's

a it's a both turning to Wickenstein as well as turning away from symbolic AI is a kind of relinquishing control in a mature way right if you if you like yeah although I although I think it's a bit different with the philosophy because uh with the philosophy I don't think I've I've I've got where I want to go but I knew where I needed to go from a very long time ago I see um which work which is what Buddhism and Vickenstein because you've obviously thought deeply about these issues as well right and so of

course you can easily kind of rec conjure up uh these dualistic thoughts with these kinds of um uh going down the sort of line of thinking that you're so so so there so it's sort of a bit different right so I've long known where what the destination should be but not known not quite got there um uh with the philosophy with how you build AI and how you understand cognition and it really has been a series series of retreats and and uh but that's progress I mean you know if you if you uh change your

mind that's that's learning rich Sutton of course famously described this you know very similar thing as the bitter lesson I don't you come so he has this paper called the bitter lesson where he says that well you know what we've learned is over the years we started you know thinking that we want to build things that we can understand and we're going to reveal and and and that's the that's the uh that's that's what makes it exciting. We'll understand these things that we're building and we and it's by understanding that we can be be able

to build them. But in fact, we've had to relinquish all of that and realize that what do you do is you is you use learning, you use scale scaling and you use search and you use a lot of computation. So you scale the training data, you scale the computation, you scale search and those are the things mindless stuff at scale is what is what works. Where do you think it's a bitter lesson because because because intellectually there's something frustrating about that. Yeah. We we want it to be this you know beautiful topography this platonic form

right but but yeah so your work uh your intellectual work as a AI researcher as a philosopher um do you think AI is going to replace a great deal of this certainly it's already augmented I mean even in my own work but but do you think yeah do do you think AI is going to replace replace the intellectual yeah replace a lot of intellectual work that you do now well um so so when it comes to the the philosy I mean there's no point in it replacing it because because you need to be the one

doing it. Yes, of course. I mean that's like getting a a robot to run around a running track for you. I mean there there that is you literally pointless, right? And by the way I think similar you can make similar answers to uh to using AI in creative contexts as well. So so um the point is not the production the point is the cultivation. Of course, of course for some people the point is the production is part of the point because they make their living out of it. So this is like really important issue for

for for philosophy professors. Yeah. Well, for philosophy professors and people working in the creative industries as well. And these are, you know, important issues. But but you know, uh but if if I'm doing my uh my my art project, you know, at home for my own satisfaction, you know, I'm not going to just prompt an AI to do it. I'm just, you know, because I want to make the thing, you know, that's the point. the same as if I don't want to make a robot run around the track for me because and similarly you can't

do you know I can't make an AI do philosophy for me because that's not going to give me insight but you can in all of these cases uh accelerate yeah use them as tools to to to help you in what about the the AI researcher point because that I imagine at least compared to philosophy is a lot more about the production right well that's true yes so so I think AI research uh as with many um you many things is going to be you know uh augmented and and enhanced uh by AI of course you

know but but not just that but the the potential of of perhaps replacing yeah well maybe um indeed yeah sure I see and uh and who knows where we go from there what do you think if you have kids how do you think about educating them right now like what what kind of skills and well great question I think we need to educate people in how to live Well, philosophy I mean yeah well yeah I mean not maybe not the kind of philosophy we've been talking about here but you know what what is a good

life it's a different philosophical question I remember reading a few years back which is uh canes John Maynard Kanes has this paper called the economic possibilities for our grandchildren which he wrote in 1930 and uh and there he's imagining a future where a sort of a sort of well utopian potentially future of abundance where you where economic challenges have been overcome and people just have can lead lives of leisure. But then he poses the question well what do we do then really you know what how how what would it mean to lead a good life

under those circumstances when a certain aspects of meaning are taken away from uh from us. I think these are extremely good questions. I don't personally have answers to those ones. I see. Well, thank you for a fascinating interview, professor. This has been this has been amazing. Thank you very much. Yeah, I've enjoyed it very much. Thanks for watching my interview. If you like these kinds of discussions, I think you fit in great with the ecosystem we're building at Cosmos. We fund research, incubate, and invest in AI startups and believe that philosophy is critical to building

technology. If you want to join our ecosystem of philosopher builders, you can find roles we're hiring for, events we're hosting, and other ways to get involved on jonathanb.com/cosmos. Thank you.