#152 Why AI Will Never Be Conscious

34:37Alex O'Connor:And I think whatever we're left with, it's not going to be like recognizably human. It's not going to be a necessarily human trait. I think almost, yeah, certainly not. That would be, I think, very strange. That would be basically to sort of deny that consciousness could be expressed outside the human case, which I think would be a very wrong claim. I was thinking of Avicenna's man yesterday because I was in a flotation tank. Oh, cool. Yeah. And I do this now and again, and sometimes it's relaxing. Sometimes it doesn't go so well. And yesterday, for various reasons, I don't know why, it didn't go so well.

35:16And of course, your eyes are closed. It's quiet in there. Nothing perfectly quiet. It's pretty quiet. You're floating. So you do kind of lose the sense of your, at least your body configuration. kind of not so sure where my hands are but other senses come to the fore and yesterday in this flotation tank you know just my the sound of my heart beating just became very very dominant so we have all these senses of the body yep and i think this is relevant firstly because it's just it's it's hard to uh it puts pressure on the idea that we can just strip away these senses because other things will bubble up.

35:58But what I think it points to, so where I land when I try to imagine what would be left, where I land is something like the feeling of being alive. Right, I see. What does that mean? It's a bit unclear, but it's what I imagine might be there when you strip away, indeed, all extra receptive perception, all visual auditory content all cognitive content too so not the cartesian rational thinking yep that's left that that your mind quietens so this is some overlap with states of pure awareness in meditation and and some psychedelics so you strip that away you strip away the experience of emotion of specific emotions you strip away the experience of the body as an object in the world you strip away the first person perspective memory is gone intention is gone can you strip away is there anything left and could you strip that away and still be conscious my my intuition is a good way to think about that is this basic feeling of being a living organism without shape boundary specific content yeah but that it's kind of persisting

37:22Alex O'Connor:in time yeah in some way i mean descartes famously to establish his mind body dualism keeping them separate the way he gets there is just by saying like well i can conceive of myself existing without a body and i can certainly conceive of my body existing without my mind just like a dead corpse and although they might be tied up in various ways that the fact that they are conceptually different means that they are different things they might they might coincide but they're they're literally conceptually you know separable and that's that's basically his sort of principal grounds for saying that they are different types of things i want to ask you if you can imagine yourself without a body and most people i think sort of go like yeah sure i can sort of imagine floating around in the void.

38:12Alex O'Connor:But I mean, given everything that you know about consciousness and your understanding of how it works, can you actually make sense of some kind of conscious being that has no embodiment whatsoever? Or is that actually something that you now, given your views, sort of can't imagine? Embodiment in the sense of having a physical substrate or having a body interacting with an environment? No, just having a physical... Okay, I just mentioned that because one can think of, let's say, an organoid as it has a physical substrate, but it doesn't have senses or motor effectors or legs or any, it doesn't have limbs.

38:51So it's kind of disembodied in the sense that we might use embodiment in cognitive science.

38:56Alex O'Connor:contact with the outside world or anything but no i mean like not even not even having that okay can you imagine i mean even like conceptually can you imagine just sort of existing in in the void with no physical i think conceptual conceptual imagination disorder is too low a bar i don't say much store break yes you know i mean they're concepts i can i can create a concept in which there is a disembodied consciousness yeah can write stories about it uh but i think this is you can't really use that to carry much weight yeah sure um it's a problem with these kinds of conceivability arguments in general i think so it's it's a bit like can i imagine a boeing 747 well they're out of service now can i imagine a boeing 787 fly backwards through the air yeah well yes of course i can imagine it i can picture it yeah right but the more i know about aerodynamics and engineering, you know, nomologically.

39:54So given the laws of physics as they are, you can't have planes that fly backwards. So that's where I land. Yes, conceptually, I think it's possible to imagine these things. But if I want to understand how things are in the world as it is with the laws of nature as they are, then the more I know about the brain, the body, how intimate these connections are and how explanatory they are as well then it becomes you know it becomes less it

40:27Alex O'Connor:doesn't really mean anything to say that i can conceive it yeah and i think that also the reason i asked you that was because there's a level at which you can conceive something and then upon learning new information you suddenly can't like a child can imagine a plane flying backwards because you just picture it in your head but once you know how planes work once you know how lift is generated it's like you actually kind of now can't imagine the plane flying backwards i mean you can you literally imagine a physical object moving across the sky but you can't imagine it flying backwards because flight is a is a is a physical concept that is applicable to the laws of physics as we know it you know and once you understand on a deeper level like it's just not even something you can properly conceive of unless you're just sort of intentionally fooling yourself and if you know or are convinced that consciousness is a product of living systems then maybe sort of descartes was just wrong that you shouldn't be able to conceive of such a thing at least in the real world you know i think that's yeah the more you know about it the less likely it becomes that that these things are separable um this goes to the neural replacement thought experiment as well if you don't know anything about neurons it's very easy to imagine that you could replace one with something made of silicon or if you if you've kind of reified the metaphor of the brain being a computer to think that it's really is a computer then you might also think that you can replace a neuron with a with a silicon thing because computationally you might be able to say they could be equivalent but the more you know about how real neurons work what they do, how their collective behavior self-organizes and assembles and changes over time, then just like a plane flying backwards, it becomes less and less conceivable for the world in which we live, for the kinds of things in the physical world which we have.

42:23Alex O'Connor:Yeah. We were just talking about Matthew Cobb before we started recording. I just had him on my show and he's got this book, The Idea of the Brain, which is a history of the way people have thought about the brain and he sort of tried he said to me that he wasn't really sure how to structure the book until he realized he could do it by what was the leading analogy like what what did how did people think about what the brain was and like it's astonishing that basically humans have always just picked the most complicated thing that exists and just been like brains are like that you know for some ancient writers a brain is like a is like a complex sort of musical instrument or some or for for some people uh when like mechanical um what's the word with you know pumping air through tubes and stuff hydraulics you know became a thing the brain began to be seen as this like hydraulic system and air was being pumped through the veins and stuff and then we discover electricity and we're like oh the brain is electrical and we invent computers and we're like oh no it's not like air going through the veins it's like it's kind of like zeros and ones and it's like bits and it's and we start confusing the map for the place because we start forgetting that this is just an analogy to help us understand and we'll never know what the next big technology is going to be but you know i like to reflect on whoever it was in like the 1800s or 1900s who said oh there's nothing more to invent anymore like we're done i can't possibly conceive anything i think that whatever comes next we're probably just going to say no no a brain is a bit like that but it's i mean it's already happening a little bit isn't it i mean even though computers are still involved with ai we have a but we have things like the internet these very distributed self-organizing systems so so the metaphor is beginning to to alter even if it hasn't completely given way to a new technology altogether but yeah Yeah, Matthew's book is great.

44:18I think it really makes the point that we always have to be careful not to confuse the metaphor with the thing itself. This is like Whitehead's fallacy of misplaced concreteness as well. We just will get ourselves into trouble. It doesn't mean that the metaphors aren't valuable. And I think the metaphor of the brain as a computer is a particularly tricky one because it's not just that we're taking a complicated technological system and trying to find the ways in which it might be analogous. The idea of the brain as a computer also draws in all these theorems from computer science and physics about the generality of computation as well.

45:02And there's an ongoing debate about what physical processes count as computation. Is everything computational? We have like John Wheeler that everything is computation, it's fundamental. So it's a bit of a trickier one, but I think it's based on two specific events that when put together give rise to a strong temptation to think that computation is all there is. And the first of these was from Turing, who basically came up with the idea of an algorithm, of computation according to Turing. This idea of mapping sequences of symbols to other sequences of symbols. And he showed this was very, very powerful with the idea of the universal Turing machine, a machine, an abstract machine that could implement any algorithm.

45:58But what does that mean? When you say algorithm, what are you talking about? Exactly this. An algorithm is something that maps one sequence of symbols through a series of steps to another sequence of symbols, like a mathematical recipe. That's an algorithm. People might use it more loosely, but that's technically, in my understanding, what it is. Mappings from symbols to other symbols. You can do a lot with that, but it's not clear that everything is a matter of mapping symbols to other symbols and indeed Turing's classic paper on this which is now like 90 years old was motivated by the the idea of the halting problem the idea of showing that there was some even within computer science there's some functions which could not be solved algorithmically so there are limits even there but then many things in the real world are not algorithmic at things that are continuous things that are inherently random being some examples so there's more to the universe than turing computation and then the second event was walter pitts and warren mcculloch right there at the beginning of ai and of cybernetics who showed that simple neural networks if you just had these these computational abstractions if you like of artificial neurons that summed up their inputs and generated an output those could implement logical operations and given other things like storage and whatever they could be turing complete they could implement any algorithm and you put these two things together and they're both really important observations and you you understand that if you strip away everything about the brain apart from you know idealize it in terms of this network of neurons that are connected um that just sum up their inputs and pass on outputs yeah you can get everything that turing computation can give you and that's very very powerful and i think that's why for decades now people have sort of thought that we don't have to worry about anything else in the brain we can throw it all away because at that level of abstraction we get everything that turing computation can offer that's useful to know but turing computation can't do everything and it's very unclear that you know it can realize all the properties of the brain and when you look at a brain you realize that in fact it's all it's you can't separate what it is from what it does that really puts pressure on the idea that turing computation is all that matters we talked about

48:45Alex O'Connor:this earlier i mean you were sort of talking about separating what the thing is from what it does and we talked about bridges and stuff but you say in the piece that like with a computer you can separate what it is from what it does but with the brain you can't and i'm still a little bit unclear on what the difference is and what you mean specifically. Maybe you can explain why you can with a computer and can't with a brain. It's partly a kind of theoretical thing and it's partly a practical engineering thing. So the practical engineering thing is computers are useful because they're built this way.

49:19We can run the same program on different computers. They do the same thing. We can run different programs on the same computer and they'll do different things they have this kind of perfect interoperability which makes them useful and that emerges from the fundamental principle of turing computation which is you know people argue about it but there's a general consensus that computation in this sense is supposed to be at any rate substrate independent yeah that once you define the algorithm doesn't matter you know that's all you need to know what the system is going to do so long as the stuff that's implementing is capable of implementing it that's that's enough and brains firstly evolution didn't design brains for that kind of interoperability you know how if i took the functional dynamics of my brain and they don't have to work in in your brain they won't work in your brain but there was never any evolutionary pressure for that in the first place In fact, there's evolutionary pressure the opposite direction because maintaining this sharp separation of scales, insulating one level of dynamics from another is energetically very expensive.

50:32If you just think about a digital computer, it takes energy to make sure that ones remain ones and zeros remain zeros. That's one reason that AI systems are so much more energy inefficient than real brains. If you want energy efficiency, which evolution cares about, and you don't need this sharp separation of scales, you're not going to get it. You're going to get systems where what it does is much more entangled with what it is because evolution doesn't require there to be a separation. even i think even more interestingly this is something which i think is a very exciting line of work uh that's opening up at the moment there may be functional benefits for systems that are entangled in this way that is scale integrated in this way um they may be able to do things that are much harder to do in systems where you try and enforce a sharp separation between different levels of dynamics.

51:39If you look at brains, they seem to be like this. In their normal conscious states, they sort of sit in what we call this subcritical regime where small differences can make differences at larger scales and vice versa. So there's a lot of dependencies across levels of description, which I think is going to turn out to be pretty important in understanding how brains do what they do, whether it's consciousness or cognition. Do you think, I mean, you were talking about evolutionary pressures to make the brain do

52:17Alex O'Connor:particular things. Just more broadly, it might bring to mind the question of evolutionary explanations for consciousness. When I've spoken about consciousness and people sort of just say, well, you know, there's got to be some evolutionary benefit. Like that's why consciousness exists, because by being conscious, you know, you avoid pain and you sort of doing things that help you to survive. And two thoughts come to my mind when I hear this. Firstly, if something is not possible, it doesn't matter if it's evolutionarily beneficial. You know, if matter cannot produce consciousness, it would also be very evolutionary beneficial if I could make two and two equal six like when i wanted to but that's not going to happen even if it's beneficial right but more to the point it seems to me that any kind of evolutionary service that consciousness can provide could also be provided by unconscious reactivity by the sort of philosophical zombie type figure or some something that doesn't have a complex brain that such that it produces experiences, but just that naturally moves away from heat or follows the sun like a flower does in the garden without requiring this thing called consciousness.

53:32Alex O'Connor:Do you think that there is a good evolutionary reason for consciousness? I ask that in the vein of saying that I think it's kind of evolutionarily redundant. You think it is evolutionarily redundant? I think it is redundant. I can't think of a service that consciousness would perform evolutionarily that couldn't be performed without inner experience being present you know what i mean yeah i i get that i i'm sympathetic to that kind of view though i don't know it to be the case so it i still want to reserve in part of my mind room for the possibility that there may be certain functions certain things that that could be evolutionarily beneficial which do indeed require conscious experience that cannot be done via unconscious mechanisms.

54:24I'm not sure if that's true, but I don't think it's excluded. This doesn't mean that it's redundant in those cases where it exists. It may be the case, and I think this is the more likely perspective from my point of view, that conscious experiences have functions for biological organisms. I'm not really on board with epiphenomenalism. It just seems like a very strange perspective. If we think about what conscious experiences are like in general, they're easy to interpret in terms of having useful functions for us. In the most general sense, they bring together a large amount of organism-relevant information across many different modalities in a sort of format.

55:19You can call conscious experience a format here that is centered on the body that involves aspects of the body over time. And that sort of immediately suggests opportunities for action and response in terms of intentions and urges and so on. Very counterfactual and empirically, a lot of things that we do, do seem to require consciousness. I mean, this is the active empirical study of the difference between conscious and unconscious perception.

55:54So, even if you can come up with a system which could implement these functions without being conscious, And then I think for the kinds of things that we are, for the living creatures that we are, evolution has sort of used the available resource of the potential for experience that living systems have and drawn on it to create expressions of consciousness which are functional for us.

56:28Alex O'Connor:You just said unconscious perceptions. I'm just intrigued as to what those are and how we study them if they're unconscious. So this has really been one of the workhorses of empirical consciousness research in neuroscience and psychology. The idea is maybe you try to match as closely as possible the signals coming into the brain, but you try and create conditions where in one condition, there's no corresponding conscious experience and in the other condition, there is. a classic example is something like masking in perception. So you can show an image, but if you surround it with other images in the right way, then people don't consciously perceive the image in the middle.

57:27But it's still presented to their visual systems. So in that case, you might be tempted to say the perception was unconscious. if you show an image really really really briefly that might be the case as well a colleague of mine Axel Clemens in Belgium is he's developed a tachistoscope so a device very simple it's just able to show images for I think certainly millisecond possibly even microsecond resolution right um a more an example we use in our lab is we use something called continuous flash suppression. So we'd show an image to one eye and then in the other eye, we show this kind of continually changing Mondrian of colored squares.

58:14And the person in that situation will only consciously see the changing colored squares.

58:22Alex O'Connor:Does it make a difference which eye you put into? Sometimes, yes. People have dominant eyes, so it can do. But the point is that you can create a situation where some sensory information is present to that person's brain and you can show that it you know makes some kind of difference yet they don't report consciously perceiving it yeah i see that's what i'm getting at yeah i just i i don't know i i i find to find out if someone has consciously perceived something you kind of have to ask them right Right. And I suppose I'm just still a little bit like I've got a bit of whiplash from split brain patients and their ability to like draw an image that they've seen without telling you that they'll tell you, no, I didn't see anything, but they obviously did because they'll draw it in there with their hand.

59:13Alex O'Connor:And that almost feels to me like when you when you ask the language part of their brain, their left hemisphere, did you see it? They'll say they'll say no. And if you ask the right hemisphere, it says yes by drawing the picture. And so I'm just I don't know, it makes me suspicious to the extent to which as long as someone says no i didn't see it we can kind of count that as so to speak unconscious yeah no i mean you diagnose the problem beautifully it's it's it's a it's a really tricky area yeah um and they're still very lively there's debate about whether anything counts as unconscious perception yeah right what one of the issues the thing you mention about verbal report is absolutely key and um you know it could be that as in a split brain patient if you verbally ask someone whether they consciously saw something they might say no but let's say you asked them to draw it or to push a button or something like that you'll get a different response so there's a complication there typically in these experiments they require on pushing buttons or something like that rather than people verbally saying what they saw and ways you can kind of counterbalance across hemispheres and such but it gets but yeah it gets it gets really tricky there's a conceptual issue here too which is that it could be that a conscious experience is still happening let's say in your visual cortex or supported by your visual cortex but that you don't have cognitive access to it at all the philosopher ned block kind of argues for this possibility in this distinction between phenomenal consciousness and access consciousness which is the idea that cognitive access whether it's through verbal report or pushing a button or whatever it might be is is optional and you could still have conscious perception even without access to it that's really really tricky to get an experimental handle on because if you don't have cognitive access to it, then how would you ever know that it was going on?

1:01:16You've got to have some other reason, other justification. Yeah.

1:01:20Alex O'Connor:I'm increasingly suspicious of the concept of the subconscious. I'm glad to hear you say that that's a normal thing that people are discussing because I don't know. I think typically we think if there's something which what I call my self, my sort of reflective linguistic communicative self can't like access we call it sort of subconscious but then i don't know i think it depends on the degree to which you believe that consciousness must be centralized and smooth you know some people interpret this split brain related stuff alien hand syndrome all of that good stuff as essentially showing that there are kind of two centers of consciousness that are in communication with each other in such a way that I guess you kind of just experience both at one time and similarly to how like my eyesight really sort of appears to me like it's one thing like I'm not sort of aware that I've got two eyes right now it's only by sort of taking one away that I can kind of get a grip on it it just sort of mixes together but it's uncontroversial that I've got two eyes there are two things seeing right now but it's just sort of mixed together some people think that the brains are doing the same thing What is your view on the locus of consciousness?

1:02:37Alex O'Connor:Is there one? Is there two? Is there many more? Is it sort of a silly question to ask? I think it's a silly question to ask. It's a very, very hard question to answer. And there's a lot of debate about the unity of consciousness, which I think gets at the heart of this. And the split brain case is one of the key pressure points. yeah um so some people tend to think of the unity of consciousness as being something axiomatic that there can be only one like highlander right um because it's very hard to imagine two separate you know consciousness being disunified what what what might that even mean it's not so much to say there's two a two conscious agents in in one body i mean that that could happen in a split brain that seems possible i mean the evidence is really mixed by the way with split brain yeah patients it gets very interesting when you look at it in in detail it's like actually there seems to be problems with integrating things across the midline rather than detecting things at all and there's all these subtleties about cross-queuing like one half of the brain can see what the the other half of the body does and might make inferences yeah based on i would love to talk about that i want to allow you to finish what you were saying but i would love to talk about that and but but yeah to the to the unity point i i think that if we so i always tend to try and not be i mean i'm frustratingly non-committal about these things like consciousness seems unified to most of us most of the time i would not want to build a theory that required that to always be so because i think it might again be be right back to our discussion of human exceptionalism now there might be cases where conscious experiences becomes interestingly distributed non-unified in certain ways octopuses give me a good provocation here where maybe their consciousness is decentralized um in a way that that just isn't the case for for animals like us who are highly cephalized so cephalized well we all our neurons are in the head and our neural architecture sort of if consciousness evolved if we think that's a useful perspective for us it's evolved in such a way that we experience consciousness from a particular perspective which is somewhere in our heads so what's that word cephalized i think it's a word i think it means that most of our nervous system is inside is in one place in our head where cephal comes from is that with like a c yeah c e p h i mean some people say kephal rather than cephal so yeah no i've just not heard that word cephalopod yeah octopus is cephalopod which literally means brain foot um oh and they have more neurons in their limbs than in their central like brain foot brain right yeah i mean it's basically it's a good description i was like if we're whatever the word of cephalo cephalo or something well we have a lot of cephalization so our nervous systems are concentrated in that's why i thought cephalopods but cephalopods don't have it concentrated no there's also the pod yeah etymology i'm having i'm having adam alexic the etymology nerd on soon i'll ask him about it but um it's hardly you know either of our fields but Yeah, I mean, the first time we spoke on this show years and years ago, we talked about the extent to which we experience ourselves as up in the brain as well.

1:06:07Alex O'Connor:And I'm always sort of, I sort of wonder, is that because that's where the brain is? Is it because it's where the senses are? You know, we talked about strapping a camera onto like your belt and then like putting on a VR headset and living, experiencing the world from your belt buckle and whether you would, the sense of where you are would move down to your stomach. um and i kind of suspect that that maybe it would i'm not sure but it's weird to think about like where i am i feel unified but at the same time i'm if as long as i pay attention i can separate out the feeling of the chair from the visual experience that i'm having um i can sort of i don't know i can i can pay attention to the fact that maybe you know maybe i feel a bit hungry and maybe I feel a little bit like, you know, I'm a bit uncomfortable, I'm going to shift position.

1:06:59Alex O'Connor:And I can kind of separate out those experiences. So I've never quite known exactly what it means to say that consciousness is unified. Because like true unity of conscious experience would seem to imply like no delineation whatsoever. There's just experience. But I'm already being able to separate out various aspects of my consciousness. Yeah, I don't think it, I don't of differentiation of what's happening within consciousness. I think most uses of the term unity in this context suggest that all of these things are happening within a unified conscious field. You can pay attention to different parts of it and they can have different characteristics, but there's no way in which the consciousness of redness in the environment is is happening completely separately from consciousness of your other parts of your visual field.

1:07:58It's all bound into one. But yes, it has structure too. I think that's fine.

1:08:03Alex O'Connor:So talk to me about this mixed evidence about split brain patients and consciousness, because like the pop science and pop philosophy version of this is like, whoa, how cool it is that if you take a split brain patient who's had the connection severed between their two hemispheres their brains seem to act independently you can show them a word to only one side and they'll tell you they didn't see it but they'll be able to draw it um and some other really weird stuff like you can i think like if you show somebody the words like trying to think of an example here but like um uh it's not a great example but like the word market and super you know and and they're on one side they'll sort of separate out the concepts they'll draw a market and then they'll draw some representation of super and whereas healthy brains would put them together into supermarkets stuff like that's going on stuff like that i mean a classic example would would be you know you you um you show the it's by it's not each eye so yeah you have to visual hemifields rather than eyes but anyway yes the left visual field goes exactly exactly but you can still talk if you set it up rightly you can still talk in terms of left brain right brain and so one thing you might do is recognizing that in most cases language is lateralized to the left brain the left hemisphere you can show let's say an image to the left hemisphere um and then something else to the to the right hemisphere and then see what happens and i think a classic example right back to mike gazzaniga and so on is that um you show something relatively prosaic to the to the left verbal hemisphere and then you show something kind of funny to the right hemisphere and the person starts laughing and then you ask them why they're laughing and what they'll do is they'll kind of confabulate a reason why what was shown to the left hemisphere might be funny yeah but it's got nothing to do with that it's because there was something funny shown to the the other part of the brain.

1:10:11So there are these amazing examples of dissociations. Of course, the most amazing thing about split brain patients is that under almost any circumstances, it's impossible to tell that anything's happened at all. I find this really quite astounding. You need these very specific experimental setups. But it does get a bit complex. Firstly, a lot of these things tend to resolve over time. Very few split brain operations completely segregate the hemispheres. these days they're not done that much because medication thankfully for epilepsy has improved a former postdoc of mine now a professor in amsterdam did a series of experiments on vision um showing that in a split brain patient actually that they were each hemisphere was able to detect visual stimuli anywhere in the visual field but what they couldn't do was integrate across it.

1:11:08So this is a bit like your supermarket argument. It was the integration across the midline that was not possible. So I still find it fascinating. One possibility, Tim Bain, one of my colleagues, talks about sometimes is that it may not be a stable thing. Maybe under some circumstances, there's one agent and under other circumstances, there are two. Again, putting pressure on an assumption we might make which is that the number of conscious agents

1:11:37Alex O'Connor:hosted by a yeah by a skull has to be the same well octopuses are great for that too because like their their limbs will kind of like do their own thing it almost seems like sometimes an octopus's arm will go over here and then it will be like oh you know what's that as if the the arm is informing the the brain but if there's a predator that comes by suddenly the legs will just like shoot into space into place and and they sort of swim away real fast as if when needed they can sort of act as one mind but when not they can sort of separate out a little bit maybe something similar goes on in the brain you said that some of these cases resolve over time well this is now i'm kind of reaching very very into the the recesses of my memory at the moment but i'm I think you see the strongest examples like the one of the person laughing when you show something funny.

1:12:32I think these are most prominent quite soon after surgery. A lot of these things that are most fascinating about cases of brain surgery or brain injury tend to sort of normalize a little bit. So another famous one is neglect, where people seem to be unaware of what's happening on one half of their visual field after brain damage. That too tends to kind of ameliorate a little bit over time. I see. It doesn't mean it's not real. It just means that it's one of the reasons these things are quite hard to study.

1:13:04Alex O'Connor:Yeah, yeah. The brain is quite incredible. Do you know about the craniopagus twins? I don't think so. This is the other. It's kind of the opposite of split brain, which I think is almost more fascinating. sure um so this is a case you know you can get conjoined twins yeah um so twins born they cannot be separated usually joined somewhere in the abdomen or the hip so they may share a parts of a digestive system or something like that or even or even a heart um there's at least one case of conjoined twins who share a brain and they cannot be surgically separated and they've been around for i don't know how old they are but i think they're in the teens now um wow and what is going on there well what is it like are they are they are they individuals do they they seem to be yeah yeah so i i've i've never met them i've only read about them secondhand yeah but they they seem to have somewhat different personalities uh but they also seem to share things so one will be able to taste what the other one eats well for instance yeah that's weird are they are they like aware of each other's thoughts it's there even a meaningful distinction between their thoughts they seem to be as you might imagine extraordinarily synchronized yeah in their in their behaviors so it's i don't know what their thoughts are but they in and i've just seen these videos i feel bad because i'm this is just me watching videos of these things but but they seem to complete each other's sentences yeah so that seems to indicate that maybe they do sort of share at least the you know the contours of a single yeah thought but then again basically they're getting pretty much the same sensory input all the time and they're having the same sensory experiences going back yeah throughout their whole lives that's it's incredible isn't it i mean i never know how to make heads or tails of of these people and in fact there's an interesting analogous question to be asked about ai which i'll ask in a minute but first just because it came to mind when you were talking about um showing one thing to one eye and not the other um you know i i really like ian mcgill chris and one of ian mcgill chris's most important concepts is the concept of focus and he says that focus isn't really it doesn't seem to be like a brain activity as such i don't quite know how he characterizes it but he talks about how like if if i played two sequences of like random numbers or words into each of your ears so different sequence in each ear and i just told you just pay attention to the right ear then afterwards you could recite you know the series of numbers but you wouldn't be able to tell me what what the left one was whereas if i asked you to focus on the left hand side you could do it vice versa and that just it's extremely simple like yeah obviously yeah i'm focusing on one not the other but that just baffles me because i'm like well what's the difference what is the thing that's changing when i just decided i'm not moving there's no new information really there's no i'm just making this decision to to sort of i'm just i'm just pushing my attention over here and pushing my attention well that's it that's the difference but like attention that just seems such a strange like what do you think attention is like do you think attention is just another kind of brain activity do you think it's a sort of pre-brain activity like a qualifier as to how the brain activity occurs like you know what i mean like it seems weird for me to say that the attention or focus is just a brain activity because it seems to be the means through which i determine what brain activity happens you know what i mean yeah i think both i think that's it would be strange if it didn't i don't think would have the functional role that it has i think attention is one of those words it's only slightly less mysterious than than consciousness but i think it is less mysterious i think it's less sort of metaphysically loaded but it's one of those things that seems fairly natural like i pay attention to something but but when you ask what that really means it gets very difficult very very quickly um but there are lots of ways to study i mean it's one of the classic paradigms indeed as you said you can pay attention to different streams of sensory information and that will make a difference to what you can remember to what you do and and so on many things it is i think something that affects conscious experience we're more likely to be conscious of things we pay attention to i don't think it's a a complete gate you know some people might take an extreme view and say we're only conscious of that which we're paying attention to right I think this is probably overstating it but I think conscious contents have become dominated by by attention and in the brain you can think of many ways that might play out you know the more extreme views might take it as a sort of really strict gating mechanism that the only things you're paying attention to get through the gates and can influence other parts of the brain I prefer to think of it using the the overall framing of this brain as a prediction machine idea that attention is firstly it's not just one thing it's many things it's kind of many ways of adjusting the sort of the gain the signal to noise ratio how much does the brain update its inferences about what's going on based on some sensory data paying attention to something is equivalent to saying this sensory prediction error this piece of sensory information is is worth uh well, I was going to say worth paying attention to, but that's a bit circular, is going to have more of an impact on the perceptual inference on this sort of continuing unfolding process of prediction error and prediction error minimization.

1:19:02Yeah.

1:19:03Alex O'Connor:So on this point of the unity of experience, I mean, let's go back to the topic at hand that you have written about and has obviously caught a lot of attention and impressed a lot of people and won a prize for its eloquence. Well, I say for its eloquence, also for its scientific and philosophical rigor. And that's AI consciousness. And we've just been talking about how sort of weird it is to think about where the locus of focus is, where unified conscious experiences might exist and how they interact and we we were on a panel hosted by brian cox a question of science which was it was good fun it was a little um i remember sort of walking in and brian sort of being like uh oh you know let's not let's not talk about this pan cycle and i was like okay because i'd never met him before so i was just sort of like hey um so gonna gonna maybe gonna maybe do that anyway if that's all right with you it was good fun um and one of the things that i said and it was kind of tongue-in-cheek and i i didn't mean for people to laugh but they did uh but it's got a serious point in it is like when we talk about ai being conscious what do we really mean like i know you kind of think it's probably not going to happen but the thing you're imagining is not happening i'm like what really are we talking about okay so chat gpt becomes conscious cool but as i said on this panel you've got chat gpt on your phone maybe i've got it on my phone it's on we've all got our own sort of accounts i'm having a conversation here even within my account there are multiple streams of conversation and yet at the same time you know like open ai's like software and computing power is not stored in my phone it's stored somewhere else and my phone is connecting to it and i i'm like what would it even mean would people sort of imagine each individual conversation is like a new conscious being waking up unaware of the other ones or is it like a conjoined twin where they're they're they're their own thing conversations but they're kind of connected and can kind of interact with each other or is there like one chat gpt consciousness that exists wherever open ai san francisco whatever is is based and we're kind of interacting with avatars or like as if the conscious agent is like sending messages back and forth you know like how can we even think about what it would look like i think this is a great observation it's um david chalmers has written a nice piece about this i think it's called what we talk to when we talk to a large language model nice um and on the one hand i think all the things you say just highlights the trouble we get into if we get captured by the superficial fact that these things are linguistically competent.

1:22:01Under the hood, they're massively different. And it's not only now a question of silicon and algorithms, but also, yeah, is there an identity? What is the identity over time? What's actually talking to you? It's not the foundation model because that's stable? Is it an instance? Is it a server farm? The server farms?

1:22:19Alex O'Connor:Yeah, right. You can put a query in. It could be processed in Arizona. Yeah. Half in Arizona and half in New York. maybe the other way around the next the next time so it's what's stable also and i think this is key you can leave a conversation for a day or you know a year right then just come back maybe not a year because i would change it but you come back to it yeah and from the seeming perspective of that conversation that doesn't make a difference yet time time is fundamental both to how biological brains work and to our experience to the nature of of conscious experience for for human beings so seen that way i think noting these differences just should highlight the danger of reaching assumptions about ai consciousness on the basis of their fluency with language their ability to pass the turing test but on the other way i think it's really valuable to sort of set aside at least temporarily not forget about but temporarily set aside the question of whether they are in fact conscious and ask exactly how you put it for the sake of argument that's let's just ponder what it would be like if they were and i think in doing this we can we can probe all these issues like the unity of consciousness that we take of as that we that are so hard to get out from underneath of in the human case.

1:23:50And then maybe it sort of gives an insight into a wider space of possible minds. These minds need not be conscious minds, but there is very likely a very large space of possible kinds of minds. And we are in one region of this space. Maybe biological minds in general are in a larger space. but they are still somewhat constrained by the fact that biological minds evolved within bodies, probably almost all of the time. Maybe think about fungi or something like that, but most of the time. But there may be other kinds of minds which don't have those constraints, which could be very different. And the comparative approach is always useful.

1:24:38And I think we can do that without making any imputations that these strange new minds need to experience anything.

1:24:51Alex O'Connor:yeah i mean i think that the extent to which it's difficult to make sense of what we would even mean by conscious ai could be an indication that the thing we're talking about is like a little bit senseless maybe um but it's also going to depend on what your theory of consciousness is and what your sort of hunch about the nature of consciousness is and you said earlier that you believe in this idea of the controlled hallucination, which you've talked about a bunch and you talk about it in Being You, your book. And I suppose it might just be helpful for people who are hearing you for the first time on this show to just briefly sort of overview where you're coming from when it comes to consciousness before you start thinking about AI.

1:25:34Yeah. So the overall perspective I have on consciousness is to be very pragmatic, really. When asked, I describe myself as a pragmatic materialist. I don't know that materialism is true, and I don't want to be a person who's sort of out there on the front lines defending materialism against all carmas. But I think it's a useful, I've always found it a useful perspective to adopt, because it gives us the resources to at least chip away at the problem of consciousness and by chipping away i take something like this what i call the real problem approach so you know consciousness exists phenomenal realist it has many different properties vision is different from emotion is different from audition is different from memory and so on what i'm interested in is a kind of unified way of thinking about different kinds of conscious experiences that explains their relation to each other that explains their character and that progressively might explain something like their dependence on a particular kind of substrate whether it's biological or not the controlled hallucination view is is an expression of this It's really a gloss on a pretty old idea, which is perception as inference, which goes back to, well, in philosophy, back to Kant at least, and the idea that there's a noumenon and a phenomenon.

1:27:04And then in psychology to Hermann von Helmholtz, he was the first person to talk about perception as a process of unconscious inference. So the brain is engaged in trying to infer the causes of sensory signals, but all of that goes on under the hood. It's not even unconscious perception. those are the unconscious processes that underpin perception yeah and what we're aware of is the brain's best guess the brain's most what the brain comes to as the most likely interpretation of the causes of the sensory signals that it gets that's why i call it a controlled hallucination um and it kind of flips what used to be the textbook view of perception on its head you know it's classical to think about perception as a process of reading out the world in this outside in direction information comes into the eyes and is read out by the brain and you get deeper and deeper and it reads out more and more complex things this view goes the other way and it says the content of our experience even though it seems to be out there and we sort of passively register it no we actively generate it and the sensory signals are used to calibrate these predictions so that they're useful for the organism yeah and some of the classic examples of

1:28:24Alex O'Connor:this are things like the the checkerboard illusion you know which alex will put up on screen right now and uh it's you know the two squares which look like they're different colors but square a and square b on your screen are the same color and you can prove it with various ways and if you don't believe us, you know, take it into Photoshop and check it, whatever. And even when you know that, you see them as different colours because your brain is essentially constructing a narrative. Like, why would you want to perceive them as, you know, the actual shade of the pixel that's lighting up on your screen?

1:29:02Alex O'Connor:Like, it makes sense for your brain to interpret the world in a way that makes sense narratively. And so there's a sense in which, yeah, we are constantly generating the world we the first time we spoke on the show we talked about like the audio illusions the yanny laurel thing brainstorm green needle thing and some of these like whichever one you're like looking out for or listening for is the one that you hear like you're literally able to construct your perceptual reality i feel like every time i've heard you talk about the controlled hallucination i've come a bit closer to kind of understanding where you're coming from but at the same time underlying all of this for me is this presupposition that there must be experience right like i think if you say there is this thing called consciousness now let's talk about like what it's doing what its nature is like yeah there's lots of evidence to suggest that we're sort of the consciousness is as much outward as it is inward that's cool but do you see this controlled hallucination view as also like an explanation of what consciousness sort of is in the first place, or just why, granted the mystery of it existing at all, it presents itself in a particular way.

1:30:09Much more the latter. I don't think it really tells us what consciousness is. I have this ongoing discussion with panpsychist philosophers like Philip Goff, who I know you know, and he will always say to me, in fact, I saw him last week, he will always say to me things like, everything that you materialist-oriented scientists come up with about the brain and its relationship to consciousness is completely compatible with something like panpsychism, probably compatible with idealism and other things as well. And I think that's probably true. I have no reason to doubt it. But the question for me is, is a pragmatically materialist view?

1:30:53Well, I don't know what things really are. I mean, And as you've said many times on podcasts with guests that you can take a chair apart, you find molecules, you take molecules apart, you find atoms, you take those apart, protons, quarks, quantum fields. What actually is, is something I'm certainly not qualified to answer. Is it mental states? Is it stuff that has, in addition to physical fundamental properties, qualitative conscious properties as well?

1:31:28I don't know. But the thing is, I think a pragmatic materialism that says at the level of analysis that we currently are interested in or I am interested in, which is brains are made of stuff. And I tend to be more interested than most people in what they're made of too, not treating them again as just computers that happen to be made of meat.

1:31:55am i am i moving the needle on explaining properties of the phenomenon or am i barking up the wrong tree entirely and making matters worse yeah so imagine a kind of controlled experiment you take a bunch of of scientists who who have a sort of panpsychist outlook and you take another bunch of scientists who have a materialist outlook, both sort of pragmatically held, just they don't really know, but that's their predilection. And then you let them loose to construct their theories and do their experiments. And then you come back in 50 years and you see from some view from nowhere, metaphysical view from nowhere, which group has come up with better, again, criteria not defined, better accounts of consciousness, made more progress.

1:32:51I don't know the answer to that. But my suspicion is that at least from where we are now, we get further with a materialist outlook. And I mean, this is what I said. And why do I think that? Well, this whole view, for instance, of perception as controlled hallucination goes some way to articulating that you know we start with a view that our experience is some sort of outward in readout and then we realize that maybe something could do with internal generative models and prediction and then we realize that that's that principle which is based on a sort of materialist account of neurons um and their dynamics we realize oh we can understand emotion through those same principles we can understand the experience of free will within that same framework we begin to draw together apparently disparate phenomena in ways which have a kind of inner coherence and which have predictive and explanatory power we may still come up short in fact we do come up short when addressing you know the question of well why is there consciousness at all and yeah but but we understand more about the phenomenon And for me, I wonder whether the more we do that, the less perplexed we'll become about the fundamental mystery that the different metaphysical frameworks are trying to grapple with.

1:34:18Alex O'Connor:yeah maybe i i think the reason why i've been sort of unconvinced unimpressed by this this view of consciousness that you have is maybe because of the fact that i'm sort of i'm mistaking what your what your project is it's a little bit like you know if we were talking about music you know there's a piano over there and you could start playing major chords and minor chords and you could start talking about like how music works you could start saying that there's like a harmonic series and like that there are kind of you know you could do it in terms of strings like pythagoras and there are sort of vibrations and you know the the ones which are the right sort of uh frequency of vibration compared to others creates the happy sound and the sad sound and you can talk about all the stuff you can talk about artistic input you can talk about how playing various ways produces different sounds and you could do all of that and have this whole music theory you can track the like any musical theoretical canon and never once have needed to know or address the question of like well what is music oh it's vibrations in the air that's what's actually happening is like you know the air is being vibrated and that is hitting your eardrum you wouldn't need to even address that to have what some people would say is like a theory of music you know a complete robust theory of music but i want to know you know what's causing that musical no i want to know that it's vibrations in the air, right?

1:35:50Alex O'Connor:And I feel like with a lot of so-called theories of consciousness, they're like these theories of music that don't address what the music actually is. And I wonder the extent to which you see your view of consciousness as being one or the other or both, or whether you would say that you're sort of leaving that question of what it actually is to the side. I think there's a, you know, if I was going to be unkind to myself it's a sort of kicking the can down the road right on on that question but i don't think it is quite like that because i think the road changes and the can changes as as you do this sure right and this is because what we think of as the the mystery of of consciousness is is a bunch of related mysteries yeah there are things like what does free will exist what is the nature of free will things like that i think the more we can sort of dissolve those mysteries or wrap them into a single framework a single approach we we do make progress i also want to know how do we fit consciousness into our fundamental view of of the universe i don't know what actually exists you know i i think that stuff exists but what stuff is i i don't know it may may all be mental states i don't know um but i think there's the kinds of theories and experiments and explanations that we can construct come about mainly through thinking through following the scientific the scientific method in general has been about explaining phenomena in terms of the physical attributes of things even if those physical attributes turn out to be non-fundamental in some way there is a suspicion that this will not work when it comes to consciousness and i just i'm suspicious of that suspicion yeah you know i want to wait and see and see what's left just as in we might strip away all the contingent aspects of conscious experience let's strip away all the answerable questions about consciousness that follow from thinking about its relationship to what we think of as physical processes and see what's left and maybe there will still be a very deep residual mystery but maybe there won't i mean historical analogy which i use with caution here because i know it's imperfect is life yeah right that life was considered uh well i mean beyond physics and chemistry that there may have to even be something non-naturalistic which seems strange talking about life but something supernatural l 'on vitale something not accounted for by the laws of nature as they were and of course we don't understand everything about life but that sense of mystery has receded and we can now see that many things we think of as as as definitional of life can be accounted for through biochemistry physics so on yeah quantum mechanics to some extent as well so the sense that there was something incredibly resistive to this kind of explanation didn't persist over time now life is not the same thing as consciousness.

1:39:17Consciousness seems to have this different mode of existence, but aspects of it are kind of describable. We can describe different kinds of experience. We want to answer questions about which kinds of systems have consciousness at all. And I think this perspective of understanding how consciousness in humans is related to the physical systems that we are and then generalizing bit by bit outwards is the right way to answer these kinds of questions back to ai you know we need to come up with a way of talking a way of thinking about is ai conscious because that underpins a lot of moral and ethical decision making it's already affecting policy and and people's behavior and from a materialist a lightly held materialist view you know we can start from the assumption that it is something about there's something about the materiality of an embodied embedded human brain that matters maybe it's the computation i doubt it maybe it's something else we now have a research agenda about okay what aspect of our materiality matters and and why and i think coming up with justified testable examinations of that question hopefully answers to it will help us say the things that we we really need to say about ai but also about non-human animals about all kinds of situations so maybe this does speak to the fact that that there are slightly different agendas here i think a science of consciousness really needs to address itself to these immediate questions as well as the ultimate questions

1:41:02Alex O'Connor:of what consciousness really is yeah well the essay is the mythology of conscious ai it'll be in the description congratulations again it's a it's a great read and it's it's well needed i think because there isn't much in the way of like level-headed let's say discussion about ai and sort of trying to calm people down on this front and not from the perspective of like you know avoiding doomerism and ai takeover and terminator robots just sort of put that on the shelf for a moment let's just think about computation consciousness how it all works it's uh it's it's fresh air so i'll make sure that's linked in the description as well as being you your book's been around for years now we talked about it the first time we met but there's there'll be more in there on the sort of controlled hallucination related stuff and i'll seth thanks for coming back it's been fun thank you alex it's been a pleasure