Microsoft AI CEO on AI Consciousness & Artificial Companionship | Mustafa Suleyman

29:48So it, these things don't just have these kind of like binary threshold moments where suddenly it's super transformational and we're all doing this thing. It's just more like a gradual integration on all the platforms that you're already using. And it was just sort of start to feel like second nature. And for many people, it already is just asking your AI is the first thing you do rather than going to a search engine or anything else. Yeah. I mean, I think we're definitely going to a post, a post-literate world in a way where it's voice first and reading, writing start to become. We go back to what Socrates would have argued for, orality being the most supreme way to showcase your wisdom.

30:23So you see it more as strictly in the realm of companionship. Like I'm not turning to this AI companion for my research partner as well. Oh no, it's definitely going to be research. I mean, it's also going to be action. So you're going to turn to it and people are turning to co-pilot all the time for like long form detailed 10 page analysis holiday plans academic essays you know plans for refurbishing my household detailed descriptions of how to fix my car i mean this is very practical it's very operational it's very productivity driven i mean i love encouraging people to do the use case of like if you're trying to fix the setting on your remote control or your washing machine or your car radio just like photograph the thing and be like, why isn't this working?

31:08And you'll get like a detailed summary of the manual. No one ever read the manuals anyway. And like, you know, so it's, it's just so simple to get help on any topic. I don't mean emotional help. I just mean like productivity solutions. And if you think about the life of a parent, how is this slotting into a parent who this is an entirely new topic for? I saw a really funny video on social media yesterday of a parent um a mom who was trying to get her kids to clean up uh their toys in the bedroom and she took a video of the toys that were on the floor and had some ai generate like a 30 second news clip um where the kids appeared in the news feature for being um you know lazy and not cleaning up their toys and then she recorded their reaction of them like being having their mind absolutely melted uh yeah i thought that was very funny it's just creative you know like people are using it for bedtime storytelling all the time um to to create stories that bring in um experiences that you've actually had in the day you know like maybe you went swimming with the kids or something or you went to the park and you saw like a green parrot um you know making that then part of a fictional fantasy story for five minutes of a you know bedtime story podcast in the evening you know that takes two sentences to speak that prompt into co-pilot and get that experience it's kind of magical yeah i think we don't really know what experiences behaviors and inventions will come of this like we're still in the earliest days so we tend to slot in technology and its use cases to the frameworks we already have yeah but what is the world when we stream intelligence the way we stream electricity and if you think about it that way what is going to eventually be invented I mean, what does this system lead to when it's watching what you watch, hearing what you hear, and it's with you all the time?

33:02I mean, I know when I'm on a run and I get a research idea, to have a system that I'm like, plot that down and do the preliminary research and start to think about it in the way you know I would and what that leads to. Yeah, totally spot on. Like this idea of streaming intelligence into every place that we're at. I think it's going to profoundly change, you know, everything because we we shouldn't. I mean, maybe I'm naive, but like we shouldn't have misinformation anymore because you should just be able to verify, you know, and get like pretty good, reliable answers with evidence to most difficult questions.

33:40or like very good practical suggestions for what to do next in any given scenario. I think it's a very kind of weird concept. Streaming intelligence into every space is a great way of putting it. I think it's a good intuition for people to kind of try and grasp. And when it becomes general purpose technologies, we rebuild society on top of them, right? So your house is built with electricity first. So we're stepping into a future where electricity will be on that scale. It's going to be in the walls. The hospital gets redesigned around it. And that's when it starts to just really click in people's lives.

34:12And it's going to lead to new fields of medicine, just new behaviors. Like we shape our tools and they further shape us. And we're at the origin of what that looks like. And whether it's relationships, whether it's friends, entertainment, medicine, all of these continue to get reinvented with technology. But we really struggle to see the future through a new framework. Companies do. People do. And that's also where disruption happens, I think. Yeah. And you're right about like the metaphors that we've got are kind of broken. I mean, even the word consciousness, like it applies to arguably, you know, dogs and horses, maybe even octopus, like all the way up to humans and to, you know, it's just like we need more nuanced and precise and accurate descriptors because we're sort of like roaming around, like grasping to try and like make sense of this future that sort of is unfolding before our eyes without really having a language to fully describe it.

35:06Totally. And I think that that's right. And we'll start to invent new terms. Like we use the term, I'm running out of bandwidth. I need to stream this thing. These were all terms that we started to create new meanings for to match the technologies that we had invented. And so, yeah, right now, I guess because it speaks with human language and we have designed it in somewhat our image, it's easy to attribute human terms to it. But I think eventually those terms are going to come. And then that gives it more of a distinguished character than just we, I, system, all of us together. Even companion.

35:38Like, companion doesn't quite capture it. What does that even mean? It's like we don't use companion in everyday life. This is actually why I kind of picked it is because it was sort of not really an overused term. But then it also has kind of funky connotations to, you know, either a romantic companion or, you know, an old age companion or just got a lot of weird references. But it's because we're kind of unsure what this thing is that's evolving and emerging. And we have to just sort of do our best to kind of, you know, use the metaphors that we do have to kind of clip and shape and sculpt it as it kind of arises.

36:10And that will happen, I think, over time. And of course, I mean, I have to ask because people are really concerned about the girlfriend, the relationship thing. What is Microsoft's policy? Does it draw a line in the sand of someone trying to flirt with their AI companion? What does Microsoft do? Very directly reject it. Yeah. Yeah. I mean, you can give it a shot. I mean, the second you're mildly flirty, we check for any kind of dependency, long term usage. um you know and actually we actually kind of lean on the side of being ultra conservative we do actually get complaints because you know well at least a few a little while ago we were if you said oh i love that thank you that was so awesome you're the best you know it kind of starts to be a little bit wary just because you said i love that and some people would say i love you and thank you but that's the definition of safe personality design is constantly pushing back naturally a user is going to be a little bit frustrated and disappointed at times because it's pushing back and that's what trust is in my opinion it's and you know we sort of trust is really about boundaries and boundaries allow us to perform you know to reconcile our words and actions consistently over time and so then you have confidence that i'm gonna do what i say i was going to do repeatedly.

37:25And we've evolved different behavioral structures to cope with new technologies in the past. So one that I like to kind of point out is the corporation is kind of a new technology. It was invented to protect shareholders and insulate them from liability in the 1600s when people were going off and conquering other cultures and pillaging them and stealing. And that structure required, in order to insulate capital and shareholders from that liability, it required the invention of new human behavioral forms, the invention of the trustee, the director, the CEO, the accountant, the HR person. These are all roles that we literally just made up, just totally invented these behaviors and expectations.

38:13And we decided they were so goddamn important that we would go and spend hundreds of billions of dollars over centuries training people to behave in this way sculpting people to serve this capital infrastructure that was a choice that was a design choice that we made as a species and it produced an immense amount of value and you know i'm not kind of here to judge it but we've made profound transformations in the past to accommodate different types of technology and we should think about the invention of the legal structure of the corporation as a technology and now we have this new moment where for the next few centuries, we actually have to figure out a new way of relating to one another and to these new technologies.

38:55And we are super adaptive and resilient. We can completely reimagine, you know, what it means to be human in the context of these, you know, basically super intelligent systems. And we have to decide what we don't do with them. That's the goal of the next century, saying no to certain behaviors, or at least saying slow down until we figured out the consequences and add friction into the system so that they arrive at a pace that we can, you know, sort of collectively manage. I completely agree. I think it's so easy to forget how much even language being an invention and the jobs that we do, all of these were made up to suit the technologies of the moment that we're in.

39:33And we are in that redesign period. And that's why I think it is so important that people step into the conversation right now. Everything is being redesigned. We're in that industrial revolution moment, but with something probably more profound than the way that transforms society. What do we want from this technology and from this future? And if you don't declare that, or if you opt out of the conversation because you just have painted some view of what you don't want to see, then you're not able to steer things in any direction, yet alone a direction that you think is going to work for most people.

40:04But I think that that's also what's so exciting about the moment we're in. I mean, it is so rare to be a part of a general purpose technology landing in society. And we know that sometimes it can wreak havoc, but the way the world changes is fascinating. And then it changes to a point where we don't even think about it. We expect that we can plug in our cell phone to the wall or that our fridge is going to have power. Eventually it's going to be that way with AI and it just moves into the background and we don't think about it. It's a really profound moment that we're in. Yeah. And to kind of understand it, you have to use it and play with it without the judgment that comes from the kind of caricature fears or just biased, optimistic.

40:47Like if you're stuck in one of those two camps, you're just kind of missing all the nuance because it's so much more subtle than that. And to do that, the good news is this. It's not like you have to build a massive power plant to experiment with electricity. like you can literally just go and like type one query into a search engine access it on a website download it on an app and within literally 30 seconds you can be vibe coding you know just in natural language programming and creating a new working application right in front of your eyes like literally in a few minutes so it's never been more accessible that's also kind of a weird thing Like it's available to everybody on a messaging app on your phone.

41:29And that means that everybody's intuition is basically as valid as everybody else's. And everyone's objections and fears and preferences has to be part of the mixing pot of shaping these technologies. Completely. I mean, even when I'm working with policymakers, I try to remind them your lived experience qualifies you for this conversation. You don't need to think you have to run and get a computer science degree because this is a very social technology. It's already here. You're used to the applications that it's going to get built within. You are qualified to step into this moment. You just have to do that.

42:04And the same with everybody else. If you can operate a smartphone. I mean, to me, surfing the web, I wasn't really surfing the web. I probably would have been in diapers when it first came out. but that would have been more foreign than the idea of just talking to an AI system. Surf the web where? What surfboard? Where is this web? Where is this wave? But AI systems, they're much easier. The learning curve is much flatter. And I think that means it's going to move more quickly, which comes with its own challenges, but it's much easier to get on board. And when we think about the workforce, I mean, if I go to a client meeting, my smartphone, my email, my computer, clients expect that I have these things.

42:43So if AI is this next frontier, my AI agents and companions will be extensions of my professional self, wouldn't you say? yeah i think that you know your people are going to have an ai that learns who you are as an individual both as a person we've talked a lot about the companion side but also as an employee as a worker as an entrepreneur um and it's it's going to kind of fill in the gaps because like because ai is more like water or kind of like clay um it will augment and make up for the weaknesses that you have or the things that you don't like to do. Um, and hopefully leave space for the things that you want to grow and develop and that you feel are like unique to you and special.

43:31Um, and it can sort of just fit around the strengths and weaknesses that any individual has. And I mean, it doesn't quite yet do that, but a hundred percent for sure in the next like three or four years, it really is just going to feel like having a second brain. Um, You know, this idea that it lives life alongside you and sees what you see, hears what you hear. You're kind of like saving your thoughts, fears, worries, experience in this other kind of augmented part of your personal brain that, you know, you can either draw on or it can proactively feed into what you want to do to make you kind of more productive.

44:12And then it's also drawing, it's your second brain that's also drawing from all of the written and oral knowledge in human history. So it is this unique moment where you become a mini superpower in terms of what you could potentially be capable of. And I think we tend to just think of email as this thing that now we just so easily accept as part of us. Everybody has an address when you sign up to do anything that's with your email address, that AI is going to become that. And it's just going to be this expectation that you have this thing and that's how people can communicate with you. and that's what you'd go everywhere with.

44:43Yeah, exactly, exactly. And we already are kind of superhuman by having these phones in our pocket. I mean, the phone is truly an incredible thing. It has a large chunk of what you just described. The shift now is that the AI compresses all the photos, the videos, the text, you know, that has been digitized and is available on the open web into these like perfectly formed nuggets, synthesized nuggets, which you didn't get back in the day in the web. The web was more like kind of, it was a search problem. Like you had to do most of the search. It was kind of indexed, but you had to go. And obviously there's something beautiful about discovering and stumbling upon all these unique little things.

45:24But there's also just something incredibly empowering about being able to ask any question, getting a beautiful synthesized summary that you can then choose the rabbit hole down or not. And that I think is just a weird thing to wrap your head around. in the context of work as well, because it's like definitely going to make your job, especially if you do like a kind of office worker type job, white collar job, you know, it's kind of able to focus on all your work documents, what your colleagues are up to, like the entire sort of shape of your organization. And the things that we're going to do are going to change in such dramatic ways.

46:01I mean, when the way you describe AI companions and AI agents, it makes me think, what do I even want to do on my phone? Why am I taking this thing out and individually scrolling through applications? That makes no sense to me. In a world where these AI companions are operational and they are functioning at a reliable level, doesn't the computing platform go into those? Isn't that the next computing paradigm, AI agents themselves? Yeah, definitely. I mean, if you think about it you pull out your phone you're basically looking at a billboard of logos that are trying to sell you some experience or make a promise that things are going to be easier if you engage in these buttons in this way um and the buttons and the UI were invented because we couldn't learn the language of computers unless you're writing software um which most people can't and don't so we had to have this interface layer which was the GUI the graphical user interface and now the computers have got so much better that they don't need we don't necessarily need that interface anymore it will still be useful because it helps to see things visually and tapping on things can be more efficient than talking but the computers have learned to speak our language and so that is a profound shift um and the new layer you're right is kind of ai that's basically the next operating system and it will sit above apps and browsers and search engines and operating systems um and maybe it means you spend less time on your phone i mean maybe it does mean that you have earbuds that are ambiently aware that maybe have visual understanding that you can, you know, sort of sort of talk to basically most of the time.

47:54And maybe that keeps you off your display. But it's a different type of interaction with technology to anything we've seen. Yeah, I think the app layer is going to get quite compressed. I mean, you can even imagine with Uber, I don't need to be watching the car. If my AI companion is just giving me an update, it's two minutes away, get ready. But then all of the applications that depend on that visual interface, but in a world where we could choose, we wouldn't want to look at it. AI is going to collapse all of that. And why couldn't my AI agent, if I want to be tracking my runs for the week, it could just do that.

48:25I don't need some standardized application. Agent, make that for me. And you already know how to slot it into my life. Spot on. I mean, you know, a co-pilot, we're working on an AI browser. And what that basically means is that your co-pilot, your AI, should be able to do the browsing on your behalf. It'll be able to open tabs, type things into the URL bar, write queries, click on buttons, process all the information that comes from that, and do it in super fast time behind the scenes. So it's spawning tens of tabs, hundreds of tabs in a virtual machine in the background, doing the kind of research for you, then coming back into your co-pilot feed and synthesizing all that information generating novel ui that summarizes everything that you've been curious about researching or learning about whatever and that is it's so it's almost like the ai sits above the browser or sits on top of the apps on your phone um and that's what i've always really want to create is like an ai that is truly a trillion dollar ai that's on your team that's on your side that is kind of somewhat oppositional to those billboards that are trying to kind of sell you stuff and persuade you stuff that is like a a filter that is really aligned to your interest you know really looking out for what is going to be most interesting most useful most good for you as that individual consumer you know i think yeah that's kind of my aspiration it's quite tricky to do but like that's that's where i'm headed it will be true and i hope people understand that how profound it is what you just said, right?

50:03We're at the moment where that app store was invented and then that behavior happened. We're at another one of those moments and there'll be a whole new ecosystem for it. But then also if you were to drill down to the micro level, a field like marketing, what does it mean when somebody has a marketing bodyguard protecting all of the stuff that we don't want to bombard us with? And now you have a system that you're going to have to try to work around. So in today's age, a marketer tries to reach the human brain. A marketer in the next couple of years has to understand how do you market to an AI that's then going to maybe take that message to the human they have a fiduciary duty to.

50:42It's an entirely different skill set. Yeah, that's an excellent summary of the situation. In fact, I'm going to steal the bodyguard concept. That's a great way of putting it. It's a good metaphor. That's exactly the thing. How can we make sure that your AI has a fiduciary duty to you, like it's aligned to your commercial interest, then you can trust it to adversarially interact with all these other AIs and these other humans and scrutinize that information, produce the evidence base, ask the tough questions. And that's what I want. Like if I go to a doctor and I have a condition, I'm probably in a heightened state of anxiety, which means my memory is probably bad.

51:20I have to talk to this technical expert that's using all this funky language I've never heard about whilst I'm in a panic about my own condition. I've got 15 minutes with them and then they send me this two-page super complicated letter with like you know crazy words I don't understand. You know people who are lucky enough to have smart capable family members that are free or can even afford a patient advocate will take an aid with them to ask the questions to remember the facts to follow up on the actions from the referral note and so on and now you're going to have a co-pilot to play that function.

51:55That's amazing. So health queries are our top use case on co-pilot. Really? Number one. It's mind-blowing. And what are people asking? People, I mean, everything that you can imagine. Like, why have I got this rash on my shin? Will this type of food make me feel bloated? You know, am I about to get age-related macular generation? Why have I got this tremor in my hand? And people are taking photos of their doctor's notes, like an entire, you know, multi-page, you know, report and being like, explain this to me in language I can understand. What are the actual risks here? Can you give me a second opinion?

52:35Like who are the top specialists in my area that can help me make, you know, make progress on this condition? And that is just like tremendously empowering and valuable to people. It's probably the area that I'm most excited about at the moment. Health care. Yeah. Yeah. And I mean, Microsoft has put its stake in the ground that it wants to move to medical super intelligence. So this is just the early days of how an AI could help you. What does that path look like? What is a medical super intelligent AI? Yeah. I mean, what I'm really focused on is technologies that actually help make the world a better place.

53:10Like I'm not focused on super intelligence for its own sake. So how could we create a super intelligence in the context of medicine that could answer So any diagnostic challenge, basically perfectly, and then use that to coordinate your care, either in hospital or out of home, help you stick to your diet, to your weight loss program, keep taking your meds, et cetera, et cetera. So a few months ago, we partnered with the New England Journal of Medicine, which is basically an academic journal that comes out every week. One of the things they do is a kind of New York Times crossword for doctors. So they collect together a really complicated case study, a patient case study, and they print like seven or 10 pages worth of medical notes and x-rays and pathology reports and so on.

53:56And then in the following week, you know, everybody tries to guess, you know, what the condition actually is. So we trained a very cool model that we call DxO, the Microsoft AI Diagnostic Orchestrator. and it basically uses all the models the ai models from third-party api providers and it creates a series of roles so they're kind of like um sort of positions that are prioritized different types of diagnostic components so one of them will be like focused on getting it financially efficient like how can i get the best possible diagnosis with the minimum amount of expensive tests the other one is like okay what would be good for the patient here you know given their history, what are they really going to care about?

54:42The other is just like, what is the best possible medical expert opinion on this topic? And they all negotiate with one another and reason collectively to decide what intervention to take next. It's kind of a remarkable system. It really is like, you know, it really is the beginning of something quite different. So a panel of expert clinicians gets these case studies right about 20 to 30 % of the time. And this is like some of the best people in the world, the best humans in the world. Our model gets 85 % accuracy at about a quarter of the diagnostic cost. So it's doing fewer unnecessary tests, which is one of the biggest causes of inflated spending in US healthcare.

55:26So we have to validate this in clinical practice. So it's just still early kind of academic work, but it's very, very encouraging, higher quality, faster and cheaper. That's the triple aim that everyone's always dreamed about in improving healthcare. And I think these are the first signs that it's actually possible. And how many years before we can just be accessing the system, this team of kind of virtual medical intelligences? Well, we actually just signed a partnership with Kaiser Permanente. We have a number of other partnerships in the pipeline that we'll be announcing soon. and yeah we're racing to get this into production as quickly as possible i think it's a real breakthrough and you can see the pipeline quite clearly that in some ways the future of the hospital some tasks come to you yeah why do i need to go to a physical facility to wait four hours for something that's going to be a 10 minute conversation and then i'm going to wait five weeks for results that whole supply chain collapses and it comes directly to you obviously if there needs to be an intervention or there needs to be a second opinion, you would go somewhere, possibly.

56:34But this is an entire reconfiguration of the whole medical ecosystem. Most of medicine is going to come to your house. Totally spot on. The expertise is going to be commodified and made zero marginal cost and available to 7 billion people in the next 5 to 10 years. So the knowledge isn't going to be the asset. Previously, expertise was the gatekeeping asset that people charged for. I think it's a massively radical moment, a very exciting transformational moment that the expertise is now no longer what really counts. Everyone's going to have access to their expertise. And that improves the quality and accuracy of everybody because everyone has access to knowledge.

57:18so then the real thing that's going to matter is judgment and care and attention to implementing the solution like you still have to interact with the real world to operationalize the testing and the treatment and the you know actual physical care and there's huge huge gains to be had there but i think less time will be spent just doing the rote information exchange and it's an exact one-for-one comparison in my opinion to what it will be like in the classroom you know instead of opening a textbook and having your teacher essentially read out from the textbook for, you know, 50 minutes and then doing five minutes of questions at the end of the class, students are going to do most of their primary learning in a personalized, interactive, mobile-based or tablet-based experience, come to the classroom.

58:08And the classroom will be about using the knowledge that you've already acquired, learning to debate, learning to be self-critical, learning to give way to other people, the social aspects of learning and the use of knowledge are going to become what we do in the classroom and the kind of learning part will happen separately. It is such a big shift. And I think people might think about what we're losing until you remember why it was invented, right? The invention of the institution of education happened for the Industrial Revolution. That was a certain type of person that needed to be produced to go to an obedient manufacturing plant.

58:44And that's why we designed it in that way. And it worked for a world that has come and gone. So now we're reinventing something again, and expertise gets commoditized. It's an entirely different era that we're stepping into. And then even for the role of the doctor, it becomes something else. So an AI student that can do the diagnostics in some ways, then the nurse does the care. So nurse becomes this new new ecosystem. And then what happens to the physician, the competition, the intellectual competition, I think moves more upstream. So it's no longer about memorizing what's in the textbook. And I know that they do a lot more than that, a lot more.

59:21But it becomes an intersection with AI and synthetic biology and all of these new engineering disciplines that are going to converge in robotics. So it's not that the physician isn't relevant. It's to become a physician or whatever we'll call that job will require profoundly more intellectual capacity in a world of these systems and everything just moves upstream. Totally. And that's the creative challenge for everybody who is a user of AI. Like we're all obsessed at the moment with like, you know, me and the other big companies that are making these things, but that's going to pass quite quickly because we're not really relevant right now.

59:58The next phase is going to be how people use it in practice and take ownership of it integrate into their worlds if you have a domain expertise in healthcare or in education or in whatever it is that you do like that is it's now one of the most creative times that any of us have ever been alive because we what we've just described is like flipping the entire system on its head and it's very unclear what the next phase is going to look like uh certainly it's going to be me and the other tech people that design that Like, we're just creating the general purpose tools. We're streaming the intelligence.

1:00:29We're creating the fire and the electricity that's out there. And now I think people are finally, like, getting their head around the fact that it's actually going to be on everybody else to decide what the boundaries and limitations are, what the guardrails are, and how it, you know, fits in these new systems. Yeah, exactly. I mean, the printing press was to create more Bibles. And now look where we are. Like, how did that turn out? An entirely new world was born. There are a handful of people on earth that are in your position right now. Transforming the world with one of the most important technologies of our generation.

1:01:01What do you want your legacy to be? oh man i didn't realize you're gonna go there i don't know i think i think um if i have a chance of shifting the default trajectory from invention for its own sake to invention for invention of a humanist super intelligence one that is truly aligned to our interest that keeps humans at the top of the food chain that always serves us and works for us collectively in aggregate then you know maybe i'll i'll feel good about that that's that's maybe i'll be proud of that that's what i'm trying to do because i think that like there's a very narrow path that we have to tread and there are lots of ways that this goes goes wonky and right now we're making a lot of the design decisions that have ramifications that will last over many decades and I just feel a great weight of responsibility for that.

1:01:59Mustafa, it's been a pleasure. Thank you so much. This has been awesome. Thank you, Sinead. Thank you. If you enjoyed this episode, please like and subscribe and follow the show and share it with someone you think might be interested. I've Got Questions was created by me, Sinead Bovell. The show is produced and edited by Tara Cutts and Sandra Itainan and executive produced by Paola Piers-Torres. Artwork by Corey Vincent at Feel Studio. you