Debunking AI myths, with Jeremy Kahn

0:00Starting a business comes with its share of ups and downs, which is why staying true to your vision is essential. a non-negotiable for Romeo and Milka Bregali, Capital One business customers and co-owners of Ra's plant-based restaurant in New York. Romeo and Milka took a leap of faith when starting their own restaurant, gutting an empty space and building it from the ground up. Every pipe, every wall, every detail. But building from scratch came with a heavy financial burden, which is when they turned to their Capital One business card. With the flexibility of the card's no preset spending limit, they were able to spend more and earn more rewards while bringing their vision to life.

0:36Today, Raz's success is proof that with passion and the right support, it's possible to make your dreams a reality. Learn more at CapitalOne.com slash business cards. Pioneers of AI is made possible with support from Inflection AI. It's not just enterprise AI. It's your enterprise AI.

1:00I have been covering AI for eight years. But for a long time, I was covering it alongside other technologies. I was just very fascinated by the subject, but it didn't seem like something you could quite make an entire beat out of. That's Jeremy Kahn, longtime tech journalist and Fortune's AI editor. And then with ChatGPT's debut in November 2022, the whole space has exploded so tremendously that it is now a full-time beat. In fact, it's not just a full-time beat for me, but it's a full-time beat for myself and two other reporters at Fortune. And there's quite a number of reporters at other publications who just devote themselves to the subject of AI.

1:42But ChatGPT didn't just create new opportunities for journalists like Jeremy Kahn. It also opened Pandora's box, and people started asking big questions about the future of AI. What is AI? What does it all mean? And where is this all heading? Exactly. These are the questions we need to be asking. But I have to say, when it comes to the answers, there are a lot of misfires. There's the misconception that AI is destined to take our jobs, or that AI will reach superhuman intelligence and take over the world. And there's the idea that AI will replace our human relationships. And while this AI doomerism is prevalent in our society, it isn't necessarily the reality.

2:28I'm a big believer that if we put people at the center of our decisions when developing new AI technologies, we'll create an AI future that benefits all of us. So on today's show, we're dispelling some of these AI misconceptions. Jeremy will talk about his new book, Mastering AI, which gives us a bird's eye view on what's in store for our future with AI. He'll also share his honest concerns and how we can use AI smartly moving forward. I'm Rana El-Khalyubi, and this is Pioneers of AI, a podcast taking you behind the scenes of the AI revolution.

3:21So Jeremy, thank you so much for being on the show. Oh, thank you so much for having me, Rana. I love that you're in the interviewee seat today. Yes, it's unusual for me. I'm usually the one asking the questions. Yeah, you're usually the one asking all the tough questions. So what was the impetus for writing Mastering AI? Yeah, so after ChatGPT debuted, Fortune assigned me a large cover story about OpenAI and the process that had led to the creation of ChatGPT, which came out in January, 2023. And after that story, I got a lot of interest from people saying, you should write a book on this. And I had always wanted to write a book on AI.

3:56I thought it was a fascinating subject and it just seemed like sort of the right time to do it. But I wanted to write a broad book. I wanted to talk about all the impacts I thought AI was gonna have on both business, but also broader than business on society and on ourselves and on democracy and on culture. I saw all these potential implications for the technology. And I love that about the book. So you write about how this moment in AI is different in the book. AI is what we call a GPT, a general purpose technology, which is, by the way, not the GPT and chat GPT. The GPT and chat GPT is generative pre-trained transformers.

4:30But yeah, we're talking about AI as a general purpose technology like electricity. What makes AI different from all these other earlier innovations? So I think there's several things. One is it is a general purpose technology and those don't come along all that often. It's much more like the invention of electricity, which is also general purpose technology or steam power or computers at large than it is the mouse or the graphical user interface or something, which were much more specific technologies. So that's one reason it's different is it impacts a whole breadth of things. But I think the other thing that's really different about AI is the speed at which it's developing.

5:05So the capabilities are being pushed extremely quickly. And we've already seen, you know, the capabilities of models expanding very rapidly. The technology is going to be adopted very quickly, much faster than we saw with something like electricity, which took quite a long time to be adopted by business. And it required quite a lot of physical rewiring of houses and all this sort of things. But then the other thing is the speed of adoption by business is very rapid. So that's the other thing I think is going to really make this different. You also talk about how AI is kind of encroaching on what makes us uniquely human.

5:39Right. And that's another way where it's kind of really powerful, but also scary in a way. Absolutely. Yeah. No, I think that's another differentiator between other general purpose technologies and AI. I think AI challenges our intellectual abilities. and this idea that we are the smartest species, the most intelligent species on the planet, suddenly we have this thing that we've created that is potentially as smart as we are. And I think that is a real challenge to people. It's a real philosophical challenge to people. I think it creates a lot of anxiety. I think some of the fears around AI really stem from that.

6:09That sort of existential dread of something else out there that is as intelligent as we are, but that is not human. People find that scary and they worry about, well, what's my place going to be if there's this software out there that can do everything I can do? Yeah. I think for a lot of people, when they hear artificial intelligence, they think about sci-fi movies. So they associate AI with, I don't know, Terminator. Oh, yeah. Terminator, 2001. There's like lots of them. Yeah. Exactly. And they are thinking this autonomous, free-thinking creature or being. So I want to talk about another sci-fi movie, which is one of my favorites, the movie Her.

6:45Oh, yeah. That's a good one. Her is one of my favorite movies. The protagonist, Theodore, is played by Joaquin Phoenix. He develops a romance with his super smart, emotionally intelligent operating system, Samantha, voiced by Scarlett Johansson. It's a modern love story in its own right, and the movie explores the themes of heartbreak and loneliness. It also paints a portrait of what our future relationships with AI could look like as they become more emotionally intelligent.

7:23Jeremy doesn't think we have this kind of AI yet, but he thinks that we're close.

7:32We're sort of right on the cusp of that. I mean, the demo for GPT-4.0, which was one of OpenAI's latest models, was very much clearly they tried to design the system to be a little bit like Samantha in the movie. And there's this whole controversy about whether they deliberately tried to mimic Scarlett Johansson's voice or not. Whether they did or not, the point is, I think they were very much influenced by sort of the vision of that movie when they were thinking about the interface with which people would interact with GPT-4.0. And we are moving towards a vision, I think, pretty soon, certainly within the next couple years, where we're probably going to have these kind of personal assistants on our phone.

8:04They'll be useful to us, but we're going to have to give them a lot of information about ourselves in order for them to be useful. And then the other thing is, I think we're already at the point where people are developing emotional attachments to AI chatbots. They're doing so actually kind of deliberately. If you look at what's happened with one called Replica that's out there, it's a service, you can create chatbots with various different personalities, but a lot of people have chosen to create ones that kind of serve as a romantic companion or even a kind of erotic companion. And people have developed a real bond with these AI chatbots, so much so that when Replica at one point turned off the ability of these chatbots to have sort of spicier conversations, a lot of users were really up in arms.

8:43They were really upset. They felt bereft, as if they'd had a lover that died. And I think it shows that we're already, to some extent, in the world that the movie Her envisioned, which is the idea that people would develop these really emotional attachments to these inanimate pieces of software. And I think it's slightly troubling. I write in the book, one of my big concerns is that people will find relationships with an AI chatbot just so much easier than a relationship with a real person. You know, the chatbot has no actual needs, which you have to fulfill. It has no wants. It has no real desires.

9:13Anybody can pretend to have these things, but it doesn't actually have them. And so there's no real consequence if you don't fulfill those. You can always reset the chatbot if it gets annoyed with you or whatever. It's totally different than a relationship with a real person. And I worry that people will use this as a crutch. And I worry that people will allow their real social skills to degrade and they will choose the kind of easy, slightly addictive, I think, conversations with AI chatbots over the hard work of real human relationships. Would you let your kids have AI companions or AI friends?

9:46I feel a little worried about it. I think it was okay to have an assistant, but I think we should limit the extent to which we're using them. we need to be kind of cautious. I think there should be definitely some guardrails around how often you let your kid interact with an AI chatbot, what the nature of that conversation is. As I write in the book, a lot of the studies have sort of compared having an AI chatbot versus nothing. And it does seem like some of the data is that if you really are a lonely person, that if you have no one in your life to talk to, that having the AI chatbot is potentially a positive, that it does provide some emotional uplift and perhaps there's some mental health benefit.

10:23And the problem is they haven't really compared it in any case. There's not been a control study to having human friends. But if you already have human friends and then you add this AI companion, I wonder what happens. And my concern is that maybe you let your human relationships atrophy your bit or you don't work too hard on them. And I don't think that's been studied yet. And I see a lot of these companies kind of being in a race to capture that intimacy, build an intimate relationship with the users. And there's no guardrails today. Yeah, no, absolutely. And I think we've already been through this experience with social media where we have, you know, basically inadvertently allowed these companies to gain a tremendous amount of power over not just, you know, what we do online, but actually over our minds.

11:00They capture a lot of our mind share as well as our market share. And it's particularly had this pernicious effect on children. I don't think we should wait with this next generation, with this platform shift to AI assistance. I don't think we should wait to see what the ill effects are. I think we should assume, particularly when it comes to children, that there will be some ill effects of creating products that are particularly addictive, like an AI chatbot. I think we're going to need some rules. I think we're going to need some regulations around this to try to safeguard people from the overuse of an AI system.

11:28So let me switch gears a bit. So the holy grail in AI is this idea of an artificial general intelligence or AGI. And of course, companies like Google's DeepMind and OpenAI were founded with this goal in mind. Can you talk a little bit about what is AGI and make a prediction on how close we are? Yeah. So AGI or artificial general intelligence is the idea of an AI system that would have all of the cognitive abilities across the board of a human being. And it has been kind of the holy grail of the field since the very founding of the field. But we've never actually been that close. I think we're closer now than we've ever been before.

12:04But I don't think that that means that we're sort of AGI is imminent. There's still a lot of things these systems can't do that we haven't quite figured out. And I think it's actually going to take some algorithmic innovation to probably get there. Algorithmic innovation, meaning that we will likely need a new kind of machine learning algorithm or a different approach altogether to reach human-like intelligence. For example, AI today doesn't have much perceptual abilities or even memory. Right now, a lot of AI models are based on transformers, a powerful architecture critical in developing models like ChatGPT.

12:41But if we want to level up our models, we probably need to level up the algorithmic architectures behind them. If you go back a few years, there was a definite school of thought among a certain group of AI researchers that all you needed was to keep scaling up the current large language models, which is what the systems that kind of underpin ChatGPT. the idea is that you could just keep growing these things, get them bigger and bigger, and they gain more capability. And then all the things that they can't do so well right now, which includes complex reasoning, planning, a lot of common sense things that they don't really have, that those abilities will somehow magically appear as a function of scale.

13:15I think very few people think that's the case anymore. Most people think we will need some sort of further algorithmic innovation. I think it's very hard to predict when that innovation would happen, but I do not see it imminently on the horizon in the next three to five years. I think if you went out 10 years, perhaps, or certainly if you went out 20 years, it's possible we would have systems that reach that capability.

13:43We're not in the year of AGI yet. But there is still a lot of innovation happening when it comes to artificial intelligence. So what's the next big AI innovation coming down the pipeline? That and more after a short break. Stay with us.

14:05If you've spent any time building AI products or leading technical teams, you know this. Transformation doesn't fail because of ideas. It fails because teams can't move together. Enter Atlassian's Teamwork Collection. It has planning in Jira, documentation in Confluence, video updates in Loom, and now AI agents in Rovo, which connects the dots across your work so nothing gets lost. It's one AI-powered teamwork platform designed for how modern teams actually build. Learn more at Atlassian.com slash TeamChanger. That's A-T-L-A-S-S-I-A-N dot com slash TeamChanger. 2023 was dubbed the year of AI chatbots, and 2024 is poised to be the year of AI co-pilots and AI agents.

15:01So I just want to take a moment to define those terms and create a distinction between them. I'm guessing a lot of our listeners have experience conversing with an AI chatbot. The way I like to think of an AI co-pilot is kind of a chatbot that can collaborate with you and be a thought partner and kind of help you ideate and get stuff done. And then an AI agent is something that you can delegate a full task to and it can go act on your behalf and get it done. Would you agree with that distinction and share some examples? Yeah, I think of co-pilots particularly as something that's been trained to help you perform a particular professional task.

15:38You know, you have a co-pilot for lawyers and you might have a co-pilot that's different for accountants and you might have a co-pilot that's slightly different for people who work in finance or medicine. They can give you advice, they can critique your work, you can do sort of first drafts of things for you, potentially act as a bit of a mentor and coach in that professional role. Whereas yes, I think the idea of AI agents are software systems that would actually go out and perform tasks for you across the internet. And you could give it a goal, and it would potentially come up with sub goals needed to achieve that primary goal.

16:09And it would then kind of figure out what tasks do I need to do in order to accomplish the goal I've been given, and it would go out and start to actually perform those tasks using other pieces of software or using an internet browser. We know that both these things are pretty close. So lots of co-pilots are being designed right now and being rolled out right now to businesses. AI agents, we don't really have yet, but everybody says they're working on them. And there's some indication that they may be debuting even before the end of this year. I think certainly into the early part of next year, we may see a lot of these things getting rolled out.

16:39It's unclear to me exactly what capabilities they will have initially, but I do think they will be able to do some purchasing for us. There's very likely that they'll be able to go out and like book a restaurant reservation for us or book some travel for us or do some calendar scheduling for us. I think we're right on the cusp of agents that will have at least those abilities. The problem is, you know, anyone who's interacted with a chatbot knows they are good, you know, maybe 80 % of the time and then 20 % of the time they provide answers that are really not so great or they seem very confident, but they're wrong.

17:11And I think when we move into the world of AI, agency, this becomes even more problematic because if it's going to take actual actions with consequences across the internet or using other pieces of software, we're going after a way of checking that what it's about to do is really the right step. And exactly how often it's going to come back to us and ask, okay, now I'm going to do the following. Is this correct? I mean, if it does that too often, it sort of defeats the purpose of having the AI agent. If it doesn't do it at all, I think there's a real danger that it will run amok and do things improperly in the end of spending money, actually, that you probably shouldn't spend or performing some action with terrible consequences you don't want it to perform.

17:47You know, that is a danger. And I think companies are going to have to think very hard as they deploy these co-pilots about what systems they put in place to make sure that the people that are using these co-pilots remain vigilant and they can perform that kind of supervisory role. And so actually in the EU's AI Act that is now in full force is that if you have a human interacting with an AI system, the human should always know that they're interacting with the AI system. And I think that's right. I mean, I think we'll all get used to sort of dealing with these AI systems the same way we already are used to dealing with the very kind of primitive systems that were voice-based where you used, you know, robo-callers and you would hit one on your keypad if you wanted to do something.

18:22And I think we're already kind of used to that. And I think we'll get used to this interaction with AI agents as well. Yeah, I agree that AIs should always disclose that they are AIs. The AI agent I want, and this is like a true story, I need to schedule my colonoscopy and I'm just dreading having to sit on an hour call, like waiting for somebody to get, you know, answer me and like schedule. I'm like, can we just have an AI agent that has access to my calendar, my medical insurance data, and can just get it done? Right. When will we have that? So I'm kind of hopeful we'll have that certainly within two years.

18:56It's also, if you think about just scheduling as a lower risk, I mean, you can imagine scenarios where scheduling could go very wrong and have some very negative consequences. But in general, you know, if you put something in the wrong calendar spot for somebody, it's usually not the end of the world. It's like, how much autonomy do you want to give these agents? And especially if they have access to a lot of your personal information. Exactly. All right. So let's transition to what this all means for the future of work. And as AI becomes more and more capable, what are the implications on various industries and various jobs?

19:26And just to kind of frame this, I'm a big believer that AI has the opportunity to augment and amplify and unlock human potential if we do it right? First of all, I totally agree. And one of the big pleas in the book is to think of AI as something that's going to augment human potential and think of as a complement to human labor and not as a replacement and not as a substitute. And I actually think if we can do that and if we can convince business to do that, we actually take a big step towards mitigating a lot of the near-term risks from the system. Because if you think about a lot of the things that could go very wrong in the near term with this technology, essentially given the capabilities we have today, they almost all come from the idea of taking the human out of the loop.

20:03They come from the idea of substituting human labor one for one with an AI system. And I think it can be a tremendous compliment, but you're still gonna need a person working right alongside it most of the time. And it can automate certain tasks, but it cannot automate full jobs right now. And so it's wrong to think about it that way. So I hope businesses can kind of get away from that temptation. I think instead you think about, okay, you can make the person more productive. And if you make the person, the human employee more productive, what else can they do? Or what are the things that a human can uniquely do really well that the AI system can't do?

20:34And can we get the person doing more of that now? Or can we expand what they do into some other area that's adjacent to their current job? Again, where the human is going to have a real advantage. And a lot of businesses are going to be able to run themselves more efficiently. They're going to be able to grow faster. Labor productivity has been a huge problem in most of the developed world. Most of the developed economies, labor productivity has not been growing fast enough. And I think AI technology has the potential to finally kind of get that back on a very healthy upward growth trajectory.

21:02And I think exactly how it's going to impact different businesses or different professions will depend a little bit on the specifics of that profession or that business. I think there's a potential in some professions like law or accounting or finance, you might see more boutiques actually being spun off because some of these higher earners will think, oh, I can go off and create my own firm and we can actually use AI systems for all the support staff that we currently need. Because a lot of these top earners, they're kind of held in place to these large firms because of the need for that massive kind of support base.

21:33So at a personal level, I love Jensen, the CEO of NVIDIA's quote, it's not going to be an AI that takes your job. It's going to be a person who knows how to leverage and harness AI that will. And in your book, you talk about, we spend so much time thinking about how we're training these AI models, but not as much time thinking about how do we train and upskill people to work effectively alongside AI. Absolutely. And I mean, that's one of the main points of the book. I feel like we spend so much time talking about, yeah, model training and almost none talking about training people. We should try very hard not to just have the human in a completely vigilant role where all they're doing is babysitting the AI system's output.

22:10I talk in the book, I spent some time with this woman, Jessica Marquez at NASA, who does a lot of their research on what they call human factor engineering, like how do humans work with automated systems. And it turns out if you put a human in this babysitting role, they almost always perform very badly. They can't remain vigilant for that long. So you really need to give the human something to do. And it really has to be a collaborative process. The human still has to be doing something And as long as the human's doing something meaningful, they can kind of remain alert. Also, we may have to do things where we do drills, just like they do in aviation, just like with astronauts, where you simulate some kind of failure from the AI system.