Madhumita Murgia: Exploring AI Safety

0:04Hey everyone and welcome to Generative Now. I am Michael Mignano. I am a partner at Lightspeed And this week on the podcast, I talk to Madhumita Murgia about the human consequence of AI, which she writes about in her new book, Code Dependent, which is out now. Madhu is a journalist based in London, and she's the AI editor of the Financial Times, where she covers how new technologies impact people and society. Madhu and I talk about the current impact of AI across the world and what guardrails builders and investors like you and I should keep in mind as we build the future.

0:39hey madu how are you hi thanks for having me on yeah thanks for being here really really appreciate you doing this i feel like this is a very exciting time to to speak with you first of all you're um you're the ai editor for the financial time so uh you're covering lots of really really interesting stories uh all the time right now and you also have a book coming out congratulations on that thank you thanks so much yeah so it's been out in the uk a couple of months and just out in the US now. So fresh off the presses for you guys. Awesome. Awesome. Congratulations. What's it like? Is this your first book?

1:13And if so, what's it like to now be a published author? Yes, it is my first book. And it's been a wildly different experience to being a journalist. It's kind of interesting because when you have to write something at 80 ,000 words, you have to just think about it so much more deeply than when you're engaging with it on a sort of day-to-day or even a weekly basis. And even though I've been writing about AI for more than a decade now, you know, since 2013, I've been writing about AI. This was like one of the most intense two-year periods, I think, when I had to kind of write at that length, forced me to kind of engage with it at a deeper level.

1:54So it's been great, though. And it's kind of interesting that it's opened up new spaces for me to talk about AI in. I think people see authors differently to journalists, still kind of see journalists as dabbling in things. Whereas if you've written a book about it, I think that earns you the right to have more opinions, which is fun. Yeah, well, it definitely sort of like cements you and cements your perspectives and your opinions sort of in the history book, so to speak, of a topic or of a subject, which is pretty interesting, right? I mean, especially as this space is naturally going to keep evolving so quickly, like this book sort of becomes a checkpoint on where we are right now.

2:39I actually like, one of the things that I feel like must be challenging and I'd love to hear a little bit about it and how you manage it is the fact that this space, AI, it's moving faster than pretty much anything I've ever seen before in tech. And obviously the process of writing a book, it takes a really long time. You know, everything I've heard, you know, it's like a two-year journey at least, or maybe 18 months. What has that been like? Like, how do you go out, decide to write a book about AI, and then everything changes every three months while you're writing? Yeah, I think the sort of cognitive dissonance of how fast-changing the technology is, and And as you pointed out, the very glacial nature of book publishing was it was a it was an interesting contrast for me.

3:29As I said, you know, I started writing about AI technologies in 2013. So I think it helps to have some perspective, right? If you have just started covering this topic or looking at it three years ago, then it probably feels really overwhelming and maybe kind of pointless to write a book in that kind of timescale. But for me, I felt like I had some sort of step back perspective and was able to kind of look at that arc and then think about what is it that I want to bring to this story that hasn't already been told. And of course, while I was writing this book, ChatGPT came out. So, you know, things changed pretty dramatically, literally, as I was finishing my first draft.

4:16Literally, I filed it in December of 2023. Sorry, 2022, just as ChatGPT came out. But I think what for me makes books lasting compared to a news story or an article or a blog, you know, it has to have something more than just like, here's what's happening, right? That's the point of a book is to outlive just the current moment. As you said, you know, it's a snapshot of history and it helps you think about things differently. That's the kind of role that a book plays. And for me, the focus had been on human beings. This book is not a book about technology. It's a book about people and how they use technology.

4:58And so actually it doesn't matter that the tech itself changes. All technologies are constantly evolving, right? I mean, you know this, you know, and but the way that we use it is the kind of interesting. It's almost like the sort of sociological story. And it will hopefully help us grapple with AI going forward as well, because you can kind of take those experiences and apply them. So that's kind of what I hope this book will do. It gives us that human perspective against a very quickly changing backdrop. So so maybe taking a step back, you mentioned you've been writing about AI since 2013. How did you get into it?

5:37How did you start getting interested in the space to the point that you were writing about it? And were you writing, were you covering for Financial Times back then, or were you somewhere else? No. So I used to be, I was trained as a scientist. I was an immunologist working on vaccine development, actually, but decided to make the shift to journalism at a point, you know, when I decided science as a career wasn't quite for me. And my first job as a journalist was with Wired magazine. And, you know, you'll know Wired, it has very sort of devoted following of entrepreneurs and amazing innovators and inventors.

6:16I myself didn't know the publication very well. I came to it as a sort of science nerd, which is fine. You know, we embraced all kinds of nerds at Wired. But tech was new for me. And I kind of thought I would write mostly about science and health, which were my areas of expertise and interest. But I kind of, it was 2012. So it was a really interesting moment in tech, right? It was when we really had sort of the mobile economy taking off the app economy and the birth and growth of many of the major companies that we know today, right? The Ubers and the Airbnbs, these were all tiny startups, even, you know, Facebook at the time, still small.

6:59It was its moment of like kind of explosive growth. It was when DeepMind had just been founded and literally as we were writing about it was acquired by Google. So there were all these things happening in tech, which meant I was witness to that at a very early stage and I learned as I went along. And that's also when AI was starting to become interesting again. You know, as you and your listeners will know, you know, AI has gone through these cycles of winters and summers over the last sort of decades. But in 2012, it was becoming more sort of interesting to entrepreneurs again, you know, to Demis at Deep Mind and others as well.

7:41And so I was just interested as somebody who'd studied neuroscience, And it felt like the coming together of that and tech. So very early on, I started looking at academics working on this and then just stayed with it from Wired to the Telegraph and then eventually at the FT. It must be a fascinating time right now over the past 18 to 24 months covering this space for FT. Obviously, so much has happened. You mentioned ChatGPT came out as you were writing the book. I mean, give us a little sense of what it's been like over the past two years. when it feels like there's literally a new, you know, jaw-dropping story in the world of AI, like almost on a weekly basis.

8:24Yeah, it's really interesting. It's hard to separate from having watched it all, but then also coming at it from the outside, you know, to get an external perspective to think about, like, was this inevitable, given what we've been seeing and writing about over the last few years? Or has this really come out of nowhere? Because for a lot of people who've only now started paying attention to AI, you know, the narrative is as if it's out of nowhere, but obviously it isn't. It's been built on these decades of work in terms of the algorithms, the data, the infrastructure, and the money, of course, that's been pumped into it over more than a decade.

9:03And so, you know, in some ways it is inevitable, but still, even as somebody who's looked at it for a long time, it feels kind of amazing and magical to see the leaps and bounds of the improvement of the technology itself. And in terms of covering it, I think what was amazing to me is like prior to 2023, I would write about it and we would always have, you know, our readers were really interested in AI as a topic because they found it kind of intriguing, you know, as this sort of cool, cutting edge thing, a bit like quantum computing, right, which I also covered and written about, where it's kind of like, don't really get this, but it sounds cool.

9:41And it sounds like we should be paying attention. And so that was kind of how readers framed it. But now, you know, the big change is everybody cannot nobody, you know, we just can't get enough of it. That it's like an insatiable appetite, people want to use it, learn more. So from a journalism, you know, from a journalist perspective, it's interesting, because it's just like, we just want more, more, more, more stories. Whereas prior to 2023, it was more me being like, guys, we need to write more about this because it's really cool and it's going to be a big thing. So the dynamics of that have really changed both internally and externally.

10:18I think the other big change is, you know, trying to separate the wheat from the chaff now a bit, because there is a lot of hype and pretty much everybody will come to you from every different industry, not just tech, right, insurance industry, retail, advertising, media will come and say, we're using AI for X, you know, the legal industry, we have law firms, consulting firms, everybody's trying to jump on this bandwagon. So a big part of my job is to be that person across the newsroom who, you know, sector experts will say, is this really AI or is this just something that they're repurposing, which is something more statistical, you know, or like straightforward statistics rather than machine learning or deep learning and to kind of separate that hype out from the real stuff.

11:04So I feel now more my job is to say, no, we shouldn't write that. Whereas previously it was kind of like, this is amazing and this is something we should write more about. How do you make the determination whether or not something's worth writing about when, again, there are so many exciting things happening on the surface, at least? Yeah, you know, this is the core competence of any good journalist. That is the whole job, right? It's how do you figure out what's a real story? How do you pick out the winners? I remember when I was at Wired, we used to do this series called 100 Hottest Startups.

11:41And we would pick different cities across Europe. It was European focused. So, you know, we would look at Tel Aviv, Paris, Berlin, and so on, and try and figure out what are the 10 hottest companies in this ecosystem this year that we should spotlight. We would get, you know, VCs and other people that are part of the ecosystem to nominate. And I remember then thinking like this, this is the hardest job because you could, there are so many great companies in all of these places and we could just pick any 10. but how you know we want to make this a definitive list um and like the process that worked for us then for me then and i think i still apply now is triangulation right it's and it's something that you know i did as a scientist as well like how how do you so you need to find the experts so you need to find as many of them as you can and they need to be across a cross section so it can't just be investors of course they're part of it but it's also the scientists who work in universities it's sector experts whether that's you know great law firms or whoever that is and and the people you know you need to identify the people who have real perspective across you know cross industry as well and once you find like you know 50 60 of those people it's talking to them it's like gleaning their expertise their sort of what are they excited about what trends what companies and then putting that all together and then doing that very human thing of sifting through it and making a judgment call.

13:15So it's not, there's no formula, I guess. And I think you'll know that, you know, as an investor too, you start to hone that instinct or whatever we call it, the sixth sense, but you need to first put in the groundwork, right? You need to talk to all the experts. And then you start hearing again and again, this, you know, this is the next really exciting thing or this thing hasn't quite worked out. So, you know, we're not too worried about that or whatever. So, yeah, it's I just try and talk continuously talk to more as many people as I can in the space, leaders, CEOs of all the major companies, try and check in with them as often as they have time for it.

13:54But also talk to entrepreneurs, you know, with starting new companies who are excited and building on top of the big platforms to to see like what are their pain points? What are they excited about? But yeah, we're learning too, just like everybody else. Right. So I want to get into the book in a little bit, but maybe while we're on the topic of your reporting at the Financial Times, I thought it could be interesting to start by just digging into some of the stories that you've written about recently, or maybe some of the things that are top of the mind. For example, it seems like you write quite a bit about OpenAI.

14:27I know you recently wrote about Universal Music Group and YouTube. Maybe what are some of the most interesting stories that you're thinking about right now that are very topical within AI at the moment? In terms of buckets, I would say we're really interested in the players, of course. And for us, that's the big corporate players because obviously we cover large corporations and our readers tend to be policy people, government leaders, and big business owners and people who run big businesses. So they want to know, how is this going to affect me? So we're really interested in the sort of Silicon Valley race dynamic, who's winning, who's losing, how are the companies comparing against each other?

15:10What are the sort of interesting political stories there? You know, I mean, within the companies and between them, what's happening between Microsoft, Apple and OpenAI, for example? Where does Anthropic fit in? What's Amazon looking at, you know? So and then Google, is it falling apart? Is it working out? Like, where is it going to end up? So the big corporate sort of, you know, the personal dynamics as well as like, how are they building a business out of this? That's something we really care about. I would say that the second bucket is the application layer, right? So we're at a point now where billions, you know, there's been reporting saying trillions of dollars going to be invested into AI to get us to the next stage?

15:54What are we building with it? And is it going to pay off? Because again, that's for us as a financial newspaper, big question is like, what is the business model, right? And there are some, you know, there are, you know, open AI has been, you know, revenues have been going up, they're making money, clearly people are interested in consuming the product. Same with, you know, Microsoft's co-pilot, but is the investment that companies are making in this technology going to be worth it? So I think I'll be looking quite closely at that, you know, what's the bear case as well for this? And, or do we, you know, do we think that someone like Nadella or Altman are right when they say this is just going to pay for itself?

16:36You know, and I think talking to investors will be interesting because, you know, I'm curious to hear a VC perspective on that too. And then of course, there's regulation is a big question and a really fascinating one at the moment. And that doesn't just include like sort of AI regulation. It's like other regulatory issues that including copyright. I think that's a really live issue. So the YouTube story kind of falls within that bucket, right? Who are the parties making deals? What are the size of those deals? So like how much are we valuing this type of data? And how is the law going to shake out on that?

17:15You know, we have live cases between the New York Times and OpenAI. And, you know, there will be others as well, not just in news publishing, right, in music and video and so on. So we're really interested where it crosses over with other multimedia too, like Disney and others. So yeah, and so regulation is a broader thing. Of course, the AI focused regulation will be interesting to see what the White House and other countries, China, India, etc. put out. So those are the three main things. But for me, the really important fourth one is the actual technology itself. That's what I'm really interested in personally.

17:56So like the cutting edge of, you know, the actual software. So where is this technology going? What are the big sort of breakthroughs that we're seeing, like that people are publishing in this area? What are the new capabilities that we'll be seeing with new generations of this model? And the path to sort of more intelligence or general intelligence? You know, how are we going to get there? And what are the various ways that people are trying to get there? I think that will be really interesting to see as well. So we'll be doing more explanatory stuff too. Yeah, that last one is interesting. You hit on a number of topics that I think are really interesting that maybe we can touch on.

18:39But maybe starting with the last one, the technology, you mentioned general intelligence. I saw, I think, either just yesterday or maybe it was this morning, you know, there was a story about open AI developing this new sort of internal system of measuring intelligence, like this one through five. You know, supposedly they're already at two. Like, where do you think we're at on that curve of reaching general intelligence? And what do you think, you know, some of the highest level societal impacts of that will be if and when we do reach that? Yeah, I mean, that's like the trillion dollar question, I guess, right?

19:16But also an open scientific one. It's not clear to me that anyone is really aware of the exact sort of levels and exactly where we are or how soon we're going to get there. And having spoken to some of the real experts in the space, you know, I saw that what you mentioned about OpenAI having this internal kind of ladder, but it's internal and there's no like external validator for that. There isn't any sort of evaluation mechanism or criteria that help. We don't have like the sort of criteria that tell us when we get there, right? And so anything that a company comes up with is just, you know, they're creating their own standards and then defining it by their own standards, which I guess is a start, but I don't see that as the sort of objective scientific measure of anything, personally.

20:10But I think, you know, in terms of where we are, again, this is what, you know, my understanding gleaned from having spoken to people in this space. It's people certainly are surprised by how far large language models have taken us forward. I think that, you know, the previous sort of understanding was more that you needed other types of intelligence that included sort of visual feedback, for example, you know, reinforcement learning from your environment in order to create general intelligence. And that purely like language wouldn't be enough to kind of create a breakthrough. But actually, it's been pretty good at simulating intelligence, and it's able to leapfrog a lot of what experts have thought was required.

20:54You know, Jan LeCun talks often about, you know, as humans, we can't just learn from reading. We learn from sort of physical feedback and visual and auditory feedback. And so he thinks you need that kind of a breakthrough before you're going to get anywhere near general intelligence. But language has been pretty amazing, as I said, at sort of simulating or leapfrogging. um but my understanding is we need at least one or two more major breakthroughs like of the level of a transformer or more before we're going to get there it's not just and it's not just a sort of um an obvious predictive path from here on to general intelligence i think it's going to look more something like that um and that's that's what we've got to wait for the sort of researchers to do.

21:41And I think there are lots of ideas out there, but nothing that's already worked. So I'm not sure if that's a cop-out answer. But based on how much investment... No, I think it's a good one. Yeah. I mean, it's going into... People seem to think a decade is a fair prediction, which I think a decade feels like a good amount of time considering how much money is being poured into this. You also talk about regulation as one of the main buckets and topics that drives a lot of your coverage. I think, you know, obviously, anytime there's a new technology, there's a big, there's a lot of talk about regulation, potential regulatory capture.

22:18And I think it's possible we're entering sort of a new phase of that, right? There are, as you mentioned, a number of lawsuits being directed towards startups. You mentioned the New York Times OpenAI case. You know, there have been a number of lawsuits from other, directed towards other companies like Anthropic from, you know, Universal Music Group and the music industry. At the same time, you know, you mentioned a story that you'd covered recently around Universal Music Group and YouTube working on a deal. And, you know, it strikes me a little bit that this may actually be another form of regulatory capture where the big players, the incumbents, the big tech, you know, are working together.

23:00Meanwhile, they're, you know, it almost seems like they're working to kind of crush little tech or kind of keep them out, you know, by weaponizing existing legal frameworks. I mean, is that sort of how you see it in your research and your reporting that we're sort of at risk in this technology wave of a new form of regulatory capture from big tech and the incumbents and our startups just going to sort of get left out in the cold? That's interesting. I hadn't so much thought of it from that perspective. I was thinking more of like sort of tech versus media, right, as the two sides of this. But it's true, you know, there's only so many companies that can afford to make deals with the FT, you know, News Corp with, you know, some of the biggest publishers and music producers around the world.

23:48Every startup isn't going to be able to go away and pay for that. And then if they then license all of the data and it becomes exclusive to them, then I guess it's going to be much, much harder to be a breakthrough startup that's trying to build something in this space, unless you're acquired by or somehow connected to one of the big tech companies that own these licenses. So I think you are going to see that kind of split. And I'd be curious to see how startups will get around that without the data that they need to train models and whether you just start to have, maybe that's the business model of the models companies, right, that startups are going to have to license or pay for those because they can't afford to build their own even small or medium models without paying off all the big, you know, data owners.

24:38From a reporting perspective, it's kind of fascinating and exciting, right, because it's a whole new industry. There'll be new business models and dynamics and new sort of applications born of that. But and I guess like from a startup perspective, like it's a challenge. There will be ways around it, but it feels like the big model owners are going to win for sure. But, you know, to look at the other side of the coin, it's a really expensive endeavor too, right? It's not as easy as just, oh, we'll pay you monthly for your models and that will be enough because it costs so much to not just, yeah, training is, I guess, the big cost, but even inference to kind of run the big data centers and to kind of maintain those models.

25:22So it's hard to know whether that will even be enough to, you know, to have a healthy business if you're just building these large language models. What about, you mentioned sort of tech on one side, media on the other side, obviously the approach that a lot of these companies are taking in these sorts of debates, it seems to fall along the line of sort of fair use and the definition of the law. And in many cases, it seems like the law is actually in favor of the tech companies and this existing definition. Do you feel like we have the right legal structures in place to guide this technology? Or do you expect that, you know, maybe the whole system needs to be sort of rewritten for this next phase of media creation and innovation via AI?

26:10Yeah, I think, I mean, I've spoken to a few different copyright lawyers about this, and I've never seen lawyers this excited, to be honest. they're just like this is amazing this is a an era of innovation for copyright law and genuinely they they feel like this is an intellectual sort of moment for for the work that they do because it's it's kind of a philosophical question of like if an AI system writes a piece of music or story like does it own that is it something that it could copyright or if it's being trained on something I wrote or something you wrote, is it just reading that and coming up with something itself that's original?

26:51Or is it essentially just repurposing and like regurgitating what we've created? And it's, you know, I think by current laws, as you say, you know, the fair use, you know, it seems to work in favor of tech companies. And, you know, the argument is it's not just regurgitating. These models are sort of creating their own new work based on this. And it's the same as sort of reading this work. I think there's going to be it's going to be a lot less simple than that. I think there will be change in the law here in the UK. The intellectual property office was sort of trying to reconceive this whole thing.

27:29And when generative AI burst into it, they've paused the whole thing and they've done a new, you know, they've basically what they call a consultation, where they've gone to all of the experts in the space, which includes artists and so on, and lawyers to say, OK, maybe we need to come up with a new law that reflects this new technology and where the data is coming from. But there's a lot of opacity around where that data comes from. So I think it's hard for anyone to really have a view without A, understanding what the data is, and secondly, B, how this works. What does it actually look like when a deep learning algorithm repurposes or remixes the words that it's learned from and create something new, right?

28:14It's until we can really see that, it's hard to determine, I think. So it'll be really interesting to see how it pans out. But I think there will be change. Okay, let's talk about code dependence. So this is your first book we talked about at the beginning of the episode. Again, congratulations. I have a number of specific questions about the book, but maybe to start things off, why Why don't you give all the listeners just sort of a brief overview and synopsis of what you're showing, the message you're trying to convey with this book? Yeah, sure. So for those who've listened to the first half of this and think this is a book about the business of AI, it is not.