The humans training AI to be better

0:03I got sent a present by Microsoft. Oh no, that's not a Microsoft jumper, is it? It's a Microsoft Christmas jumper. Oh God. I don't know if you can see. Of course it's got Clippy. It's got Clippy on it. It's got a Minecraft thing. Can you see? Internet Explorer. You get the worst swag. No shade.

0:27Hello and welcome to the Times Tech Podcast, where every week we unpack how technology is reshaping business, culture, and everyday life. I am Danny Fortson out here in Silicon Valley. And I'm Katie Prescott here in the city of London. And in this episode, we're going to dive into what's really being seen as the jackpot in the world of tech, artificial general intelligence or AGI, and the bitterly fought race that's going on to build it. We'll look at what it is, what it takes to get there, the algorithms, the chips, and most importantly, of course, the many, many humans in the loop. Always humans in the loop.

1:05Always, always. Yeah, so our big interview this week to talk about all of that is Edwin Chen. He's a former Google and Twitter data scientist who now runs Surge AI, a company that uses human expertise to train AI systems. They say their mission is to, quote, raise AGI with the richness of human intelligence. Terrifying. Really great interview. I'm looking forward to that. And we're also going to be talking about one of the original tech giants, not the one on my Christmas jumper. No. It's big rival, Apple, as it approaches its 50th anniversary. Can you believe it? Makes me feel so old. I do. Doesn't it just?

1:47It's seen a series of high-profile departures and lots and lots of questions over its own AI strategy. So we're going to be asking, has something gone wrong at Apple or is there still plenty of juice? I see there. How many bars on the battery? Exactly. Left in the world's most valuable foam maker. Yes, but first, before we get to Edwin and Apple, let's start with the basics, shall we? What is AGI and why are so many people convinced it is just around the corner? How long have you got? But when people talk about AGI in very, very rough and broad terms, they mean this sort of hypothetical AI that can understand and learn and perform any intellectual task that a human can without being limited to one specific skill.

2:43So it's almost like, I guess, human-like intelligence. But of course, as everything in tech, it's not actually that straightforward. And lots of different companies and people interpret it in different ways. Yeah, that's right. So if you ask 10 different people, tech experts, what AGI is, you'll get 10 different versions. But invariably, people see AGI, they call it as the final boss, if you put it in gamer terms. Yes, exactly. Another way it has been referred to me, which I think is quite pithy, but kind of gets to the point is humanity's last invention. Ouch. Because, of course, once you build it, we're kind of done.

3:20It's going to do everything for us from that point forward. It's going to be better than us at everything. So it's a really powerful and a very risky technology. You need strict safeguards. Maybe it's an overhyped idea that may never materialize. Maybe it's a huge economic and geopolitical force. It's going to be weaponized or regulated. Maybe it's a philosophical question about whether machines can ever truly think, quote unquote, or be conscious. So just to give you a quick sense of when you talk about AGI, everybody looks at it a little bit differently. It all sounds a bit wild because, of course, it is.

3:54It's worth saying, though, that some companies have actually got written definitions of it. So, I mean, like OpenAI, for example, is one of those. And it calls AGI highly autonomous systems that outperform humans at most economically valuable work, which is quite a pithy way of putting it. Yeah. And as we talked about before, that AGI definition, at least for OpenAI, is critical. It is written into this contract it has with Microsoft, and they've agreed to set up an independent panel to decide if and when they reach AGI, because once they do, then there's a kind of a splitting of the corporate sheets between those two companies around kind of what Microsoft can access.

4:34They have full access to everything up until you get AGI. And then after that, they have a little bit of a kind of a conscious uncoupling from their corporate marriage. So the race is really, really on to hit that rather nebulous term. And when it comes to when this might happen, it feels like estimates are just all over the place. So some people say 50 % chance that it might come sometime this century. and others think that it may be as soon as the 2030s and others say just might never happen at all. Yes, you just spoke to Jan LeCun about this. And Jan, of course, is the recently departed AI chief at Meta.

5:16And he said it was not going to be a moment, so to speak. It was going to be a slow evolution, that there's no such thing as AGI because human intelligence itself is highly specialized. so he prefers to talk about human-level AI. And he thinks we're still pretty far from that. Yeah, I mean, he was quite categorical about that at the time. And he said that LLM-style systems, so the large language models being created by the likes of OpenAI, are just not going to get us to the point of AGI anyway. And as you can hear, dear listeners, we're going down the rabbit hole of all of this because it's incredibly complicated and everyone has such different views on it.

5:58But just to come back to that original point, it is seen as the ultimate jackpot in tech at the moment. And it's a scientific breakthrough, yes, but it also unlocks untold riches for those who get it, like OpenAI and what it does to their relationship with Microsoft, but also massive geopolitical power. So it really is the holy grail that everyone's chasing. And I think it's worth just lingering on this whole concept because again, it serves a few purposes. One is it's the marketing. It's the kind of the pixie dust that Silicon Valley has sprinkled throughout this whole AI bubble. It's really kind of a fuel of this AI bubble.

6:35Be like, look, we're building this thing that is going to result in effectively millions and millions and millions of digital workers, as Dario Amadei said, a country of geniuses in a data center that are going to be able to do basically anything you or I could do, but better. And so when you think about just like the economic prize for private companies is kind of incalculable. But also what does that mean for society? And we've had lots of people talking from Sir Demis Hassabis to Sam Altman about like, guys, things are about to get really weird and things are going to get turned upside down because we're building this kind of digital God that's going to be better than us at everything.

7:15And then there's that question of how you even do it, which I think brings us really neatly onto today's guest. Because if we are going to build something close to AGI at all, the question is what kind of data and what kind of human input do these systems actually need to work? Exactly. And, you know, on this pod, I think rightly, we've spent a lot of time talking to the people who are kind of building, the developers, the open AIs, the Googles, the Anthropics, who are building these systems. But this week's guest is Edwin Chen. As we mentioned, he's a former data scientist at Google and Twitter.

7:48He was recently named amongst Time Magazine's 100 Most Influential People in AI. And that's because he's built this company that most people will not have heard of. It's called Surge AI. It's become massive in just like five years with zero VC money, which is basically unheard of. But he made a few million as a coder, as you do in Silicon Valley. and he started this company that is now supposedly worth$30 billion, meaning his 75 % stake is worth a cool$18 billion, roughly. It's unheard of in Silicon Valley. It's really unheard of when you're talking about AI, when you think about the amount of investor dollars that are flowing into it.

8:31What Surge AI has done is build this platform that he says basically draws on over a million people, mostly in America and Western Europe, English-speaking countries. To teach AI models, not just labeling cat versus dog, but testing them, probing their weaknesses, giving them feedback, drawing on kind of an army of PhDs and white collar professionals who kind of know what good looks like. Yeah. To get to kind of take these kind of chatbots from the ones that were telling us to put glue on pizza three years ago to these things that can, you know, solve the most difficult physics problems. And I guess the idea there is the better the input, the better the output.

9:12Exactly. Yeah, I think what they are framing that might be useful as we go into this discussion is like, you know, they've assembled a team of a million specialized AI coaches. Basically, nope, wrong. Yes, good, bad, do better, whatever it may be to get us to this kind of AGI moment. So we should just hand over now, but I should also say I've been doing some additional reporting on the story for a piece we're going to be running in the paper in the next week or two, which adds some more meat on the bones of what he and I talk about so we can pick up on that after the interview. But I wanted to start by asking Edwin to explain in human terms what his company actually does and why he thinks better human data is the kind of the key bottleneck to AGI.

10:01Welcome to the pod. I guess I wanted to start with, for kind of the folks at home, if you could explain why it's important when we think about AI and this race toward these ever more intelligent systems. What we do is we basically teach AI models and then we measure how well they're learning. And we do that by bringing together the smartest people in the world to teach these models. And then we combine them with this incredibly sophisticated technology that we build to make sure that they're doing a really good job like one of the things they often say is that we want ai models not just to be book smart like we don't want them to just be learning from books we want to be them to be incredibly smart at the real world we want to be street smart and so one of the things we do is we train ai models to be curious and funny and to understand all this ambiguity in the world and we wanted basically to be the kind of ai that you would call at 3 a.m and not just when you want to pass a high school test.

11:00So it's really this aspect of infusing AI with all the intelligence that humans have, all this like rich human experience. Is it accurate to say that you guys have something on the order of a million-ish people kind of labeling data and testing these models? Yeah, so we have over a million people on our platform. The funny thing is we actually have, But what I believe is the largest group of PhD experts in the world on a platform teaching these models. I mean, one thing I would say is it's a lot more than just data labeling. We have literally Harvard professors on our platform. They're not just going and categorizing a piece of text.

11:43They're not just going and saying whether this image is, you know, a cat or a dog. What they're doing is they're using all of this experience and education and just like real world life that they've had over the past 30 years, teaching students, doing their own research, and they're basically giving that same experience to the models. So on the one end of the extreme, the most basic is a bunch of people in, I don't know, Philippines or some other part of the world labeling basic like this is a cat, this is a dog, this is a tree, this is a banana. And on the other end, you have your Harvard professor, or your astrophysicist would they be testing the big models gemini or chadgiputti or whatever being like ask them a question see what the answer is and if they get it wrong telling them what the right answer is or to change their answer or that kind of thing yeah essentially so we basically don't operate on the on the first spectrum at all what we're doing is really pushing the frontier of all of these elements and so yeah you've probably interacted with them yourself and you see just how incredibly intelligent they are like it's actually very challenging oftentimes to find what lesson the models need to learn like what piece of information they don't already have what type of reasoning ability they don't really have kind of writing ability don't they don't really have and so like first of all you need like these incredibly smart humans to go in and probe the models to find their weaknesses and then yeah once you find one of these weaknesses i mean there's a bunch of different forms of teaching and training these models one of the forms might be like let's say you asked to write a poem or asked to solve a math equation and it didn't do it correctly one of the things you might do is demonstrate the right approach so that's only one in many many different approaches that we use there's lots of companies labeling data that are then fed to these big model developers is your secret sauce to kind of like we're gonna just really provide you super high quality in-depth layered information that you're not it elsewhere?

13:41So first of all, absolutely. So we focus on quality at the end of the day. We've always said quality is a number one principle. We were basically founded because we saw that the biggest problem to reaching AGI was going to be the data, the frontier intelligence that was going to be needed to teach them. And so we are more than just data annotators. We're more than thinking about it is we have these incredibly smart people on our team like Berkeley PhDs, Harvard PhDs, Stanford statisticians. Think about them like the equivalent of Google search, where Google searches is the incredibly powerful machine by the algorithm.

14:21And it's helped surface the best of the information on the internet. And it's removing the worst and worse. In the same way, we are trying to extract the highest quality intelligence from, you know, this vast amount of people. And so we have to measure them, we have to build all these algorithms to find the best people, we have to build all these kind of content moderation like systems to remove the bad people, and so on and so on. But what that means is we can actually run these A-B tests. So the same way that Google search or YouTube or Facebook or whatever it is, we'll run thousands of A-B tests to figure out, okay, what actually improves our systems, we will do the same things.

15:00And then because we have so invested so much in measurement as well, yeah, we can actually measure whether our mouth quality is improving. One of our big differences is that we have a ton of technology to make sure that we have the best quality. Or another difference is that our primary goal is AGI. We want AGI to happen. Like my background, it's in AI research. My background, like everything I've done since I was a kid, yeah, it's basically combining data with language, with math and computer science to reach this goal. You mentioned AGI. It goes by different names it seems by the month super intelligence agi this idea that we're going to have these digital like god machines that are as good or better than humans at all cognitive work why is that a goal when i think about agi it is the goal of okay how do we help humanity yeah there's just like all these problems in the real world that humans just honestly can't solve on their own we haven't done a good job solving it and if we get ai yeah tremendous intelligence to help us reach these goals, then, you know, that's just better for us.

16:01I think people always talk about AGI and they think of it as this fundamentally different thing that we've never seen before in the world. There's also this view where it's just like the next step in technology. Do you agree with this notion that it would be another kind of pithy term is that it's humanity's last invention? in other words once we build these things we can kick it we can relax and these machines are gonna be like all right we need to sort out cancer communicable diseases climate change whatever just go do it machine we're gonna relax and just i'll be on universal basic income and sit under a tree and write poetry and relax i worried that's that's what will happen i don't think that's what we want to happen if that makes sense.

16:48If you think about what's happening today, even with education, a lot of kids these days, they're using AI to basically not do their homework. They're basically using AI as an excuse not to learn. But I don't think that's what we actually want for humanity. But I think we're not thinking hard enough about how we're basically for the decline of humanity. I do think it's a real worry that we should have. I worry that, yeah, if we reach AI, that will lead to this even more and more, just in the same way we're seeing it today already. and I just don't know that we are thinking enough about how to make sure that that happens one of the things that we don't think about is like what we're doing is we're trying to create AGI not that AGI replaces us but yeah makes us better in some sense helps us explore the galaxy helps us cure cancer so that we can do even more I don't know if you've heard you know Eric Schmidt he does the rounds a lot and he's called it the San Francisco consensus which is that AGI is coming and it's going to come in the next whatever two three five seven years it's coming soon and that kind of to what you're saying we are not ready for what is about to be unleashed from where you said because you're on this cold face in a way that most people are not do you think agi is a thing that will happen and two do you have a sense of time frame recognizing that any prediction is going to be wrong because that's just how predictions go yeah i do believe that ati will happen like i i see the trends even today when we add more morbid data that we create we teach the models in ever more sophisticated ways yeah we basically always see them improving so i do see this path to agi i don't believe that would be as soon as ever pay the else thinks like honestly i think a lot of reason that people have such short timelines is because there's this silicon valley impetus towards hype and the need for fundraising and yeah you need 70 investors on a store you need to press on a store and so on and so on and yeah like the The easiest way to do that is by saying it's going to come in a few years.

18:45It's like marketing by fear right now. Exactly. And so I do believe that timelines will be longer. I don't know. There will be either, I don't know, 10, 30 years. But there's just two big differences between what's happening with the ACDA and what we need. One is that they are often just textbook smart. They're not really smart in the real world. An example of that would be, yeah, all these models can go and win these IMO gold medals. and despite that they yeah like they often just make mathematical errors in fairly basic real problems they have it's because they've been trained on these very like synthetic almost very closed environment problems that just don't mimic the real world ambiguity and real world context that we have in our data k lives so that's one problem another problem is there's like a big difference between doing something 80 percent of time and then actually getting it to you know 99 99.99999%.

19:39And people think, oh, yeah, like you've gotten already 80 % of the way there. It's only another 20%. So we're just one fifth of the work. That's actually not true at all. Like every gap like breaking like half the gap is just as difficult as bridging the last half gap. So I do think it will take a lot longer than a lot of people think will be around in the process side. And on that point, do you think that leads to a world where, again, given your knowledge helping these companies build these systems, there seems to be a differing philosophy there is the open ai chat gpt approach of like we want to be the center of everything we want to be the only app you ever need to do anything and then there's actually well you need like a team of digital butlers you need your accountant you need your travel agent you need whatever person that manages your inbox whatever it may be and so you have a team of AIs that help you manage your life and do stuff?

20:36Do you have a sense of where this is likely to go? Yeah. So I think it's a really interesting question in that I used to think that all of the AI models would be commoditized because I'm a strong believer in kind of thinking that all the iron that they're commodities to each other. But then what I realized is that what you choose to optimize for, like basically what you choose as your objective function, what you choose as your metrics, your measurements of progress, that actually shapes your models very, very differently. Let me give some examples. It's like, yeah, you see some of the frontier labs, maybe today, they're starting to optimize less for, is this AI going to be useful to me?

21:17And more, is it good for engagement? Okay, are users going to click the like button? Are they going to have longer and longer sessions, just longer, longer conversations? How much time are they spending on these on these AI chatbots. Okay, today, they're spending 30 minutes on average, how do we get that to two hours on average? So that's like one possible way you can you can choose to optimize your AI models. Another potential way of optimizing your AI models could be some notion of usefulness. Okay, did it increase my productivity at my job? It make me more productive, so I could focus on the higher, higher, higher leverage tasks instead.

21:49And if you think about it framing, it's like, okay, so imagine you have some question like, you ask it, what is the capital of France, The first model, the one that's automizing for engagement, it's going to be like, oh, what an amazing question you're asking. This is exactly the right question. You're thinking in just the right way. Right. Like people, people love it when you praise them. So it's going to praise you. It's going to give you a long-winded answer. It's going to say the capital of France is Paris. But did you know that Paris is known as the Civil War and it was founded in the ABC?

22:16Then at the end, it'll be like, oh, by the way, do you want to learn more about the capitals of other countries? Like, it'll ask you these follow-up questions. Because yeah, like asking you these questions is the best way to maximize engagement. In contrast, the other model, the one that's optimized for usefulness, maybe it's just going to say pairs. So if you asked Google to build a search engine versus Facebook to build a search engine versus, I don't know, Twitter or LinkedIn, you all have different conceptions of what a search engine should be. Like they have different metrics to be optimized for.

22:41In the same way, we're going to have these AI models that are going to be optimized for different things and it will shape them fundamentally in ways that could be good or bad. yeah my pet theory is that this is all going to kind of more or less equalize and then people are going to differentiate based on the personality of their preferred llm but if you see what's happening with like chat gpt for example in particular they seem to be really leaning into engagement and this is all like oh we're going to launch erotica next and we're going to kind of 800 million people are using this thing lots of people are using them as companions and therapists and all of this kind of stuff that's it's to your point around worrying about where this goes that That just seems, oh, this is just social media 2.0, like on steroids.

23:24Yeah, it's almost like open eyes. The companies it's trying to get beat with, it's maybe not Google, it's actually Netflix or Facebook or video games.

23:37You mentioned Silicon Valley. And based on what I read, you have a very kind of, in a way, a very Silicon Valley story. So could you give me just a brief potted history of how you ended up starting this company, what you did before? Yeah. Before Surge, I worked on AI at a bunch of big tech companies. So I was basically a researcher at Google, Facebook, Twitter, just working on understanding our users, understanding how to use our own platform, basically figuring out how should we be training or basically our acronyms or social media runs. How should we be building a better newsfeed? How should we be building a better timeline?

24:15And it's kind of funny that we talk about social media 2.0 and this notion of engagement because what I actually worked on at all these companies was what led me to base pre-search. You know, when I first joined these companies, it's like, okay, what do we train our newsfeeds or timelines or YouTube homepages on? The easiest thing to think of is, okay, let's just train it on clicks. The problem is, okay, I tried training on clicks and then you just see this incredibly almost like dystopian clickbait rise to the top. Like I remember I worked on Twitter and yeah we trained our model on clicks and retweets and what happened?

24:49We got these you know like just post like what are the grotesque things that these celebrities have done in the past 20 years. And so we thought okay maybe we could just we could trade these calls on retweets instead. Or maybe we try to retrain them on comments instead. No like no matter what we did every time we tried to optimize for engagement we would optimize for you know essentially the engagement And so I basically had this idea instead of it was two things. So one was what if we leveraged humans and human data to basically define what we wanted instead? Like at the end of the day, it was almost like, why are you turning on clicks?

25:27Well, clicks are meant to be a proxy for what the person actually wants. So why don't we just ask the humans what they want? And then the question became, okay, humans can want a bunch of different things, though. It's almost like we needed to constrain the space. Okay, similar to how we were talking earlier about different models having different personalities or different models having different objective functions. Facebook, YouTube, Twitter, you all have the equivalent of feed. But what is the principle? What is your thesis for what people want to see? Go back to the early days of Twitter and Facebook, for example.

25:57Twitter's goal was to be this kind of public square. You wanted to convene the most interesting voices around the world. Okay, then think about Facebook's goal, original goal. It was about connecting you to your friends and family. so as opposed to these kind of like distant voices around the world yeah like what kind of content would make you feel closer to them and so they just have these like different principles and different theses if you take these two ideas it's like okay so how do we define what each product's thesis was and then how do we distill that into a set of guidelines that our human annotators should actually follow and then from that you know how do we build a system that means that we can get this really, really high quality data.

26:36So go look at your feed or go look at his feed. Take these principles in mind. How do you use that to define and measure what the best posts are? Like that concept, like this is incredibly complicated. Is there like a capital of data annotation? Is it like Manila or Delhi? No, we are always striving to use the smartest quality people in every field. and so we're not like the companies of your where the goal was to basically find the cheapest capital possible in the most scalable way no like what we're doing is we care about quality we're trying to teach these incredibly powerful art models like one of the things we often say is we want to sculpt them towards the best of humanity so that's cool and i mean as i mean like you know most companies obviously care most about english so you know probably the biggest countries we're in are English-speaking countries.

27:27But it's also true that people want I mean, all these frontier labs, they also want their models to be global. And so we basically have like you think about a million administrators, like they're everywhere around the world, but they're training models to be good at the language, the cultural nuances that they face. How do you find a million people to do this work that is like I imagine a lot of these places, it's probably you or I probably wouldn't want to do it. We want to be a platform where you and I actually really, really want to do this work. Again, I think there's a big contrast between what we do today and this view of data labeling that people have from even two or three years ago.

28:02And the way to think about it is, so like imagine you are a theoretical physicist, and you're one of the best theoretical physicists in the world. You spent your entire life, you know, learning physics and trying to unlock quantum physics. And what is it that we need on our platform? Well, it is I mean, these models are actually pretty good at visits already today. So what you need to do as a theoretical physicist on a platform, it's like you're pushing the model to be even smarter. So you are figuring out what advanced visits does it not know today. And then you're basically coming up with these incredibly interesting problems.

28:35Like again, we're not just trying to teach them textbook visits, we're teaching them to be a research collaborator so that you you know, like when you're trying to solve some I don't know, some special thing within the string theory, you're going to be able to collaborate with SAS when have interesting insights for you and so on and so on. So then you as a physicist, you're really working with CI to come up with all these frontier problems. And when it fails them, you are basically helping teach it how to solve them. And oftentimes, because these problems are so difficult, because the models are so good already.

29:02Oftentimes, it's like you doing fundamental research to solve the problem alongside it. So our theoretical physicist, he's like, okay, I'm gonna work with you guys. And then he realizes as he starts wriggling with you, or she starts wriggling with you. Oh, I'm helping build the thing that's going to be better than I am. You have people who like call you up and be like, I'm out. I don't want to speed my own demise. I mean, I think it's actually very, very different. Like the view that we have, it's not speeding our own demise. And I said, some of the way I said earlier, like one of our goals is we're not trying to replace humanity.

29:33We're trying to almost like augment it or help it do things that we just never could have done ourselves. Not long ago, I was in London. We had our big annual tech summit. And one of the speakers, Sir Nick Clegg, he's used to be the deputy prime minister. then he was at meta for seven years and he was like britain is becoming a vassal state of like a handful of silicon valley tech empires basically and i think implicit in that is this idea is that there's a real group think out here and that this place is fundamentally both boring and uniform it's like keeping up with the joneses but like to a really really high level oh here's my latest startup, there's not a lot of conviction other than I just got to get my next round, I got to get my valuation up, I got to get more clicks.

30:22Does that kind of ring true to you? And do you think that is dangerous? Again, when we go back to this idea of AGI, and who was building these things and who they're building them for? Is that a problem? Yeah, yeah. It's almost two separate questions. So one is, it's almost like if you think about the idealized view of Silicon Valley and Silicon Valley startups, like in your mind, it's supposed to be okay, startups are this renegade culture from, I don't know, corporate world where people have these big, bold ideas that they believe in. And you're going to do whatever it takes to push your view forward into the world because you're, again, you have this unique superpower that only you can achieve.

31:01Instead, it's like just the same as working at big tech, except you're just like following the playbook that your VCs have told you, just like a get rich quick mindset. And I worry about that for AGI because if Silicon Valley is the epicenter of AGI and this is the mindset that people have when you're building it, how does that shape the objective functions that you are going to achieve? How does it shape your goals? How does it shape what you measure? In Silicon Valley, I mean, engagement is such a well-known term precisely because it was born out of all these startups trying to chase user growth and chase getting to the top of the app store.

31:42and being able to tell their receives that they have X million users today, and they're going to have a billion users in two years. And when that is the goal, as opposed to like, you're right, like, admittedly, it is more near, but what should the right goal be? I don't know. Like, is it to be useful at your job and to replace the increasing boring parts of it? I mean, that's one measure, but it's not the only measure. I think there is this harder question. and this harder question of what you should actually be trying to build ai towards and in silicon valley you just have all of these vc constraints you have these user growth constraints etc and this group think that even if you try to secure principles it's very very hard because you have all these people surrounding you who are trying to push you into a different direction i do is there a world where you surge you decide at some point to get high on your own supply.

32:30In other words, to make your own LLM, because you talk about trying to sculpt the best of humanity into these models, you have a million people behind you helping create the data, which is in many ways, the most important kind of ingredient. That's something you'd ever consider. I think it's an interesting idea in that you think about the kind of the three pillars of making good AI, they are the algorithms, they're to compute and the data. And then I think we are like we uniquely understand a lot about things about data that almost like nobody else does so i think it's interesting the idea that i've also thought about let's see it being interesting but i just feel like we're so busy today doing doing everything we're doing that yeah it's not not a reservation right now

33:18wow what an interesting interview no i was just re-listening to it a few things this company is one of the most important companies you've never heard of because they are doing this very specialized high value kind of testing and labeling that all of the big AI labs use. Like every AI chatbot that you know of uses search there and they have these million people kind of fine tuning these models. And so it's this kind of, it's the unseen kind of engine room of this AI revolution. Which is something I hadn't thought a lot about because we talk so much about Frontier AI Labs scraping the internet and using all of the data that they can get their hands on, but not the idea that they're essentially putting human brains, specialized human brains into it, which is what he's talking about.

34:14Yeah, and as I mentioned, I've been doing some reporting for a piece we're going to be writing in the paper, And they put me in touch with one of their experts, whose woman, she's 25 years old. She's a really renowned mathematician in Brazil. Her name's Emily de Oliveira Santos. They found her because she was initially got drawn into work, believe it or not, on Humanity's Last Exam, which is this project by a non-profit to get all the best people in their fields to come up with the hardest problems imaginable. Yes. And then create this test and see basically once AI really passes that test, then we know we've reached AGI.

35:00So the ultimate Turing test. The ultimate Turing test, exactly. She did a bunch of problems for humanity's last exam and then it was interesting because the point around like where do you find all of these people and they're like they found her initially or at least the folks from humanities last exam found on this this maths website called math overflow have you ever been on math overflow all the time every day it is like the wonkiest math nerd place of all time and it's like a q a site for like super complex problems so like stack overflows for coders this is where the math exactly Exactly. Go to hang.

35:38Exactly. But it's math overflow. Right. I was looking at a couple of the kind of entries. I was like, this is like a different language. But they found her there, and now she's creating problems to throw at these AI systems. And again, coaching them through it. The way that the company framed it is like, think of, you know, if you're a golfer. Think of these LLM systems as golfers. Do you want to spend a million hours on a mini golf course? Or do you want to spend a million hours on like Pebble Beach? or pick your like really hard professional course and get coached through that. And that is the difference between they say like what they're doing relative to what a lot of these pennies per hour places in the developing world are doing when it comes to training AI, which I just thought was really interesting.

36:25But also hearing from this academic, I love doing this stuff because actually I get to do this really frontier type of work and I'm getting paid for it. It's just fascinating. and again, just completely unseen. And they're making money because of the contracts that they've got with the big labs like OpenAI and Anthropic. That's right. And if you read up on it, their data, their services, they sell it at a massive premium to a lot of what their rivals are doing. Because again, they're drawing on these professors and kind of experts as opposed to the kind of more commodity, is this a cat, is this a dog, data labeling.

37:04Yeah, and I do think the other thing that I think got cut from the interview, Edwin, they're based in New York. They're not based in Silicon Valley. He was out here and he's, I'm out. I don't like the group think. And if we're talking about kind of elevating human expertise and the richness of human experience or whatever the tagline is, what better place than New York to do that from? Which is like this big, interesting, messy, chaotic place. But we have an Apple story to get to. We do. Do we not? And you always like the Apple story. I feel like sometimes you take a big stick to Apple. Next year is its 50th anniversary.

37:42So half a century since the Steves, Jobs and Wozniak, started it all off. Not our editor, Steve. Yeah, when people talk now about AI, you hear about Microsoft, you hear about OpenAI, Google, Meta, Amazon, but Apple doesn't kind of feature. Correct. Kind of why Apple, I guess, is in the news right now beyond this 50-year marker is that they've had several executives leave, announce kind of the exit, several top executives, including the head of user interface, which is going over to Meta's super intelligence effort, the head of their AI efforts. There's kind of a changing of the guard, which comes I think roughly a month after.

38:27They've been trying for years now to integrate their own AI model to power Siri and Siri is still Siri, i.e. not good. And they rely on chat GPT for a lot of their an open AI for a lot of their AI. Yes, indeed. I think it was two, three weeks ago. They said, actually, there's in talks with using Google's Gemini to power their AIs while they try to figure out their own. So basically putting Gemini inside the Apple machine to kind of make it work better because it's being left behind. And so it's just kind of, we talked about it before of them having put, I think it was$700 billion spent buying back their shares over the last decade, something on that order and just trying and failing and trying and failing and trying and failing with new stuff.

39:17And here we are at this moment where it's like AI is here. That opens up a whole new bunch of possibilities around devices and how we interact with this technology which feels fundamentally different because you can talk to it and it talks back and it does stuff for you and you've got johnny ive apple's key designer historically now working with open ai on their mystery device on their mystery device which could come out as soon as next year which they haven't said what it's going to be but there's a lot of speculation online not surprisingly might not even have a screen you know it may be just be one of these always on microphones like my friend Steve.

39:59Your pendant, your AI pendant. My pendant, exactly. But some type of microphone that just is just like context aware you can talk to, it talks back. Who knows what that means in terms of privacy, doesn't feel great. But it does feel like there's this device moment happening that is being powered by the AI revolution that could be potentially quite big and probably the biggest moment since the launch of the iPhone in 2007. Yes. And again, Apple is behind the eight ball. They're just not kind of anywhere, at least that anybody knows about. I was going to say that we know about, because this really does feel like when it comes to AI, it should be Apple's moment, because what they do incredibly well is devices.

40:41And so it will be fascinating to find out what is going on behind the scenes there. It feels a bit weird talking about Apple falling behind, really, doesn't it? I could just never would have thought we'd be having this conversation a few years ago. They're still worth$3 trillion. But the thing you have to be careful about it with Apple is like they're kind of their strategic laggards. They're never, you know, they weren't the first smartphone. There was a time when everybody had a Blackberry. Yes, I had one too. I had a Blackberry. But then they came out with something that was just so wildly different and better that Blackberry died a death.

41:20but they just kind of they are known for kind of sitting and waiting and coming out with the right thing as opposed to like just rush something out so they're not they don't fall too far behind so i think it's a high stakes moment that they're that they find themselves in and a bunch of people coming out the door some new people are coming in and everybody's spending billions and trillions of dollars trying to take advantage of this moment yeah i know it's worth saying that i mean as you said at the start people are leaving but to your point about people coming in they have just poached that senior ai exec from microsoft amar subramania as their new vp of ai so they've also recently launched apple intelligence ai get it oh i see that oh yeah put ai in the uh in the iphone and and macs and stuff so it's not it's not like nothing's happening but it does apple intelligence Regents is kind of like the dampest of damp squibs.

42:14It is. You know, they're like, ta-da. But we should also say, whenever we have an opportunity to give props to a friend of the pod, or to kind of bring up a friend of the pod, we should. Listeners may recall, back in April, we had a company called Limitless AI on the pod. Their founder, Dan Soroker, he created this pendant that you wear on your lapel, and it listens to you all day, every day. and not to be your friend, but to kind of like be your kind of life coach. So it would summarize your day and then give you notes on how to be a better dad or a better partner to your husband or wife or whatever it may be.

42:48And it all felt a little bit like kind of like, I cannot believe this thing is real. But they'd raised tens of millions of dollars. They'd put this pendant out in the world and it was just bought by Meta. No, go on. So Meta has bought the company in the past few days. and what was really interesting the day of the acquisition limitless put out an update to their terms basically loosening their privacy effectively wow we can use it to train llms we can do this we can do that whereas before it was much more kind of tight the regulations were much tighter but meta comes in buys limitless i'm don't know the details of how much i'm presuming they didn't they didn't put a number on it so i'm guessing it wasn't like a blowout number because also limitless didn't look like it was this raging commercial success but i think it is again when you think about what met is doing with glasses yep and now they're buying this which is kind of again i'm sure there's some interesting tech that they found in there that they they want to get their hands on.

43:59It goes back to this kind of ambient computing. What's the next thing after the iPhone? The race for AI devices. Yeah. Yeah. Is it a combination of glasses and a listening device? You put a listening device inside the glasses as well. Like, how does it all work? I think it's just, it's totally fascinating time. Anyway, well, that's it from us for this week's episode of the Times Tech Podcast. So if you're enjoying the show, follow or subscribe or leave us a review or you can email us we have an email address techpod at the times.co.uk and danny we had an email this week from philip who came up with a really interesting question which i'd love to look at for a future episode which is all about infrastructure he says we talk a lot about the money that goes into it but he wants to know how long do the chips in data centers last which is a great question when you're talking about the amount of money that we often are with many many zeros going into these things so he said in his email data centers form the infrastructure but it's not like a bridge because obviously more advanced models come out so what happens if the chips put in the sheds of this year become obsolete well that's what's interesting i'm not sure if you saw michael berry famed big short trader who made a ton of money shorting the mortgages in the 2008 financial crisis.

45:22He's made a huge bet against NVIDIA. He has a big short on NVIDIA, specifically around this question of how long these chips last. What is their useful life? His allegation, which NVIDIA says is ridiculous, is that they're pushing out depreciation far longer than their useful life, which embroiders their finances. And thus, he's saying, basically, a crash is coming, these shares are going to fall. And it's been a big discussion point in Silicon Valley of kind of what the right answer is and what does useful mean. Maybe you're using them initially to train and then it moves to inference, i.e. just first you train the model, then these chips are used to kind of run them to answer the queries, which are kind of two different functions.

46:08But anyway, it's a big question and really gets to the heart of the bubble talk we've been talking about. Well, we should get some chip experts, I'm going to say chippies, but you know, that's the word for carpenters here. Not that sort of chippy to talk about it because it is a brilliant topic. Next week, we are going to be speaking to Harry Stebbings. I'm really looking forward to this. So a podcaster originally, venture capitalist, the founder of 20VC here in the UK, and I'm sure we'll have tons to talk to him about. So if you've got a question for harry or us at all drop us an email at techpod at the times.co.uk goodbye