What happens to Google when AI answers everything?

12:13Liz Reid:Yeah, for sure. did you see this viral post of airbnb co-founder joe gebbia in some coffee shop in san francisco with what looks to be the same device that was in the viral fake alexander skarsgård super bowl commercial that we talked about recently did you see this oh i did and i've been up all night thinking about it i don't have this totally confirmed but my understanding is that this was a paid thing. I mean, it's pretty obvious when you look at the video, the way the camera like abruptly goes down at the end as if he's like, you know, getting seen and needs to hide that he's filming.

12:52Ellis:It's also like the most freaking famous coffee shop in San Francisco sitting at the bar at sight glass. This is no iPhone 4 left at the bar. Right.

13:01Liz Reid:So my favorite theory that I've heard is that this is a very clever, subtle marketing campaign from some kind of startup who is also asking people on X through like marketing agencies to reply and boost this post. And that to me suggests there is a startup that is doing this weird looking AI metallic orb device. People think it's open AI. I don't think it is. I don't think this is the Johnny Ive way of launching a product at all. And I don't think this is what it will look like. But, you know, knock on wood, I guess I could be wrong. But I just want to know what startup this is that's doing this.

13:40Ellis:It looks, for those who haven't seen it, the case looks quite a bit bigger than an AirPods case. And it kind of looks like a big stainless steel, almost like modern clamp, like literally a clamshell with a little indentation to put your thumb. And then the earbuds are also stainless steel. And instead of only going in your ears, they have a bar that almost like a piece of jewelry that some people might wear as earrings wraps around the back of your ear. I think some people were speculating it was like bone conduction or something like that. I don't know. I honestly don't appreciate these types of stunts.

14:15Ellis:I really don't.

14:16Liz Reid:We'll see. We'll see if it ever ships.

14:17Ellis:Maybe it's just because I don't know how to make them for companies.

14:20Liz Reid:All right. Well, this week we have our second Google executive on the show. We had Samir Samad, the head of Android on, who was actually one of our best performing episodes. So the pressure is on for... Liz Reed, the head of search at Google. I've known Liz for a little bit. I actually interviewed her recently at the IAB Publisher Conference. She had to tell a room full of publishers why Google wasn't destroying them. And I got to moderate that conversation. So that was fun. Maybe not as fun for her as this episode of Access. But that was where I last saw her. Liz is a really interesting figure in tech.

15:02Liz Reid:I mean, she obviously runs search, which is, I don't know, we were talking about this in an interview, maybe the most influential software product of all time. Certainly.

15:11Ellis:She kept it real, I thought. I mean, I wasn't super familiar with her, but for a 20 year Google veteran, I was expecting like, you know, a lot of a lot of scriptedness. But it's interesting. She seems like a true Googler, almost like a scientist who's just working on problems they love. And so I appreciated her candor.

15:35Liz Reid:With an incredibly important job and an incredibly unique vantage point on the industry. So yeah, it was great to have Liz on. Let's throw it to Liz. Sounds good.

16:01Ellis:Hey, yo, Mr. Pit. You tell yourself no one wants your college-era band tees. But on Depop, people are searching for exactly what you've got. You once paid a small fortune for them at merch stands. Now, a teenager who calls them vintage will offer that same small fortune back. Sell them easily on Depop. Just snap a few photos and we'll take care of the rest. Who knew your questionable music taste would be a money-making machine? Your style can make you cash. Start selling on Depop, where taste recognizes taste.

16:59Ellis:tabs are in the row? Are we grouping? Are we a pro at this? Not very many at the moment. So hopefully it doesn't crash. That'll be the goal.

17:08Liz Reid:When Alice and I were waiting for you to get on just now, we were just saying like, gosh, you've got to be in back to backs all day long. Is that like every day for you? I try. I always try to make sure I have some thinking time and some email time. But yes, to a first approximation, there's a lot of conversations on quite a wide range of different topics too. It'll be like, let's go brainstorm this product question. Let's brainstorm the future. Let's fix this current product. Let's go talk about legal. Let's go talk with you all. We just celebrated someone on my team who's been 20 years at Google right before this.

17:40So that was kind of fun.

17:41Ellis:Oh my goodness. Did they get a special treat or a new type of hat for Google Legends? I mean, we do a few things for them, but yes. Yeah. 20 years is quite a long time. I mean, that's about your tenure as well, right? I am, yes, 22 and a half maybe now. Oh my goodness. I know you've moved around a bit, but I was at Snapchat in the early days for almost eight years. And I feel like for every year you're there, you're attached to one more permanent recurring meeting as you sit in and try and do things. I mean, even like, how does your inbox look these days? I feel like the longer you've been at a company, the more in disarray it can be.

18:19Well, I switched a few years ago from geo to search. So that created a reset on most of the meetings and the inbox. There's lots of exciting things. And then we have new features like AI overviews in Gmail that you can play with and hopefully help make some of it easier. But it's a good place for AI to improve my productivity. There's lots of opportunities there.

18:41Liz Reid:You were super early at Google. You were one of the first female engineers. Is that right? I was in the New York office. I was the first female engineer, but not not in the company for sure. I think there were, I don't know, less than 2000 people when I joined. I don't know if it was like 1000 or 1200 or something like that. But the New York office was 10 engineers ish when I joined. It's rare to work at a place for that long. I feel like especially now, I feel like millennials and Gen Z are used to job switching and hopping around a lot. and it's kind of how things work. And I'm curious. It's interesting because I joined, when I joined Google, right?

19:22Like I didn't come from a family with tech background. Like my mother was a math teacher. My dad was selling insurance. And so like in my mind, people didn't change. And I was from a small town, right? And so people didn't change jobs all the time growing up. And I started at Google and people were like, yeah, you know, generally in tech, you're only at a company for like one to two years. And I was like, what? What is this concept that I'm just not familiar with. But that was the expectation they set is I'd probably be there a couple of years and then move on. And, you know, it's been 10 times that since.

19:54Liz Reid:Well, I was going to say, what's been keeping you there for over two decades? There's got to be something. Besides money, of course, besides Google stock. No, I mean, I think there's like three things I generally care about, I find. One, I want to work with like people I enjoy working. I tend to work hard. I spend a lot of time. I would like to be people I respect. I feel like I can learn from, you know, it's intimidating and fun to feel like you're not the smartest person in the room most of the time, right, on the thing. And, in fact, you're maybe one of the least smart people in the room, right?

20:30So, like, that sense of energizing. But that people really want to create great things, I think, is one. The second thing, the mission of the projects I've been on has really resonated. So I was on Google Maps for a long time. Now I'm on search, right? These are products that not just billions of people use, but they turn to for things that matter, right? It's not just this is just a job. It's okay. People now use this so that they don't get lost. Like when I started, this is hard for some Gen Z to understand, but like we grew up in a world where like if you made a wrong turn or you went to something, you might be like stuck wandering around.

21:07Liz Reid:I remember printing MapQuest directions to get to Disney. When I first started, there wasn't MapQuest, right? It wasn't even MapQuest. There was an Atlas in the back of your car. And that was what you hoped you did. And otherwise, you stopped at a gas station. So the freeing power of being in this world where you can go to a strange country. You don't even speak the language of a navigated. What does that unlock? Now you have a thing where people turn to search for their fun trivia. And they turn to it for their health questions. And they turn to it to figure out how they educate themselves on a new skill or learn something.

21:42And that's kind of very inspiring. And then the last thing I like is I like challenges, right? Like if I, you know, you can always learn, but like there's different levels of learning. So over the years when I got sort of most asking the question about was I going to stay, it was never about the mission. It was never about the people. I always really liked them. It was like, am I challenging myself? Not. And I, but I found a place to change within Google that put me a little bit in the deep end. Definitely still the case these days with how fast the world is changing. But I like that sense of learn to swim and face the challenge.

22:17Ellis:Well, thank you for Google Maps. Because my first and only car accident was trying to read a MapQuest printout. When I was 16 years old, rear-ended the crap out of somebody, was quaking in my boots, had to call my dad. Hi, dad. I know you're listening out there. Thanks for coming that day to help deal with that for me. and now it's just such a fact of life. And I mean, I think that raises, you know, an interesting question that we wanted to talk about is for these products like Maps and Search and otherwise that people have made such a part of daily life. How are y 'all thinking about bringing AI into this stuff?

22:58Ellis:I mean, people have such expectations for what these products are. And I think we all know that nobody likes when their furniture gets moved around, when you keep putting new things at the top of the page, when new features come out. And I feel like Google search has remained pretty, pretty solid in its vision for like what this perfect experience is. And I feel like whether it's with knowledge graph or AI, that's kind of getting shaken up right now. How do you guys think about the pace of, of introducing those things? I know there's a lot of pressure from the market to just go fast, fast, fast.

23:32So I guess a couple, a couple of things um one i do think it's like easy to think about search as sort of somewhat static before ai but actually if you go back in time like the knowledge graph was a big thing and it was you know over 10 years ago and like you know sometimes if you go and you do a printout for for people about like what search used to look like 10 years ago they're like what like what is this product right you sort of forget how it how it changes over time um across we introduce things like lens and voice and all different along ones. I do think we try and take that responsibility very seriously.

24:07Okay. Like it is, it's not just like nobody likes the furniture, but like people are using it in a way that they rely on. Right. And so you don't want to make it hard. But also there's just tremendous opportunity. And so you think about how to balance it. So I think we, you know, first did this with SG. Okay. We want to go figure out how we allow early adopters and they can opt in and then we brought it to AI overviews, but we did it amidst the SERP. When we introduced AI mode, we didn't like swap everybody over from SERP to AI mode. We made it possible to enter along. And so trying to allow people to sort of participate more and more as they're ready in different ways.

24:47I have been surprised by how fast people like appreciated AI overviews, started using AI overviews, just started searching more, right? I think it's for many people in the world, not necessarily in your circles, but like AI overviews is how they've experienced AI sort of first and primarily in their life, right?

25:11Ellis:Are you accusing me of not hanging out with normies? Well. Because I sometimes do. I sometimes do. I make no accusations, but I feel like sometimes in Silicon Valley, We live in a world where it's surrounded and then I go back to my home in New Hampshire and it's a very different environment. And so this is often how people encounter it. But I think for a lot of it, when we do our job right, it's so intuitive and it sort of opens up things that they couldn't do previously that they just start doing it and they start adapting it. So we saw people use search more with AI overviews and we saw it much more quickly than we generally do.

25:52We always sort of assume, to your point about moving furniture, that there's a learning effect. It takes time. You unlock something. How long does it take for people to figure out how it works? Does it work? Does it even exist? I think if I gave, you know, even Googlers a survey of what are all the features in search and I threw in like a few fake ones and asked them, could I identify the fake ones? I'm not sure they would all be able to identify which ones were fake and which ones were real because there's so much possibility. But people really picked up with AI overviews very quickly. And that was really, really delightful to see.

26:25Liz Reid:When you took over Search, it was a really critical time for Google. People were going, you know, oh, Chachapiti is going to eat Search's lunch. This is all zero sum. Google's in trouble. And here you go. You raise your hand. You're over on Maps, which is just kind of cruising along. And I feel like it's just been on this consistent upward trajectory in terms of importance to the world. You're like, yeah, I'll go take this challenge. I've never actually heard you talk about that. What was that like deciding to go take on search at that moment? Because I imagine there was just a tremendous amount of pressure.

26:56Liz Reid:You've got to secretly kind of love the pressure, I think, to take that job in that moment. Well, so actually I moved to search before that moment, but search was sort of in a few different pieces. I moved actually, I don't know, a year into COVID maybe, but was a different time of pressure and others. Like you join a new team and you can't meet any of the people in person, right? Like, you know, for all of these years, a whole bunch of how you go and you build relationships and you meet people in person. I had grown up with the MAPS team for years and years and years. They knew me, the people under it.

27:30And then I'm like, hi, I'm your new, you know, manager, your new leader. Sorry, you can't see me in person for months. I switched in part because like, candidly, my boss was like, I would like you to do this job. But like, how do you say, how do you say no to search, right? It's like people will sometimes say like, you know, how do you say no if somebody asks you to do the thing? Not just a beloved product, but kind of like the essence of what Google was about.

27:53Liz Reid:I mean, it's definitely top three most influential internet products of all time, maybe arguably the most.

28:00Ellis:You called it even like one of the best businesses of all time last night on the phone, Alex. And I was like, whoa, actually, yeah. Yeah, I think it was like both tremendously exciting and tremendously intimidating to like move and be leading a big part of search. And then to take on, to your point, then when I got asked to take on all of search sort of in the middle of this time, it's a lot of responsibility, right? Like you have multiple billion people using this product every day. It has been iconic for the last 25 years. And you're like, I am going to shepherd it. And, you know, I'm going to hope I'm going to.

28:37Ellis:Are you satisfied with your impact, Liz, that you're affecting multiple billions per day? To be very clear, my team will tell you I am never satisfied, right? I am always a person who's like, oh, but here's what could be. Here's what the opportunities. Here's how I could do better. I just love how all the tech companies are like, come to our company and you could have big impact. We have one employee for every 5 million users or something like that.

29:08Liz Reid:For Google, that's a crazier ratio. Even though you guys have a lot of employees, it's still a crazy ratio. It's like actually, so like my town that I grew up in for much of my childhood is 900 people. Okay. I had a bunch of cousins. It's not like I had never left the town. But like it is mind boggling to work in tech with this idea that what you do, it touches it touches people halfway around the world. And like it also touches your mother. Right. Like if you don't do a good job with this, then your mother, my mother is a very kind person. OK, but like like like she's entrusting a bunch of choices, like how well what does search tell her?

29:50Right. Right. And that that is a it's a big responsibility, but it's also a big opportunity. Like everybody I know, nobody I know is like, I am bored. I don't know what to do with my time. I, you know, wish I had more work to do. Right. Like like people have a lot of things they have to do in their life and a lot of things they'd like to do if they could find more time. Right. How do you navigate search so that you can help people enjoy the things they want to do and get rid of a bunch of the stuff they don't? And to me, that's a very fulfilling thing.

30:22Ellis:My four-year-old says she's bored constantly. Okay, four-year-olds do. I do agree with you that like four-year-olds, my children sometimes tell me they're bored, but none of the adults I know feel bored. But the children feel bored, yeah.

30:35Liz Reid:Last time we saw each other, you were telling me actually your kids are frequently telling you to fix things in search. So you also have to be tech support for your kids. They do give me bugs and they send the ones. And then my daughter tells me I should dog food my product. I was like, I'm pretty sure I'm doing that every once. But she's like, you asked this question. You should just use your product and see how it does. And I was like, OK, yes, I understand.

30:57Ellis:Speaking of kids, though, I feel like, do you remember those moments early on in Google where everybody was accusing Google of outsourcing all of our brains and all our knowledge and what's going to happen to children and their minds when they don't have to use a calculator anymore? LandaVision, they could find everything on Google. And that just kind of came and went. And do you feel like it's any different this time around with AI? Yes and no. I think there's always this question about like, are we still teaching people to think? Right. And I don't think creating tools means that there's no opportunity to think.

31:38It just means that we need to like make sure when we're teaching people to think that we're that we sort of are aware that the tools exist. Right. I got this feedback about Google Maps, by the way. Like people can't like on the one hand, people have lost the ability to navigate using, you know, just their memory and their towns as much on that. On the other hand, like you freed people to explore the world.

31:59Liz Reid:I've been living in L.A. for five years. I do not know the difference between the one on one and the five because I literally never look at highways or roads. We lost some skill, but as a result, you unlock to new things. OK, I think what's a little different with AI is it's changing quickly enough that there's a little bit of this challenge of like, do teachers know how, like, do we know how to help teach kids in the presence of these tools? Like calculators didn't kind of go from like nowhere to like in everybody's hands overnight. And so you could sort of think about that. And so I think, you know, whatever tools they are when done well, they allow you to do harder tasks, right?

32:38They allow you to put your critical thinking on even more difficult things. Done poorly, you're asking people to do things the tools can already do. And then, you know, people cheat, right, in different ways. And so I think the challenge for society is to figure out how are we using the tools to push what we can do even more, not to replace thinking, but to use it for the next task.

33:02Ellis:It's funny because I know the Google famous interview questions, which I know have changed a lot over the years. ostensibly were a way to test your thinking and your reasoning. I mean, what do those even look like in five years? Well, five years feels like an eternity now. I talk with my team, let's do two and a half. Let's do two and a half. You know, I think in a simple, well, even two and a half years, like if you go back two and a half years ago, like, do you know where we were two and a half years ago? Go back in time. Alex didn't even have a newsletter back then. Yeah. On that thing, right?

33:35You know, that was the biggest change. What did we do? That was the biggest change in the world in the last two and a half years, for sure. I think that questions, though, are a lot about, like, continue to be. Can you think through? Can you reason? Can you use tools effectively? It's not like we asked, you know, software engineers when I started, can you do assembly language, right? Like, you weren't expected to use assembly language. You weren't expected to write code without an ID. You could use an ID. Okay. now how do you collaborate effectively with AI right and so I think those questions are not sort of pretend there's no AI but can you do it thoughtfully can you use it to orchestrate can you can you understand how to debug things can you think through what things to consider right if you assume that the tools get much better but that they're imperfect do you have the reasoning to look for where they might be imperfect and I think that's how those questions evolve and that can be in software engineering or it can be in product management or it can be in data science right like you what's you know you can offload the easy questions on data analysis what are the hard

34:38Liz Reid:questions a bit of a galaxy brain question that just came to me are you guys talking about internally a future where maybe agents are the ones crawling google the most on behalf of humans and like it's not humans going to google but but agents will be do you think that'll happen Well, I certainly think there will be a world in which sort of agents are doing a lot of interaction on the internet, not just people. Okay. I personally don't believe in a world where it's all agents, right, in the sense that I still think that people sometimes want to hear directly from other people. Like it's different, you know, do I want to go and say like, Ellis, please tell me what Alex says?

35:23Or would I just be like, Alex, what did you say? It's not always fun to be disintermediated. You often want to hear from the source directly. But the fact that you can kick off 10 agents and you have limited time and the agents don't necessarily, I do think probably means there's a world in which a lot of agents are talking with each other and not just with humans going forward as we evolve.

35:46Liz Reid:And does that impact Google positively or negatively search? I think that's all about how we all navigate the innovation space. Right. Like I think done right, it impacts it positively. Right. OK. Like search enables people to do harder tasks and search is a great tool other people can use and agents can use. And what does that look like? I think we'll have to figure out. But I think Google has a long history of saying the world keeps changing. I mean, mobile was a this may be hard to believe now, but at the time when mobile came, mobile was like this big question for Google. Like Google was a desktop product and could it navigate?

36:24And oh my gosh, the screen was smaller. And are there going to be any ads? And is the whole business going to fall apart, right? I mean, like this was a major question when mobile came out. And the company ended up stronger through it because it rallied to the moment. And I think we are at heart a tech company. And this is a time when tech is flourishing in terms of like tech as a technology, right? As to like what is its core DNA is being great at using technology. and this is a time when the technology is being amazing. So if we all do our job right, I think Google will do well as it navigates it.

37:03K-pop Demon Hunter's Saja Boys breakfast meal and Huntrix meal have just dropped at McDonald's. They're calling this a battle for the fans. What do you say to that, Rumi? It's not a battle. So glad the Saja Boys could take breakfast and give our meal the rest of the day.

37:17Liz Reid:It is an honor to share. No, it's our honor.

37:20Ellis:It is our larger honor. No, really. Stop. You can really feel the respect in this battle. Pick a meal to pick a side. Hey, participate in McDonald's while supplies last.

37:34Liz Reid:Spring Fest is heating up at Lowe's. And for a limited time, we have the extra big deals you need to impress guests. Get your outdoor space ready and save$50 on the select Cobalt 24-volt blower kit. Now$79. Plus, save$80 on the Charbroil Performance Four Burner Grill. Now$199. It's springtime, and our best lineup is here at Lowe's. Valid to 413. While supplies last, selection varies by location.

38:07Ellis:I work with a lot of startups for most of the week in my work at Meaning. And one of the tried and true startup arguments to investors press otherwise is that we are going to be able to move or act more quickly than the incumbents. And a lot of times it's Google or Microsoft or Apple. But it feels like in some ways that y 'all have really kicked yourselves into gear and accelerated the pace over the last few years. Is that just my perception or was there kind of a critical moment internally where it's like, uh, AI is happening. Let's go. I don't know that I would say it's like a single moment.

38:50I think you have a series of moments that build. I mean, I think Sundar started talking about this is going to be in, you know, an AI company. I don't know how many years before some of these things come. But I do think what sort of has like been accelerating is what's possible with the tools and what are the opportunities, right? Like the technology is getting so much better faster, right? It's not that in some of these cases, people didn't build demos five years ago. It's they built the demo and like it didn't really work. And it was really hard. And it was going to take like 20 engineers, two years to tune it to try and get that.

39:27And like what used to be 20 engineers, sometimes that task now you can have one engineer do in less time, right? And so I think that really unlocked. Like, so a bunch of the search engineers have been in search a long time, like 10, 15. The number of like 20-year anniversaries we celebrate every quarter is kind of mind-boggling to me, right? Like sometimes like these engineers and product folks had these ideas 15 years ago and they didn't work. And they had the ideas 10 years and then they had the ideas five years. And then you like try it and it's like, oh, it works now, right? Like that's energizing.

40:09And that energy builds on top of it, right? Like when you see that you try something and it works, when you see that you can move faster, it makes you want to move faster because like we all want our time to like be valuable, right? For it to have impact. Like if we're going to work, you want it to reap rewards. And I think it's energizing to people to see that it feels like they're not optimizing as much. They're inventing.

40:35Ellis:Any examples of those things that were thought of 15 years ago that are only possible now? I mean, I was thinking of Google Duplex being able to call places for you and make reservations. I mean, that was an agent before we called it an agent. Now it's actually possible. So I would say that's a good example where it was an agent before we called an agent and we could do very few verticals with a lot, a lot of work. Right. And now our ability, you know, it's like, OK, you had this whole team that worked on it for this one thing for many, many months. And now the rate at which we can expand. And I think like with the recent launch of personal intelligence, I mean, if you go back long enough at Google, we tried to do this effort in which we were putting more personal content and using it to leverage.

41:16But I don't think it felt like it unlocked it in different ways. I think people had played with, I mean, LLMs had been in search long before. We had Bert and then we had Mum. but they were mostly used in limited use cases because the quality wasn't good enough the speed wasn't good enough right like you can't you can't come and be like guess what I can do this thing that's better but you have to go wait 20 seconds like people people on search are sensitive to 100 millisecond difference right like they will search more or less for 100 milliseconds Right. So you had to get the tech to the point where like, wait a sec, if you did this online, it was fast enough for people and it was good enough for people to be better.

42:06Ellis:Are Burt and mom the search goats who've been there for 20 years or those technologies? Burt and mom are search technologies.

42:14Liz Reid:There is probably a Burt inside Google. Sure, there's a BERT somewhere in the company and possibly in Twitch. Otherwise, yes, there are a lot of ICs on a first name basis, but BERT and MUM are technologies. They were sort of early uses of LLMs, but they were mostly on the ranking side because when we tried to do it on the UI side, we couldn't get it fast enough and high enough quality. So if we did something, we're going to do it offline, sort of pre-processed going forward. I think Lens when it first started was like kind of cool, but like didn't work most of the time. And now it feels like you just take for granted that you can like take a picture of a shoe and get the right particular Nike model and not just, oh, this is a Nike shoe, right?

42:57Like those things weren't possible years ago. So I think there's, I mean, I think they're just all over the place of what's possible. But sometimes it's not just the big things. It's just the little things. Like I tried to make this thing work internationally and I couldn't. One of the ones that's been most excited to me is in the US, we think of like, well, most content you care about is on the web. If you're in India and you speak Hindi, let alone you speak a different dialect or language, right? Like, no, the information is not all on the web, right? Certainly not in your language. Okay. And it used to be like, oh, what are we going to do?

43:35Are we going to try and translate all the content on the web into all the language? which is that's not scalable. But now with an LLM, you can take information in one language, understand it, and then output in another language. Like that opens up information.

43:50Ellis:The YouTube auto dubbing, I think is incredible. Like I've been watching a lot more international videos. Yeah. Right. Like, and that, you know, okay, well, were you going to learn how to speak all those languages? Were you going to hire somebody in a hundred languages or even 10 languages to dub your thing? No, you weren't going to be able to scale that. So I think that ability to like unlock content to, you know, everyone, not just speakers of languages is fascinating. And it means we have a lot more impact in a great way in helping people around the world.

44:23Liz Reid:Google is a large company, and I don't think people from the outside fully understand the difference between what your team is doing and what Demis and the Gemini team are doing and how you guys work together and also how you have different directions that you're going. I remember when you and I spoke at the IAB conference recently, you told me that Gemini has a different North Star than Search. I think that was what you said. Can you explain what that means? Because from the outside, it looks like AI Mode and Gemini are converging on each other pretty fast. So it's interesting. So maybe to just start with Demis' team.

44:58Demis has like obviously different sub teams under them. One of them is Karai's team, which is working on the underlying model. And then there's teams like that are working on the Gemini app, which is sort of what you're working on them. And the way we work with them is sort of a little bit different than that, right? Like we're sharing a lot of the underlying model and technology. And so we work very closely with the teams on the model. And we'll work like bi-directionally. So like number one, they'll go, here's a new model. Here's what's now possible. Like how do you want to use it? But we'll also go and be like, okay, I understand you have this lovely benchmark.

45:33But let me tell you what it looks like in practice. Like, you know, 80 % may seem awesome on a benchmark. You can't be wrong one in five times in terms of not showing people the sports score. Like if they ask for the sports score, they want to see the sports score 100 % of the time.

45:47Liz Reid:You're popping a lot of AI researcher bubbles, it sounds like. There's a lot of give and take about the reality of, you know, what does it take to build a product that like people can rely on and build? And so like we push each other in a bunch of ways. I think with Josh's team, we share some of the same underlying technologies. We share ideas. But they do have different North Stars. And what I was saying with the other one is like Gemini's focus is on sort of being this assistant. And so it tends to lean in more heavily on things like productivity or creation, right, on ones. Search is more information-based.

46:26and it believes that often in those information use cases, you also want to connect and hear from other people. And so how do you bring out the web? And so we sort of focus on making those two things awesome. But then like you can't, it never works to go tell people this product can only be used for this or that and then like hope that they understand, right? Like people will come to the product and do whatever they want, okay? And if you take a non-search Gemini example, people come to YouTube for things that you might think to come and search especially like in India the YouTube versus search usage is very different than in the U.S.

47:05right how people use them so you kind of have to meet users where they are if they come to your product and they want to use these things then you should make it possible and we want the capabilities to be used you take something like Gem Nano Banana, which in its first version felt like mostly sort of fun for creativity and doing that. Nano Banana 2 allows like really good infographics. Okay, well, infographics is back in that information space. Okay, so how are we going to use that to convey, right? And so we think about it as like sharing some of the capabilities, but building excellence in key areas, if that makes sense.

47:42Liz Reid:Do you think they ever converge and just become one thing or do they always stay separate? Because I guess if I ask Gemini a question and I ask AI mode or AI overview a question and I get the same answer, then why do they both exist? I don't know the answer is the short answer. I think what we see is some areas they're converging more and some areas they're diverging more, right? And so what are they going to net out? Do the areas that diverged eventually all come or do the areas that diverge become even bigger over time? I think we'll see. I think, you know, if you ask a math question, we should give you the same answer, you know, to a first approximation.

48:21But even in a math question, you know, we've started to experiment more with not just, you know, links in the product, but like videos in the product. So in search, you might expect to get like great video tutorials from like those cool teachers that make those concepts you know memorable right and so you still want to answer if they ask the question but maybe we're going to encourage you to take the next step a little bit differently um than that i think with some of the verticals we're seeing how they go um and there's sort of some not complete divergence but but you know how do you experience it what needs to be quick and easy um how do you want to surface the web i think is an area that we're still very early on and ripe for exploration We've started to do more things with things like inline links.

49:11You know, how do you quote and allow people to dive deeper into that? So I don't know, in all honesty. But I think we are right now at a point where, depending on what angle you look at, you'd think they're getting closer or they're getting further apart. And so we'll see. It's not – AI mode has not been out, like, for general use for a year yet. Like, we sometimes – I sometimes forget this, maybe the world. Like it's relatively new. And so and even Gemini app, given how quickly the world is evolving, it's changing a lot. And so I think we'll have to see. And then like, who knows, maybe agents will mean like the right product is neither of the two of them is a third product altogether that they merge into.

49:53I don't know yet.

49:54Ellis:So to be totally frank, I spend a lot of my AI time on ChatGPT or Claude or other various types of services. And I've kind of kept my Googling to like the stuff I've always Googled, song lyrics, pictures of Tom Cruise and other celebrities. Like that's what Google is kind of about in my mind. And is there an effort to kind of get those 800 million chat GPTers back to Google or is it kind of about the next five billion more so? Well, I guess I would say a couple of things. I think most. Feel free to tell me I don't matter because that's a very valid business strategy. What I'm actually going to tell you, what I actually tell you if you use your example is you're still using Google.

50:39Okay. And so like what's interesting in many of these cases is that the sort of balance of how people are using multiple tools is actually like sort of really evolving, right? I think if you go back some amount of time last year, where did most people do image generation and what was considered like a place people went to for image generation? And where do people do a lot of image generation? It's like changed a few times about where the centers are. And so I think right now we're in this world where it's very fluid. Like people's habits are not solidified across. You know, you essentially listed two other products you're using besides search.

51:20So you haven't picked a single one and you're still using search in some cases and whatever else, right? And I think that's just indicative of where the world is. It's still figuring out. I don't know, by the way, that we're going to end up in a world where there's only one product, right? I think you also saw this with, you know, people will be like, oh, social media, there's going to be one winner. Okay, well, you've got YouTube shorts and you've got Instagram reels and you've got TikTok, right? But actually the space that, although there's three of them there, the space is much bigger than any of them were at any time, right?

51:51Like the pie has really grown. And I think that's what we're generally seeing in the industry is people are using the tools collectively much more. And so there's a lot of growth. But I think what we mostly try and do as like, I don't think we think of it as like the next 5 billion or the 800 million in like that simplistic way. It's really like what feels best in class with the market? Where are the growth? Where are the things that people aren't even thinking about? How do you just really live up to the mission and help people with as much as possible? And then teach people about what's possible.

52:20A great thing about search and a hard thing about search is most people use it. Therefore, most people think they know how to use it. Right. And so there's this interesting thing in search and teaching somebody like you unlock this feature today. Oh, yeah.

52:38Ellis:I mean, in startup terms, it's a great wedge. Like, I think a lot of people don't even know what they would ask ChatGPT or even what it can do quite yet. And if you just kind of start revealing that stuff through Google, I mean, I'm very receptive to like the idea or hypothesis that Google may actually win out here, given all the existing use cases where people are asking questions. now the answers are just going to get dramatically better more informed better cited all that stuff

53:07Liz Reid:well but yeah ellis there's also your behavior that you talked about and like you know liz my wife chloe is a is a chat addict every time i look at her phone or laptop it's chat gpt and i just asked her the other day ahead of this interview i was like how much do you use google nowadays since i got you hooked on chat gpt in the last six months and she's like maybe 10 as much I use it to fact check the AI a lot. But she's definitely not doing those kind of in-depth, highly monetizable searches that she was. And there is probably a decent chunk of your most highly monetizable users in North America and Europe that have shifted their behavior that way.

53:51Liz Reid:Maybe not. But I'm just wondering. I mean, obviously, Google's doing very well financially. But yeah. Like the space is just growing rapidly, right? It's really weird for people to think about in that most people think they're asking the tools they're using all of the questions they could think of. And so like everything is zero sum, but that's actually not what happens, right? You kind of go through this calculus without even realizing it, which is like, I had this question. First of all, do I even realize I could get the answer? And then if I realize I could get the answer is it worth my time?

54:26Like, is it going to work? Is it not? Do I have, like, is it hard to ask? Is it not? And so I think what we're seeing is, like, simultaneously, people are adopting more tools, and search is growing, right? Because the possibility of the tech is just allowing many more questions. And so I think the space is just getting much, much bigger. And if you think about that, like, when you didn't have a mobile phone, and the only time you ask questions online was when you were at a desktop and you booted up your desktop. And by the way, you had to hope that nobody else in the household was on the computer at the time you wanted to ask the question.

55:02You just didn't ask the question. And I think we're seeing the bar get lower and lower. And so the amount of questions people are asking, the tasks that they would have struggled with that now instead of struggling with, they ask for help, right? Like people use the tools to help them rewrite an email. They didn't previously use a tool. They just struggle with it and bang their head and spent 20 minutes trying to write the email. And now they ask for help and write the email faster. So I think we just think the space is growing very, very rapidly.

55:31Ellis:I think there's also one thing that we haven't discussed yet, which is when you think about what is likely to make your first AI experience or retentive AI experience good, it is if it gets you, right? And I think you could argue that Google has better context on you than any other company. What are your thoughts on that and bringing that stuff to the fore? Is that creepy for people or is that magical? And do you have more context than anybody else? Well, there are several billion people in the world. I don't think we can make a statement about do we have more context than everybody for every person in the world by any stretch.

56:07But I do think in a bunch of cases, we do know, you know, the systems know a bunch and AI's ability to parse it together is doing that. I think personal intelligence was like one of our first steps there, right? And I think it can be really delightful and surprising sometimes what it gets, right? So I had a coworker who asked, he was like stuck in the airport for like a few hours, and he's like, what should I do? And I was like, well, given this passion, there's this cool store at this terminal. And by the way, you need to leave 40 minutes ahead. And this credit card actually lets you get into this lounge, right?

56:44Like a bunch of things that he wouldn't have even thought to ask. And it wasn't just like a basic recommendation, right? It wasn't like, you know, which restaurant should I eat at, right? So I think it can be very delightful. One of the things we did as part of it was that it is a deliberate consent. You have to opt in, okay? Like we don't want to do it to people. If you want your information siloed, you should be able to keep the information separate. But if you want that value, we shouldn't be like, well, you might not want the value. So sorry, Ellis, you can't get it, right? Like, it should be in the hands of people to make that choice.

57:19But then we think there's nothing. It's like one of my favorite things I do now is like, I ask many more questions of like, okay, I have this challenge where like, okay, I have three kids, I have a job, I'm going to do something with them on the weekend, how much time am I going to spend figuring out what I'm going to do with them on the weekend, right? Because I'm going to spend the time with them. But I don't have like an hour to prep. so if I don't have enough time then I'm going to do something boring which is the same thing I did like two weekends ago but now I'm finding that I can use personal intelligence to understand more of like where I like to go with my kids what I enjoy and then it just speeds up finding something fun to do.

57:56Ellis:Deeply relate to that use case yes people are like you know people have time to have fun they just don't have time to plan planning is hard. How many people do you know like spend like 30 minutes looking to pick the movie that they're going to then go watch for like an hour, right? Like it's just ridiculous on some of these things. And so we should be able to make that much easier for people.

58:16Liz Reid:Don't get Alice going on this. Alice is a planner with that stuff for sure. Liz, I wanted to know how you're feeling about the health of the web right now and what Google actually indexes. And there was something you said at the conference again, where we had that an interview recently where you basically warned a room full of publishers to just try to use the tools in a way that doesn't produce more slop and to actually use them to elevate. And I have to imagine Google is trying to fight back a ton of slop right now and spam like you all have never seen. And yeah, is the internet in a precarious moment right now, the open internet?

58:58People have created slop for a long time before it was AI slop. AI slop allows them to create it faster OK, but it's not a new problem. It's just sort of the next level scale of the problem, right? People would pay people in other countries to create, you know, rip off essentially surface level content, 20 versions of the content, et cetera, on the ones and so does. So that's something that like Google does have a bunch of experience of fighting spam. And we have to like stay on our toes. Like it's a cat and mouse game. You are never done. Right. The commercial incentives for other people are there.

59:33So I think there is a lot of AI slop being generated for sure. I also think there continues to be a lot of great content generated. And I think sometimes AI helps people create better content. And sometimes it helps create slop. And so I think it's important to separate, did you use AI as a tool or did you create slop, right? You're trying to bring to life the vision you have for what a great living room could have to write something. Okay, like that can sometimes be very hard for people. are you good at one thing but you're not as good a speaker at some other thing right you you know can you use it to help you write better and more clearly but I do think we have to fight it from the search side of making sure that we get the the great content and I also think it's a time for creators and journalists in different ways to say like are you producing great content that is really interesting to people, or if you're creating the same junk 100 ,000 other people are and just hoping your content is going to surface the top that's not as good, right?

1:00:38I think it's been pretty interesting to see how much movement there has been to user-generated content in the world, right? Like people are switching what they go. They're listening to more podcasts. They're hearing directly from more creators and influencers, right? But sometimes people are like, oh, well, there's no journalism these days. And you're like, wait a sec. But people listen to four-hour podcasts, right? It's just going to different places. It's different places, right? It's in ways that it feels like it brings it alive to people in different ways. And so I feel like it's still an exciting opportunity for creators in so many ways.

1:01:18And part of our job is to make sure that we surface that awesome content, right? that the people who create that content feel like they thrive in that world.

1:01:28Liz Reid:Does it mean, do these shifts you're talking about mean you all have to be a little more creative with how you index and what you index and that you can't? I mean, a lot of this stuff's gonna be paywalled. Yeah, paywalled or it's audio only or it's like hidden behind an RSS feed or that's proprietary to some app. Like how do you actually index all this stuff when the bottom does seem to be falling out of like traditional search traffic reliant publishing. I mean, I'm sure you've seen that chart that's been going viral recently about all these tech outlets that have lost 50 to 90 % of their traffic, including the Verge where Elson and I worked.

1:02:05Liz Reid:And that just has a real profound shift on where content shows up. And then it's your job as the head of search to like still index that content, which means... There's a few different things. To your point about audio, the great thing about LL... One of the great things about LLMs is they're multimodal. So we can actually like understand audio content and video content, actually the level we couldn't years ago. So that becomes exciting. Some paywall content, people still allow Google to crawl, right, and surface. So it's not necessarily that it's absent or we have deals with some set of ecosystems where they give us content streams across.

1:02:44I think paywalls is complicated for users because, like, they're not always willing to pay. And so it's a complicated calculation. You know, how do you think about that? But I generally feel like, you know, search is able to find most of the, you know, the content that's out there. And in fact, you know, new opportunities are being created with the fact that, like, now you can understand audio much better. Now you can understand video much better. Now you can understand not just the video transcription, but what is the video more about or what's the style or other things like that. So I think there's still a lot of content available.

1:03:21I think everyone together is trying to figure out what makes great content, how do you reach audiences. I think one thing Google is trying to do a lot more of, we've taken small steps so far, but I want to do more. How do you help when there is that relationship? So we did this thing with preferred sources. um uh there have started to be experiments across google with you know how do subscriptions change right so like if you love this source and you do have a relationship with it then that content

1:03:52Ellis:should surface more easily for you on google okay i have banned i have banned so many sources from my google news over the years it's like my magic i feel that's when i feel most powerful okay but But if a user loves access, but in a positive direction, Ellis, if a user loves access and they ask a question, and you had a great story on it, the question shouldn't be how does your content rank equivalently for 6 billion people. It's like that's really interesting for me. So surface it higher for me. Reinforce and build that connection. If you have a cooking subscription with someone, can we make sure that those recipes show up even more and that you know that you can click through, right?

1:04:39Like people hate getting links that they get to and then they have to bounce back because they can't get through to the content and they're not willing to do it. But sometimes those people have decided to pay somebody and not the other six people. Well, then we should surface the one that they're paying for and not the six that they can't get access to more. So I think there's a lot of opportunity to also think about, given that in many ways, I think the sort of connection between audience and individuals is getting stronger. How do we actually help with that as opposed to treating all sources equally for all people?

1:05:16Liz Reid:Add sources to your preferred sources. You should. Sorry, I'm just going to keep saying that. You should. This is the bane of my existence naming my publication sources, Liz. This happens all the time.

1:05:27Ellis:Good idea with that, Alex.

1:05:29Liz Reid:If only I had a friend who knew how to name products. Yeah, no, I hear you on that. I mean, I do think that's the challenge for you guys, right? Is the shift to paywall, the shift to audio. I would love to be able to ask Google, hey, what did Liz say about so-and-so in the last episode of Access? And it gave me like a snippet with a link to exactly where that was said right in the episode. but not quite there yet. But it sounds like that's the aspiration. I think you want to do more of that. Okay. But also like, let's say there's 20 interviews and that are all paywalled and Ellis has a subscription to one.

1:06:04And he's asking something about what I said or somebody else said, right? Like we should return, we should make it easy to find the one that he can actually access. And then maybe you all have to figure out how to make it easier. If somebody doesn't want to buy the full subscription and what they want partial, I don't know. We'll see. Micropayments hasn't really ever taken off. but maybe that will change over time. But I think there is an opportunity to make with personalization, not to just do personalization and sort of small ranking twiddling, but really strengthen these are the people you trust from.

1:06:38Let's make it easy for you with AI to access that information.

1:06:41Liz Reid:Well, Liz, probably the next time I'll see you will be Google I.O. in a few months. Sounds right. It's already coming up. I'm sure you're already in rehearsals. Um, can you, can you give us a tease? What, what, what is next for, for search and for Google this year? Well, I was going to say we are, uh, we are actively building, but, but what will make the, the, the stage will be an evolving thing over the next couple of months. But I think we're very excited across the company. Like I think as the tech unlocks new capabilities, it's also very weird because like traditionally you plan for IO like several months in advance.

1:07:14And now you're at a time where you're like, okay, but like what happens if we figure out something in the tech April 1st, right? Like, are we gonna ship it at IO or not? So we'll see how things bring us. But I look forward to seeing you at IO. I think there'll be lots of great news to share then.

1:07:31Ellis:That's got to be so tough. We had to do the same thing with the Snap Partner Summit in the early years. You know, you're planning it five, six months in advance. And now it's like half the startups I work with are pivoting during our five-week sprint, because it's just so tough to stay on top of all the changes. But it's tough and incredibly exciting. The fact that you're pivoting is because something now is possible, not because something now that you thought was possible is not, right? Like that's a fun time to be in. Yeah.

1:08:03Liz Reid:All right, Liz, thanks for joining us. Thank you. Appreciate it. Have a great day. See ya.

1:08:11Ellis:And that's it for this week's show. Thank you, Liz, for coming on. Don't forget to like, subscribe. Everywhere you get podcasts, we are access.show on the internet, and you can find us in video at AccessPod on YouTube.

1:08:24Liz Reid:And if you like this episode, please leave a five-star review or thumbs up and share it with a friend. It means a lot. And you can find my newsletter at sources.news for all your scoops.

1:08:36Ellis:You cannot find my newsletter because I am too cool. However, you could find me at Hamburger on Twitter and at Meaning.Company for all of your startup storytelling needs.

1:08:47Liz Reid:Access is part of the Vox Media Podcast Network, and the show is produced by Hooked Creators.

1:08:52Ellis:Bye.

1:08:53Liz Reid:Bye.