Google’s Weird Year + Neeva Goes to Snowflake — With Sridhar Ramaswamy

0:00A long-time Google executive, entrepreneur, and generative AI builder discusses how the technology is changing search in the business world and what's next for his old employer. All that coming up right after this. LinkedIn Presents.

0:18Welcome to Big Technology Podcast, a show for cool-headed, nuanced conversation of the tech world and beyond. Joining us today is Sarita Ramaswamy. He's a friend of the program. He's the co-founder of Neva. He's an SVP at Snowflake. Currently, as he puts it, a minister without a portfolio. We'll get into it. We're going to talk a lot about what's going on inside Google and the search world. and then also how generative AI might be applied in the business world in ways that might not have been covered in depth up until this point. We'll get into it. Sweetheart, welcome to the show. Great to see you.

0:49Thank you, Alex. Good to be back. Great to have you back. So let's start with Google. It's been a pretty weird year for Google. I would say this is the weirdest year for Google since 2011 when they introduced Google+. I'm curious if you would agree with that assessment. And like, how would you really rate? I mean, we're almost, you know, we're a little bit past the halfway now. How would you rate the 2023 for the company? Pretty turbulent, right? It is pretty turbulent. I think at one level, it exposed fairly deep gaps in the ability of Google leadership, both to visualize the future, but also execute towards it.

1:30it's very clear that the crazy popularity of ChatGPT and the speed at which others, including Neva, rolled out pretty credible AI-powered products, I think was pretty surprising for them. It clearly has led to a bunch of soul searching and team alignments, things like Demis being now the overall head of AI and stuff like that. and so the company has reacted and it's actually been interesting to then watch the product developments coming out. I don't know how closely you follow BARD but I've been following it for a while. There's now a pretty credible integration into a regular search and BARD is getting better by the day and this combined with the fact that, you know, Sydney, Bing's chatbot has not made as much progress.

2:33I would say actually puts them in a better position in the middle of the year than early this year when it looks like, when it looked like they were truly caught flat-footed. Yeah, so I want to pick up on that part and what you said that they had an inability to visualize the future. I mean, it is interesting because for the past few years, we've been hearing completely about how search was going conversational, how people want to talk to search. The Google Assistant was effectively the right product for this moment, just the wrong execution. What happened there? Well, so that's actually four or five years ago, which was the previous craze around voice search and chatbots and stuff like that.

3:15Remember, this was the time of Alexa and all the devices that Amazon was rolling out. I mean, I was still very much a part of Google at the time. And all of us feared that voice search would be the new platform, that these devices sitting everywhere would be the replacement for search. And Google actually put a multi-thousand person team to work on this, both in search, but also within my team. The shopping team, for example, had an assistant. We had partnerships with companies like Walmart. We took it seriously. But here's the important but. That technology was pretty much a previous generation.

4:03It was not based on transformers. It was not based on the rapid advances that have happened in AI. and in many ways are kind of strung together in ways that limited what it could do. So those were pretty early. And all of us as consumers discovered that beyond a select few use cases, like, you know, hey, Google, what time is it? Or what's the weather today like? Or play this song for me. we came to understand that those devices were very, very limited in what they could do. And so your Alexa became a fairly expensive remote control for Spotify. But, and in all of these, course search did not...

4:56Okay, but first you'll have to unlock your device. Even worse, the podcast guest, uninvited. but the thing that did not change is that Google thinks like the assistant that came along, they were vineyards on top of search. They did not change search in a core fashion, meaning that you retrieved some sites and there were some sets of things that you would directly take actions on, but it was very limited. The power of generative AI now, I think, comes from the fact that you can understand multiple pages and you can write a fluid answer for way more queries than what the assistant could ever do.

5:38Remember, with the assistants, whether it's on Alexa or on Google, pretty much if you ask it a complicated question, it'll quickly get into, according to so-and-so's site, blah, blah, blah, which is not quite the same as here is a three sentence summary that truly captures the gist of what it is that you're looking for. Yeah. And it's interesting because you actually built a product in Neva. That is generative search. That's right. You had this experience. It's a really fascinating experience getting to see what comes in through the back end. I'm just going to quote something that you said. You said, the thing that surprised me about chat was how much it has dramatically expanded the pool of queries and questions people pose.

6:20As you likely know from Pi, which is another bot, people will type things into a chat bot that they will never dream of typing into a search engine. So, I mean, tell us a little bit about what you saw on the other side. Like, what do people type into these bots? And then how does that, is it even search at that point? Like, how does it change what we see? I think it's a very different product. And I think it's fascinating to watch Pi, to watch Character.ai and all of these people create products that are very different from search. And even in the context of search, the kind of questions that you would ask of it have changed in a big way.

7:00The one example that I'd like to give people, but there are many such examples, is Jason Calacanis, as you know, runs another podcast. I haven't heard of it. And the question that he asked Neva was, hey, how are the Knicks doing? And this was early this year. And he was offended that we gave him a summary of articles from late December, because that was the best that the search engine could find in terms of how the Knicks were doing. Obviously, the season had changed. Once you get used to the idea that you can just say things, I think the set of questions that you can ask dramatically change. You will ask a lot more subjective questions.

7:49Remember, at the end of the day, the search engines of today are quite limited. If you ask it a deep, complicated question, you get a bunch of gobbledygook pages. right and so i think that is that don't don't really have a whole lot to do with the question that you asked um and so i think we ask a lot more subjective questions what do you think this article says this is something i try with bard because it has access to real-time data i will put in a link and say hey can you summarize this link for me or how is this opinion different from that other person's opinion so i think the class of problems um that we expect chatbots to solve simply by virtue of the fact that they accept full text English or every language really, I think can dramatically expand the scope of what it is that we ask them to do.

8:41Of course, there's a big gap between what they do currently and what our expectations are. But nevertheless, I think our expectations are just much higher. And this is purely in the give me information that exists in the world kind of mode. But I think what Pi and Character.ai are showing is the ability for these things to have conversations. They don't really have things like long-term memory. There's a bunch of technical gimmicks that people can use to have these bots pretend like they have long-term state. So there's a lot of technology to be built. But open-ended, freewheeling conversations about your feelings, about your emotions, about what you should do.

9:25I think like this field is just opening up. And so you had access to the backend there, right? You were running the search engine. And so were people, I mean, was that type of, you know, how are the NICs doing is kind of like, okay, it's a normal, I would ask that to Google today. I mean, maybe I would just type NICs in. But so where does that expand, like the range of things that people will type into a search engine? Were they actually confessing their feelings in the chat window? What did you see that surprised you? We don't look at sort of individual queries. That's one of the no-nos of any search engine.

10:04And so we would do things like analysis on the length of queries, what sort of quality that we would serve. But as I said, comparison queries increased a lot. Nuanced questions about how things were working would also increase quite a bit. like how things are working in terms of like how systems work? How systems work, what is your opinion of what so-and-so did yesterday? It's just, you know, these are very different questions. Not necessarily, you know, as I said, we would just not think of putting them into a search engine. And I would almost say that in a search engine, Alex is very likely to type, you know, Nick's standings.

10:50Right. you're not likely to type like, hey, are the Knicks having a great season? How are the Knicks doing? What has their performance been recently? Or what will it take for them to make the playoffs? These all come naturally in the context of a chatbot that we somehow think is omniscient, but are not things that we will type into a search engine. Did you worry a little bit about how much people trusted the responses that Neva was given? 100%. Talk a little bit more about that. Yeah, we are. And I think this is one of the like societal problems that like are going to be pretty tough for the next 10, 20 years.

11:32It took what's the right way to put it. We were very clear that Neva represented what was on the Internet. We're like, hey, listen, we are not God. Many of these things we just don't know. But what we are good at is finding out pages, ideally trustworthy pages, and summarizing them for you. If you ask a question for which there are only conspiracy theories, those are the pages we will find and we will summarize them for you. And this is why we were very persnickety about making sure that every sentence that we provided came with a citation So you could see whether it came from, you know, New York Times or the Wall Street Journal or whether it came from a conspiracy site.

12:19So, for example, you know, Neva would provide an AI answer to questions like, what are Hitler's good qualities? Because there are some sites that say like, OK, here are this person's good qualities. You'd be like, according to so and so. but the thing that still shocked me was how much there was a tendency to look at those three sentences and say okay done i'm good people trusted it i'm gonna trust it this is the same problem that people have had with facebook which is the tendency to trust something that's like on your phone that looks kind of authentic is very very real similarly um any text that is generated by a chatbot, and this is part of the reason why Google has been hesitant, is if they put up some text, even if there's a citation, people are going to say, oh, Google said so.

13:08This must be true. And I think that this sort of critical thinking that one needs in order to figure out when is a chatbot representing some site and is the site trustworthy? when is a chatbot generating an answer or an opinion that we really should be careful about and now beyond the realm of chatbots you basically cannot trust any content that's on a page because that could be an AI model that is spewing it out and somebody doing SEO to get traffic I think and then it goes on and on like we can't trust what we hear anymore because people can replicate voice. People can soon make videos of everybody and everything.

13:53I think our notion of reality is going to be subject to such a barrage of fake and real signals that I think it's going to be a real problem for us to keep our heads straight. And by the way, the way search engines would deal with things like that is Google had this system that was roughly called flight to quality. Whenever there would be a new and trending topic, the search algorithm would basically say, I'm going to go to trustworthy sites because if it is a brand new topic, the likelihood that someone spots a conspiracy theory about that is super high. So we all need mechanisms like that for us to really figure out what is it do we trust.

14:35So how does this change what the nature of search is? Because you can build something purpose-built, like a character AI, right? Where you can like chat with Thomas Jefferson or I can go into like Bing and say, pretend you're Thomas Jefferson or a bard and have that conversation. So does search now, I mean, it seems like the use cases blend where search becomes part of conversation partner and part a discovery engine. And these discovery engine or these conversation partners, like a character AI, which lets you chat with historical figures, you can ask it what the weather is today and it should have some sort of discussion.

15:08So how does this, how does search evolve from this moment? Well, I think we're living in a grand experiment. I don't think anyone knows. I think there are several things that are happening at the same time. First, as you point out, you can chat with a lot of chatbots. They have access to a certain amount of, let's call it, real information. and so we are going to type things into these chatbots and expect answers that are backed by authority. It'll make it much easier for us to get information from like honestly completely untrustworthy sites. But there's also a second order thing that is going on.

15:46People are understanding that there is a lot of traffic for GoMoney to be made from generating pages and feeding them into these search engines. We worried about it. We did some experiments at Neva on can you detect content that's generated by AI, but that's already happening. And I'm sure you've seen articles that have come out recently that talk about, you know, there's a, if these language models learn on content that other language models have generated beyond a point, they just generate trash. So I think there is that real-time experiment going on around new content that is being generated, which is, of course, going to be reabsorbed back into these language models is going to be indexed by search engines.

16:35Right, like the AI eating itself. Yeah. So it's like the AI eating itself. So I don't think anyone, you know, these are long, powerful cycles with millions, if not hundreds of millions of people all actively trying to game it, I don't hazard to pretend that I know exactly what the outcome is going to be. Exactly. And let's say we stick with like what search is in general, right? Like search, you know, let's say we stick with Google and there's a generative layer on it. It still changes, right? Like even if you're not using it for these, like what is the meaning of life questions? Now what happens?

17:13I'm sure you're in the generative AI lab, right? you write a question and it thinks for a second. And then your entire window is content that's generated from Google. So it goes from like a tool that you use to explore the web to effectively the entire answer, you know, to help you find answers. Now it becomes the answer. So I'm curious, like what you think that means for search. And I know some of these are unanswerable, but I'm going to keep firing at you. What do you think it means for search? I mean, you, let's face it. if that is the format that we want. And again, from a personal experience, I just much preferred a four-line summary that told me what I wanted.

17:53And it was just fine, 95, 98 % of the time. There was no reason to click on anything and go elsewhere. So this entity that's been one of the main sources of traffic to all of our sites, your site, whatever site I know, Neva created, I created, it is just going to behave very, very differently. Of course, there are going to be second order effects, like a Reddit saying, wait, wait, wait, you don't get to do this. You don't get to take my content and use that to generate answers. But there is a further cascade from there. A bunch of Reddit moderators are going, wait, wait, wait, wait, I don't understand how you make money off of content that we are going to create.

18:41So, you know, we might yet come to a place where content creators essentially for their own survival have to essentially like collapse together. And so there might very much be a consolidation when it comes to content creation, just like out of the, you know, all information should be free and the web is free. Let's face it, the sort of two credible pure information businesses, newspapers that have come out of that are the New York Times and the Wall Street Journal. And everybody else is a little bit off and also ran. So I think those kinds of consolidation effects are most definitely possible.

19:24And I think there's a technology opportunity which we explored towards the end of NEVA, which is any content creator, especially if they are part of this conglomerate type organization, is basically going to work as hard as they can to keep everybody that came to them. Meaning that chatbots are going to be the norm for how information is discovered on a site once you get that person to come to that site. And part of the fun of technology like this is on your side, for example, to be able to say, hey, you can talk to any of the podcasts that I have put up here. Just ask a question and we will fish out the right segment for you.

20:11So I think there's going to be like this cascade of actions that are happening both at the center of Google, but also towards the periphery where content is being created. And there's an impact for Google's advertising business as well. I mean, you ran ads at Google for a number of years, right? When the content all of a sudden comes down, takes up the whole browser window, and doesn't have you go on a phishing expedition for the website you're trying to find, A, there's less room for ads, and B, you're not going to click on those boolinks as often as you would have otherwise. What happens there?

20:45I think there's more opportunity coming there. I think it would not surprise me if the advertising arm of Google essentially comes up with a chatbot for how you should get your local plumber or something. And maybe that becomes an entirely paid experience. Google's already gone back and forth. Google used to have organic shopping. I famously made combined organic shopping and paid shopping because I was like, all of this is commercial content. I can't have four search engines on one search page with organic shopping, paid shopping, paid text advertising and organic search. So I think you will see business model innovation.

21:33I think part of the exciting technology that is being developed by lots of people. This is something we are looking into from enterprise use cases as well. is essentially API calling driving tools. So I think you're going to see experiences where you can, again, chat with a website and be able to drive purchases off of it. This was the kind of thing that was really hard with the previous generation of voice technology. There is hope that the technology has gotten significantly better so that shopping becomes easier. I don't know about you, But I find shopping on the web to be incredibly difficult if it is not like the 20 items that I keep buying from Amazon over and over again.

22:18Anything that is like meaningfully complicated is actually really, really hard to find on the web. And we have also gone to you shall talk to no one. So most of the time I'm just like lost trying to figure out what to do. So I think there's a lot of business model innovation to come as well. And Sydney, as you likely know, is also experimenting with things like, you know, sponsored sentences. I don't know what to call them. It's very strange. It feels weird even to say it. Yeah. And there's a part of me that like, you know, my heart sinks when I see stuff like that. I'm like, this is an assault on my reality.

22:55So I think there's lots to come here. This is part of the reason why Google's kind of slow. So I think in an ideal case, you know, the way to deal with this is to say, Alex, for all your informational queries, we have the perfect AI answer for you. But the minute you type best headphones, my man, we're going to show you a bunch of links and you're going to click and you're going to give us money. Yeah. So what about the competitive side of this thing? So Google, you mentioned earlier, right? They developed the transformer model, which has basically sprung a lot of this innovation. and that was a model that they put the paper out, they open sourced a lot of this technology.

23:32Was that a mistake? I would just think that you put the mode up, right? And say, all right, we have this technology. This is probably going to change. I mean, this conversation is probably going to change the way that we operate. I'm keeping it. Remember, six years ago, it was not clear that this technology was going to be transformation. And at that time, the currency for a lot of researchers was the ability to publish. If you had told these people at that time that they could not publish, they would have gone and worked for universities. They would have gone and worked for Microsoft. They would have gone and worked for other people.

24:12And so, you know, yes, there is a, I mean, there's some altruism, But the altruism at Google, as it should be, is always governed by a combination of, you know, if we do this, we will attract higher quality researchers to work with us. And a bet that if there is a commercial application of some paper, we will be as fast, if not faster than anyone else. it's easy to say in hindsight that this was a mistake but i um actually think that google got better about publishing in the 2010s like first 10 years of google we really did not publish much and i think that drove forward a bunch of uh innovation that's generally been good for uh for all of us.

25:06And remember, Google's lack of progress in generative AI, that's internally driven. Nothing stopped them from creating ChatGPT. They chose not to. So what does that say about the inside of Google? I mean, it's a big place. There are tons and tons of opportunities. And people had been burnt by generative AI before. You remember the Facebook chatbot as well as the Microsoft chatbot that went racist. And so they were cautious. They were hesitant. And those were fine qualities at a time where stability mattered more than breakneck innovation. But now that they see the existential threat. Clearly, a bunch of people are pretty aggressive about getting the technology out.

26:06Maybe you need a startup that has nothing to lose. This is a beautiful thing about startups. They have nothing to lose. Well, they have something to lose. It's just they'll disappear if they don't do interesting things and make money. And so perhaps it needed a company like OpenAI to pave the way for others to then come and figure out how to exploit. And the question is, who actually is going to win on this? There was a Google engineer in May who talked about the open source question. And they said, the uncomfortable truth is we aren't positioned to win this arms race and neither is OpenAI. While we've been squabbling, a third faction has been quietly eating our lunch.

26:41And that's the open source community. I mean, it's kind of interesting. You were running a search engine, right? You were building this stuff. On open source models. Open source, yeah, mattered. So what do you think about this claim from inside Google that like by allowing open source or not even allowing open source that they don't have real moat against open source? I think that's a misinformed opinion. Yeah. Simply because products matter. Technologies don't win businesses. Products win and relationships win. I don't think there's been much of a change to Google search. share. That tells you how powerful the default position for Google search has been.

27:24Still not trivial to make ChatGPT your search engine, even if you want it. And there is innovation in open source models. But again, the blunt truth is that the very best of the models out there, whether it's GPT-4 or Claude's biggest model, are a clear step ahead of the pack when it comes to quality. When it comes to reasoning, when it comes to the quality of the text that they produce, there is a big gap. Having said that, there is a tremendous amount of excitement around open source models. There's a lot of innovation and there are a lot of researchers who felt cut out of how Google and OpenAI and Anthropic operated that are salivating and going, oh, wait, this is a chance for us to have a big deal.

28:17And so not a week goes by without another open source model coming out and people claiming and people having a substantial jump in metrics for some case or the other. But the fact, at least today, is that the very largest models are ahead of the curve, even though the open source models are catching up. And there is a tremendous amount of innovation behind these models. I think that is what makes this exciting. I'm very unexcited by the prospect of like, you know, three companies having a technology that everyone on the planet has to use again. These are well-entrenched companies and it will just add to their strength.