Creating the future of search and competing vs Google with Perplexity AI’s Aravind Srinivas | E1770

0:00Your interviews with Brian Chesky, I learned a lot from those episodes actually. me too also i liked his idea that like they were going to only ship the number of features he could keep in his brain and that his brain would be the maximum you know size of the canvas so if if i can't one person can't keep all these changes in their brain let's put those changes into the next six month cycle i thought that was pretty yeah awesome as well i actually borrowed a heuristic from there adapted it for our company which was if the person building the feature doesn't know how to write the code for it. They're very good programmers, but if they're finding it a hard time to break it down and actually implement it, then it's not worth shipping.

0:40This Week in Startups is brought to you by Crowdbotics. Great ideas can change the world. And Crowdbotics is the fastest way to turn those ideas into code. Get a free scoping session for your next big app idea at crowdbotics.com slash twist. Vanta. Compliance and security shouldn't be a deal breaker for startups to win new business. Vanta makes it easy for companies to get a SOC 2 report fast. Twist listeners can get$1 ,000 off for a limited time at vanta.com slash twist. And OpenPhone brings your team's business calls, texts, and contacts into one delightful app that works anywhere. Get 20 % off your first six months at openphone.com slash twist.

1:25All right, everybody. Welcome back to the program. we have been having an amazing array of founders who are taking on the challenges of implementing AI in the real world. And today will be no different. We have our invent Shren Evas on the program. He's the founder and CEO of perplexity AI. Welcome to the program. Our invent. Do you have a nickname or you go by our event? Arvind is good. Thank you for having me here, Jason. uh great to have you and um listen you there's a big battle going on between chat gpt and bard and you're right in the middle of that you are doing perplexity ai and you are trying to compete with these two giant you know software developers tell us a little bit about how that's going and how you see perplexity.ai which you can go check out right now The interface will look familiar.

2:22How do you plan on competing with them, and how is that going? Yeah, so firstly, we started off a week after ChatGPT came out. We put it out, and there was a lot of difference at that time, which is we were just a search bar, and we gave direct answers with citations. Whereas ChatGPT was this entertaining, hallucinatory bot that was not just correct many times, but it was also equally wrong many times and its mistakes were also entertaining, right? So we focused a lot more on revamping search, realizing that, you know, 10 years from now, no one's going to be asking for 10 blue links. You're going to ask for answers.

3:06So we might as well start it today. And the technology for that was ready. Both ChatGPT and us were basically being powered by GPT 3.5. That was the fundamental breakthrough. And then after that, GPT-4, even better than that. So that's kind of how it started. And we were seen pretty different. It's like, oh, you know, if chat GPT lies or makes up things, there's this other side called perplexity. You go there and like it's going to be this boring, educated uncle kind of product, but useful and you can trust it. And that's kind of how we grew. And then Bard came out. I believe BART still hasn't solved the fake news problem completely.

3:52It does hallucinate and it doesn't actually have like real citations sometimes. So we are better than BART in the context of search. But Google is also rolling out this thing called Magi or they call it search generative experience to the public. But the Wall Street Journal called it Magi. so they're trying to do something pretty similar to us and so far the experience at least from what I've seen myself use that and other people who've used that is it's not very different from the way they used to extract text from the top link and put it space at the top it's not very different from what they've already done before and they cannot afford to use really powerful models so like the search traffic that they have and And if they actually want to really get it right, but in the actual search bar itself, outside of bar, they're going to lose a ton of revenue.

4:49Yeah, so they have two challenges there. And so have you done a crawl of the web because you are giving citations and you have a language model behind this. So tell us what is underneath the hood here, because I have been saying, hey, if you're going to get a bunch of information and present it to me in a beautiful answer with bullet points and numbers like perplexity just did for me, I asked it, hey, what are activities I should do with my seven year olds? And they like cities and the outdoors. And it gave me four popular destinations for cities and four popular destinations outside. Really good suggestions, really tightly summarized.

5:29And then at the bottom, it said, hey, and then here are three citations, TripAdvisor, U.S. News, Family Vacationist, and Today, the Today Show. So what's underneath the hood here? How is it generating the answer? Yeah. So LLMs are these great reasoning engines. You throw a lot of text at them and tell them what to do with it, and they'll do it for you. And then there's the other part that's great, which is having a good index and a ranked version of the index, which is a traditional search engine. And what we do where we come in is we combine the two together. We say, hey, like elements are great.

6:06We'll figure out what content to throw at them for a given query. And we'll instruct them on how to actually take all the text that's thrown at them in the context of the query and get the needles from the haystack and present it in the right format to the user. So they're doing more of the reasoning job. you're not actually doing, pulling up actual facts that's been stored in the LLM itself, because some of them could be right, some of them could be wrong, real actual facts are in your web pages. So that's the content that we want to take. And we have like our own index. And also like we rely on other index providers.

6:39And we collate from multiple different indexes, multiple different crawls of the web and pull up the relevant links. And then we ask the LLM to do all the reasoning on top and then we give you the answer. Now the magic is that all this happens so fast. We've put out the product in December and back then the latency used to be like five to six seconds per query. In fact, one of our investors, Daniel Gross, he used to joke to me saying you should call it submit a job and not submit a query. It's that slow. And now it's like almost as fast as Google, like you're hardly waiting. The summary is like really generated really quickly.

7:15and we still have so much more room to improve there. And I think at some point, you're just going to take answers as the de facto search experience. That's kind of what we want to bring together. And our primary superiority over the existing products is the speed at which we deliver the really accurate, well-collated answers from so many different sources. But you are built off of today, ChatGPT4, correct? Yes, we heavily use ChatGPT 3.5 and 4. And we also use a little bit of our own LLMs for many other things. Every question you ask on our site, you see a few related questions that are being popped up, right?

7:58That's actually one of the favorite parts of the product for many of our users because they like asking more. And that is sort of generated with our own LLM, for example. So there are some parts of the product that we use our LLMs, but I would say like most of the... heavy lifting is being done with opening as elements right now. And so you added right now, does that mean your plan on building your own? Because it does seem like you're directly in competition with Bing Bing has the partnership with chat GP4. So it's almost like you're both using the same underlying technology. Correct. They already have some scale.

8:35So that would be a difficult race there. So how do you look at chat GPT fours relationship or open AI's relationship and Microsoft's access to it. I think we just need to win by building a superior product. There is just no other way. And I believe so far we have done that. We have not won against them, but they still have a lot of distribution through Windows devices. So a lot of people just go to Edge and they can start using Bing chat. But people have, despite that, won against Microsoft in the past. like Google, everyone went and searched for Chrome as the first search query on like Internet Explorer to install it.

9:17We all did that despite the friction they added. So there's only one way to win against a person who has much more distribution than you, which is a superior product. Now, about using the same underlying technology, it is the case today. The reason is they have the best models. and there's still a lot of differentiation you can have in how you harness the power of these models. These are so general purpose machines. It's almost like you buy the engine from somewhere, but you're building a whole car with a lot of different parts and you can still build a better car. And if it is the case that OpenAI is going to be the number one place by far and you want to give the best product to your users, you do need to use their model.

10:02So like there is no like you can say, yeah, I'm going to use my own model because I don't want to use someone else's. But then if the search experience is pretty there compared to what you have with OpenAI, you're not going to get users. And then you build your own modes of differentiation and other ways that just the person owning the LLM cannot build as good competitive product as yours. So just the LLM is the only reason this is working. We don't have a chance. But that's not the case here. There are so many other things needed to be done to give you this experience where there's real-time facts being pulled up and presented in the right manner, super fast, reliable, and make the product engaging.

10:41So all that also matters. So, for example, people have done comparisons between us and Bing, and we have much better accuracy in terms of how correctly we cite things. A lot of academic research has been done there. People spend on an average like two minutes more on our site than Bing. So that engagement is much better there. So our bounce rates are much lower than Bing. So basically, we only lack in one thing, which is number of views on the site. But that can only be addressed if we're given sufficient time to grow and make people aware of us. All right. We all know the one thing that separates great startups from the good ones is product velocity.

11:23What does it mean, product velocity? Fancy term, right? You've got your product and you've got velocity. Speed. The speed in which your product improves. So can you ship updates? Can you release new features? Can you do bug fixes? Can you iterate on the interface? Can you solve problems for your customers? And can you do it quickly? Because you're not alone. You have competitors and your customers have choices. They may solve their problems by writing their own custom code, or they might use your solution. This is what startups are about. How fast can you get that product velocity going? And so, you know, how do you supercharge it?

11:58Everybody says, okay, yeah, we want to go faster, but you got to go faster intelligently. And Crowdbotics is going to help you do that. They're your CTO as a service. Basically, they provide you with the most optimal architecture to get your product to market as fast as possible. You'll have access to an on-demand product manager and developer talent, and they will help get your app into production 10 times faster than conventional development. Crowdbotics can work with your in-house dev team, or you can just have them work independently. And you own all the IP. you own all the source code let the folks at crowdbotics supercharge your product velocity today no more waiting get a free build plan at crowdbotics.com slash twist that's a 499 value just for the twist listeners you get that for free that's c-r-o-w-d-b-o-t-i-c-s.com slash twist for a free build plan how do you um get the citations if you were asking this query i just about like, hey, what cities should I take my seven years old to and then what outdoor locations?

12:56How do you actually get the citations because chat GPT for they don't provide citations? Or do they? They have this thing called browser plugin, which is basically powered by Bing. But people hate that experience in the sense it's really slow and clunky. Yeah, it is slow and clunky. Yeah. And so how do we do the citations, We basically pull up the relevant links to your query from a search index. And then we combine that and tell the LLMs to write the answer. We basically ask the LLMs to go read all those links. And then pull up the relevant paragraphs from each of those links. And then make an answer out of whatever you thought was relevant.

13:36But write down the answer as if an academic or a journalist would write it. Where each part of the answer has the corresponding citation. Like Wikipedia. You basically say, hey, like, I want you to do the job of what a human does on Wikipedia, where when they're writing something about a new person or a new phenomenon or a new city, this is basically going and like picking up a lot of web links about that, sifting through them and reading them and then coming back and writing an essay on it. Right. So that whole human labor intelligence needed to do that is being automated now. All of this happening in like seconds.

14:15right that's yeah worth like hours of human labor and that's the value we're actually adding to everybody got it and so you collect all those links give all those articles and then give the the summary of them basically instruct the llm to like hey behave like a wikipedia person just just write it like this so the core of this is prompt engineering and knowing how to prompt engineer um for different types of queries because different queries might require a wikipedia editor or other ones might need more of a sensibility of a journalist. And the LLM knows the difference between those things? You need to make it know.

14:50That's the skill there. And you're right, prompt engineering is a big part of it. But just because somebody might have your prompt doesn't change much, actually. Prompts can leak. So it's all about orchestrating the back end, making it work with the right sources, too. so there you know there's a steve jobs movie with kate winslet in it where there's a scene between wozniak and jobs where wozniak's like i'm the guy writing all the code and i'm the i'm the code you don't write code you don't do design why do you why does everybody know you and not me and he says i play the orchestra so that's basically where anyone who aims to build a long-lasting company on top of lms the thing you need to be really good at is playing the orchestra like having so many things work together reliably and efficiently and correctly and super fast so one of the pieces is searching the webbing finding the right articles the next piece is knowing how to write the answer right what are the other pieces here uh the relevant relevant parts from each article too like ah Article has a ton of content in it.

16:04You only need a few for the query you ask. Making sure that you write the answer in the most accessible way. Initially, we just started off with just putting text with citations. Then people were like, hey, I want neatly formatted answers. I want markdown in it. I want code to be rendered in a specific way. I want images in it. I want videos in it. I might want to customize it according to the domain I'm searching in. And then people keep asking for more and you learn more about it. The second part of Google's mission, right, making it universally accessible and useful. So the first part is organized with this information.

16:40The second part is basically where LLMs are adding tremendous value now. And how do you deal with specific verticals of data that are more siloed? I see one of your co-founders or one of your founding team members was from Quora. You, of course, have the Reddit data set. Great for conversations. You have Twitter great for debates and funny one-liners and breaking news. You have Yelp. You have Google local. You've got all these silos of data. I asked it, Hey, what are some great Greek restaurants? Did a pretty good job of telling me Greek restaurants in the Bay area. And so how do you think about those silos of data?

17:21And are you intercepting searches and saying, Hey, this search is about local businesses and restaurants. The search is about something that the Reddit data set would do better with. How do you think about that? Yeah, so the part about data, like, you know, the access and things like that, it's an ongoing debate. And I don't have, like, you know, very strong opinions on what each person should do. Ideally, if there is a need for us to pay any party for their data access, we'll do it. As for how we do it, like what links we know to use for which query, we do, like, take your query and figure out, like, which category it is and, like, try to use that information to give you the right sources.

18:00It's pretty hard, actually. Google does a tremendous job at this. And we are also doing something called focus searches where in the search bar, instead of using all of the Internet, you can go and pick like academic or you can pick YouTube. You can pick Reddit, Wikipedia. And you can just. Yeah. Yeah. So there is a dropdown called all. Yeah. And I could just pick YouTube and then YouTube. You have access to the corpus of all the transcripts or just the metadata, I guess, and title. For now, we use metadata and titles, but that's already amazing. Sometimes I can't find some videos on YouTube directly, but these LLMs are so good at doing the relevance ranking.

18:40That's much better than the YouTube search algorithm. The language models do better than Google's native search algorithm. Wow. Sometimes. Not always. Got it. Most of the times it's equal, but sometimes it's just really good at these fine-grained... I was trying to find a video of like, oh, so there's a scene in this movie I want to find. for watching for inspiration or something. And then I couldn't find it on YouTube and I come here and I get it. It's very useful for Reddit. Like I want to like learn about like, you know, the nothing phone, like, you know, who's even using it? Who are those million people?

19:12And then I don't have time to go to the subreddit nothing phone and like score over all these like links. It's very useful there. People use it a lot for Wikipedia. Like if they just want to focus on one thing, like I was talking to the founder of Wikipedia, Jimmy Wales, and he literally just asked for this feature. like hey i just want to do search over wikipedia with an lm and i was like that's a great idea yeah so i think i think they're building it now within wikipedia so um interestingly i did a search uh for interviews uh with the ceo of airbnb mine didn't come up but other ones did but then it came up with i did ones from the past year and uh man that was kind of a bingo it kind of nailed it uh which is a kind of a nice feeling um i really think that's a creative idea and i can see how what you're talking about is got some um there is some point to this which is if you narrow the scope or you build some interesting prompt engineering or narrowing uh and thoughtfulness you can get to a better answer so what's going to be your business model here you talked before about how google is not going to make be able to make it work with advertising there's a group of people who believe that uh the chat interface will cannibalize their existing business so do you agree that the this chat gpt style interface or just the chat interface let's leave the gpt out of it um nobody owns a chat interface but is the chat interface anti-advertising or could advertising be integrated into because on all in a lot of i think three out of four besties thought hey advertising is not going to work and i thought i think advertising is going to work great inside of this you have your citations but you could put right in embedded in the discussion you know all kinds of interesting things so if you were asking about places to travel with your kids and i'm disneyland and you didn't make it i could put in there hey and if you're thinking about outdoor stuff disneyland And also has this adventure park and they do the safari.

21:22And I could have like a really AI generated answer at the bottom. So it gives me the correct answer or what it thinks is the correct answer. But then it also gives an ad engines answer to it. So am I right? Or are my other three besties right? You decide. I'm more with you here. Oh, you are. Okay. So firstly, I think relevance can be even more targeted now than ever before. What is the purpose of Google? It's just bringing two parties together, the advertiser and the consumer. And they help you connect these two parties together with their query and link matching, right? At the end of the day, the advertiser wants to get their content to the consumer of the content.

22:09And LLM can give you that needles in the haystack even better. Like it's even more targeted. honestly uh that if i were an advertiser i would just kind of focus on selling myself really well writing even better marketing copies with llms uh catered to the person i'm trying to sell to and we introduced this thing called ai profile on perplexity where you can just write about yourself yes i saw that and and that way you the the results are even more catered to you and then if you're an advertiser you can say i want to like target people who are off like having all these attributes in their profiles um and then uh the ranking will automatically take care of that so in some sense you're you're you're creating way more relevant and targeted ads than ever before i don't know if you use instagram but my experience in instagram is that the ads on instagram are even more relevant than than um often is that that is the case here take a look at this can you see my screen here's the query i did uh based on our little back and forth here will llms will the chat interface be uh accommodating to advertising well i put in here you're the ceo of disney parks pitch me on why i should take my seven-year-olds to one of your parks and so imagine this got appended to my previous search which hey what should i do with my seven-year-olds in a city or outdoors and i says oh thank you for considering one of our parks for seven years here are reasons we believe you'll have an unforgettable experience number one a place where everyone is welcome two more value and flexibility three uh disability access service that's kind of weird uh number four new attractions and experiences that's really good memorable music that's good too actually uh park reservation system that's great uh we hope you'll consider this and then here it could have bookings and would you like to talk to an agent you have further questions and you could just hijack somebody's chat stream uh for your own purposes you they could be thinking they want to go to europe for the summer and then you could sell them on going on a european disney cruise or something and i i think that this kind of um style of advertising where a company ceo starts a discussion with you in chat gpt uh and in a in a chat interface uh is going to be magical yeah and and like you said you know you can give you an answer that's sort of neutral and unbiased and it's not targeted at you.

24:43And I can also say, by the way, in case you are actually looking for something very much to you, and if you already shared that information with us, fully transparent and you're in control, we're not going to do it in a creepy way like Facebook. Then we should be able to give you the answer. We should be able to help the advertiser sell to you even better, right? So I think basically, I'm going even more abstract first principle is thinking that it's not clear how you do it in the product and how you build a business model. But at an abstract level, the point of advertising is to reach the right person to sell to.

25:22And this can help you do that even better than the current system. So therefore, you should be able to figure out something at a level below this. if you're a sass or services company that stores customer data in the cloud then you need to be uh sock to compliant you knew that from a third party and you need that third party to close big deals and if you want to get compliant easier and faster you need to use vanta v-a-n-t-a vanta makes it so easy for you to get and renew your sock two on average van customers are sock two compliant in just two to four weeks prepare that to three to five months without Vanta.

26:00And Vanta can save you hundreds of hours of manual work and up to 85 % of compliance costs. This is a total no brainer. And Vanta does more than just SOC 2 compliance. They also automate up to 90 % compliance for GDPR, HIPAA and more. You can't afford to lose out on major customers. We all know that. Listen, it's a hard year. Last year was hard. You can't lose those major customers because you don't have your compliance dialed in. Just work with Vanta. Get your compliance automated and tight and tight is right. Lock down those big deals. Here's the best part Vant is going to give you$1 ,000 off that's 10 hundies get$1 ,000 off at vanta.com slash twist that's vanta.com slash twist for$1 ,000 off your sock too.

26:41So you've raised some money and you're currently trying to grow the company tell me a little bit about what it's like to try to compete in this area for talent. people are raising you've raised a lot of money but people have raised even more and there's a massive talent battle going on right now is it better to just hire great developers and have them learn because you're not building the fundamental model you're building something on top of it what's your strategy for talent here yeah so we we don't waste time trying to hire people that same old man will be hiring anyway. It's very, it's, you cannot compete.

27:22They have way more cash, way more like, and they can give way less percent of the company because they have way bigger valuation. So what we do is go for these people who are still trying to get into AI, very talented engineers who haven't done AI before and want to be part of an amazing product that's growing. and they want to feel the dopamine from shipping every week and want to see their stuff actually being put out. And there are quite a lot of people who are like that, who haven't done AI before, very talented generalist programmers. That's another thing that I look for, which is, are they generalists?

27:58Can they do back-end, can they do front-end, can they strategize for the product, can they do prompt engineering? Because all these are new skills. Prompt engineering is not like a... You cannot ask for years of experience there. it's like a few month old skill so you just need to be somebody who's like pretty logical and like pretty good at like getting things done you were at OpenAI for a while I was at OpenAI yeah how long were you there and what did you work on generally speaking I worked on like diffusion models and like conversational models for like chatbot, not exactly like chatgpt but more like trying to get another modality into like conversations so that's kind of what I was focusing on but the reason I started this company was because ever since I came to US for grad school in Berkeley I was always interested in starting a company and I was trying to look for people who were like me before who were like PhD students who started a company and I could only find one example from the past that I really resonated with was Larry and Sergey so Larry is my entrepreneurial hero like he he's the only reason I kind of wanted to do a company uh like in fact in a book he's written like he'd either do a be a professor or he would do a company and he would never work for anybody else i had more constraints in my like you know immigration and other stuff like that to have to like sort of work for a bit get some money and like learn more skills but that was sort of always there and it's not planned but it's just more like a coincidence happy coincidence that i'm working on search too um but yeah being at open ai was really helpful.

29:39Back then, there was no chat GPT, so I didn't foresee the future where OpenAI is so successful. But there was GPT 3.5 and it was pretty good. We knew a lot of things were happening. Nobody knew that if you put out these models in the chat UI, the world would go crazy. That was very unknown. So, the fact that people are so used to the modality of chat because they live in it all day long. This was the breakout moment for AI because AI stuff had existed, people were using it in the back ends to serve you up your for your page on Tick Tock or fill in your search query or giving you a couple of words ahead in Gmail and finish your sentence, all that stuff that was happening.

30:23Yeah, but the it needed the interface to make it work. Yeah. Fascinating. The generosity of the models was also amazing. But it was all if you remember opening, I had a playground where you could go and enter a prompt and in green text you'll see the completions but nobody cared about the average person in the world did not care about it and then no you put it into a chat ui and then the world goes crazy right makes you wonder if there's another thing that you could do that make the world go even crazier and i gotta think uh and i'm interested in your thoughts on this were siri and alexa yeah just far too early they had the ability to understand what you were saying yeah they just didn't have the ability to give you the right answer or any answer exactly i mean you could barely call you know you'd be like okay call my mom and it would be like calling mother teresa and you're like no no no no no no it's not what i want and just even getting it to play the right song took three tries now with chat gpt and and all these language models and bard and poe and what you're doing of perplexity it feels like talking to the computer would work and i don't know why this doesn't exist yet it's gonna happen it's gonna happen yeah like i had perplexity as running in the background on my phone in my earpieces yeah and i could just whisper to it and say hey hey perplexity what are some greek restaurants near me uh that have uh lamb and that are over four stars and it just gave me the answer back and started talking to me and i could take out my phone.

31:55Yeah, that would be so magical and just using the the language models as your interface. But using voice and having a talk back to you would be incredible. Yeah, it doesn't exist. In five years, I think what's going to happen is built we'll talk. We'll all wear glasses, we'll talk and then we'll see the answer render in our glasses and then or it can speak back to us and we we can listen via the glasses or whatever. okay why doesn't it exist today like as we speak I think you can stitch together a demo with a speech recognition model and LLM and then a text speech model right yeah the latency wouldn't be enjoyable like it's mostly on the LLM side not even on the speech side you can make these ASR and DTS work pretty fast but if you had to wait for two to three seconds it's a bit like talking to a socially awkward person like they would be like staring at you for like two seconds and then giving you back the answer right yeah yeah so that's the experience you would get you it might not be very enjoyable like how you and i are talking right now i think for that you need even smaller or even faster lms and ah so it's not it wouldn't have the response time that people would find not annoying it would be it would quickly become annoying to have it giving those pauses yeah i find it quite charming now when my chat gpt interface like takes a second or bard is kind of skipping around and it stutters and then it plays and i'm like wait a second and then bard now just gives you the answer straight away boom yeah it doesn't do the typing but i've got i think the open ai apple app uh ios app has like kind of haptics in it yeah where it's like typing i think it's part it's kind of a gimmick right it's not we chose not to do it uh but There is this thing where you stream the output tokens, token by token.

33:49The reason we did that is because you perceive the latency as lower. Like if I waited in my backend to generate the full answer and then display it like in the barred style, you might just be like, oh, what the hell? I don't want to wait. I just bombard you with a huge paragraph. It may not be as fun as like anticipating, like you're reading along with the model generating tokens. that's a different kind of UX. I like OpenAI's choice here, but we didn't do the haptics thing because I found it pretty annoying to use when we were beta testing it and so did the others in our company. So that said, you know, like, here's the thing with TTS, like you have to generate the full answer before feeding it into the text-to-speed system.