#175 Aravind Srinivas: Revolutionizing Search with Perplexity AI

0:00We just built it as a tool that was useful to us because we kind of had no idea of how to build a company, how to build a product. but neither three of us had done anything of this nature. And we initially built a tool like ChatGPT where it'd be like a Slack bot and then we would be able to ask it a bunch of questions and it would just give you the answer. We realized that it would hallucinate a lot. It would just make up stuff and it was not trustworthy. And my co-founder had this idea that, okay, like what if you ground it in real web links? And then we tested early feedback and then the people who used it in the beginning, They were all like, hey, look, I know that you just built it as a cool tool, but I actually find this to be better than Google.

0:43Hi, I'm Craig Smith, and this is Eye on AI. In this episode, I talk with Arvind Srinivas, co-founder and CEO of Perplexity AI. Perplexity is reshaping the search landscape with what it calls its conversational answer engine, providing direct answers grounded in real facts. The company has made a splash by claiming its goal is to supplant Google, but Aravind is realistic about the challenges. He talks about building a reliable search and ranking system to ground Perplexity's large language model in trusted information. I hope you find the conversation as fascinating as I did. I'm Aravind Srinivas.

1:29I'm the co-founder and CEO of Perplexity. Perplexity is a conversational answer engine that directly seeks to give you an answer to any question that you have. And the way it differentiates from existing tools like Google search engines is that you get the answer instead of getting a bunch of links and having to open them and read them yourself. And the way it differentiates itself from other chat tools like ChatGPT that also directly answer your question is that it's grounded in real facts and tells the user exactly where it's pulling its content and the answer from in the form of citations or references.

2:08So it's like if Wikipedia and ChantGPT had a baby and used all of the data from the internet, that product would look exactly like perplexity and that's what we are. Yeah. And how did you come to this? What's your educational and career trajectory? Yeah, so I originally come from India. I was born in India. I lived there for the first 22 years of my life. And I came to the United States in 2017. I had gotten an undergrad in electrical engineering with a lot of focus on computer science and machine learning too. from Indian Institute of Technology, Madras, IIT Madras. And it's one of the top engineering colleges there in India.

2:58And during my undergrad period, I got to intern in many American universities and Canadian universities. One of them was Turing Award winner's lab, Yoshua Bengio. He's one of the deep learning pioneers. And I got really excited by that. And I wanted to do a PhD combining deep learning and reinforcement learning because DeepMind was the hottest lab in town at the time. And they were ruling the roost with all their RL progress. So I really wanted to do more research there. And I got admitted to Berkeley, which is, again, one of the top American universities, particularly for AI. and OpenAI had a strong connection to Berkeley.

3:53They used to recruit a lot from Berkeley. So I got lucky there and got to intern at OpenAI at the end of my first year. And that was when I got to meet, you know, amazing, amazing scientists and researchers there. Folks like Ilya Sutsky were chief scientist and then Alec Radford. and they were building GPT-1 at the time. It wasn't called GPT-1. It was just called GPT. Nobody knew there would be like so many generations of it. But it was clear it was something very different. So I really wanted to study more about it, like try to change my topic in research from RL to generative models, generative AI.

4:44we used to call it unsupervised learning at the time. So I kind of like trying to do more of that is what we were thinking. And I got to do a pretty good PhD. You know, like we published a lot of good papers on transformers. And the highlight of my PhD was working with people at DeepMind during my PhD. And DeepMind and now Google Brain, but now it's all the same. and during one of my internships at DeepMind in London I used to be launching jobs during the day and then I used to be in their library during the evening because as an intern like you just want to stay in the office as much as you can and like learn as much as possible right so I did that and I stumbled upon this book called How Google Works by Eric Schmidt and In the Plex and How Google Works.

5:43These were two different books. And I really loved the entrepreneurship conversion from research. Like, you know, I always was excited about entrepreneurship. I watched movies like Pirates in Silicon Valley and TV shows like Silicon Valley, but never thought I was one of those people because all the examples there, like mark zuckerberg or bill gates or uh steve jobs were all like undergraduate dropouts and i was here already completed my undergrad right like i could only get to the us because i completed my undergrad and uh so i was like okay you know what it's over i'm already i'm only born to work for these people i'm not born to like actually do a company myself um but then i looked at this example of Larry Page and Sergey Brin and they were basically PhD students completed their undergrad and then got an idea and while working on their PhD and made it into a legendary company where I'm actually interning in at that point so that really appealed to me and I went and asked one of my mentors at Google hey like what do you think is the page rank of you know 2019 and you know we would we would we would talk about what is the most important idea and everybody would just say transformers and this was still at a time when there was gpt2 and there were still a lot of skeptics at deep mind but like oh what like it's it's cool but it's not really intelligent it's just like you know autocomplete but again like look how far we've come from there.

7:21So I went and reached out to the inventor of Transformers for an internship. He was a Google brain. Yeah. Who is that that you reach at? Lucas? This is the first author, Ashish Vaswani. Like when you look at the Transformer author list, it's Ashish Vaswani. He's the first guy. He's a core contributor. I got to work with him and learned so much from him that like you know super grateful he's also an investor in perplexity yeah so i i we worked on making transformers like a universal computation system not just for natural language but also for computer vision for rl and i was very inspired by his way of thinking about how like transformers are like you know one universal architecture for everything and um that got me into like either i was deciding between at the end of my internship i was decided my phd was also coming to an end And so I was deciding between either becoming a research scientist or starting a company because I always wanted to start one.

8:21And now that I understood Transformers pretty well, it's time to go and build a product using it. But the conviction was not there. Like Transformers was still like sort of something where you would see research progress, but nothing like a product like GPT, you know, like 3.5 or, you know, DALI. None of these things existed at the time. There was GPT-3. It was a toy. You could see it kind of worked, but most of the times it wouldn't work. So that conviction was not there in anybody to truly go and launch a product. And there was no example of any successful business being built with it either.

9:00So I decided I would just stick with research. I also had a lot of immigration issues, visa issues. So I just stuck with research. Now, I joined OpenAI after my PhD to work more on like, to learn more about another class of models that was also very important in generative AI called diffusion models. So that was what I was spending time on as a researcher. But then I got to see, you know, like companies like Jasper and CopyAI and GitHub Copilot all emerged as very successful products being built on top of GPT 3.5. and making a lot of revenue, like a lot more revenue than even OpenAI was making itself.

9:45And that moment made it very clear to me, okay, look, your entrepreneurship time has arrived. I was already like 27, 28 at the time. And like, I was like, look, I don't have too many more years of high intensity work left in my body like i can work super hard even in my 30s i can work really really hard but i don't have the 100 hour work weeks week after week after week for like five years grind left my my my mind won't work even if my body wants it my mind will get zoned burnt out i'll get zone i'll zone out in meetings i won't be able to digest complex concepts as fast anymore I won't be able to code for many hours together anymore.

10:34So I realized that and felt like this was now or never for me. And also, by mid-30s, you usually kind of build a family, stuff like that, right? Like there are so many other constraints outside of yourself. And so I took the plunge. Okay, the market is there. Products are being built. Technology is improving. costs are going to come down the models are only going to get better so now is the time so i decided to leave and i cold emailed a few investors like elod gill and nat friedman elod gill is legendary silicon valley investor basically every unicorn in the internet and mobile era you could look at it and he would be an investor in it airbnb stripe dropbox open door um i don't know pinterest figma he's an investor in all of these and so like i i emailed him on linkedin and he just responded in like an hour and um and then he converted me into a into like a founder very fast like he's very good at convincing people to leave and naturally i'm just curious what was the linkedin uh message um i don't know i can't remember like the specific wording but it was more like hey a lot i'm working at open ai i work on all these you know diffusion models and stuff and i'm a researcher but i want to actually start a company that's what my heart is in and uh i don't have any idea for any any product or market i have like literally zero clue it's not my world i'm coming from an academic world but um i would love to use your advice on on on figuring it out if you if you can spare half an hour that'll be great and um he replied very instantly you know so that actually sometimes i'm very grateful like why would why do people care about some new person you've never heard about of course i had that track record and all that stuff so that definitely helps but still someone has to take their time and do these things and i i deeply i'm grateful for him for that um and then now friedman do i twitter dm'd him and he just responded very fast and fact that elod was already talking to him definitely helped and And so these two committed to invest in me without any idea.

13:16Like I just had a few demos. And they said, okay, look, we can put a million dollars together. Like 1 to 1.5 million, something of that nature. And friends, past friends also were supportive. And they, together, we pulled together like 2 to 2.5 million and started the company. uh now you can ask like is that a good amount of funding it's it's very little funding honestly think about like compared to other ai companies that don't have anything and just go and raise like tens of hundreds of millions directly when was this uh initially june july we started talking around june july and and the funding came together in september what what year is that 2022 before chat gpt now you can say if if i knew chat gpt was going to happen like it would have been a dumb decision to leave open ai right you never knew like that that's kind of how the world works you you make your own luck and then you can ask like do i regret it would i rather be at open ai than to do my own company i don't regret it at all it's just in fact i feel more um proud of myself that like despite such a massive success um perplexity itself has its own name and like a different value proposition in the in the field in the product space and um you you truly want to build your muscle and prove to yourself you're capable of a bigger challenge by doing your own company the amount of energy and like commitment and it takes to do this is just a lot more than and writing a small component of a large system that gets another larger company a lot of credit.

15:08Yeah, so that's kind of how it all came together. I knew my co-founder Dennis Yaratz from my PhD days in Berkeley. We wrote the same paper, but independently we discovered the same thing and wrote the same paper one or two days apart. And then he brought in another co-founder who i did not know before but he knew so it's johnny ho who was like world number one at competitive programming and they had worked together in this company called quora which was a question answering site so that's kind of how it all came together for us and uh till date like you know i think we all three have very complementary skills which is pretty rare in a founding team usually you try to like start with people you know and then try to scope out roles in our case there was at least a pair of people who knew each other but um it's not like we three had worked together in fact dennis and johnny themselves hadn't talked to each other for a while because they were like not working together for a while but they have like different skills like dennis is very good at prototyping and testing new ideas quickly johnny is very good at actually building systems back end and uh strategizing and like being being very methodical about things i am very good at big picture and vision and like uh actually convincing people, recruiting, fundraising, and positioning the product well, talking to users, those kind of things.

16:49So it was the perfect team, in my opinion, to do a company in the space. And all three of us are very fast iterators. We don't like stasis. We just kind of like one constant iteration and improvement every week. So that helped you kind of feed off each other's energy in the beginning. And that really helped a lot. And yeah, we stuck through. There were so many challenges, so many difficult moments, so many times when I felt like, why am I even doing this? But, you know, it's all worth it to make the planet smarter. Sure. And so you've raised over half a billion at this point. But let me ask you about the product.

17:43So this is Perplexity AI, and you described it as a question answer platform. platform, what I don't understand about perplexity, I understand a little bit about what you're doing in the background. But let me ask you first, who came up with the message that you're a Google competitor, you're taking on Google? So I have never really wanted to do that, to be very honest. It's a pretty bold statement. Yeah, that's one of the reasons I said at the beginning, I want to talk to you about your PR. I mean, where did that messaging come from? Because, yeah, it gets a lot of attention, certainly. I think I would be saying the truth here.

18:35So I guess the truth is that when we built this tool initially, we just built it as a tool that was useful to us because we kind of had no idea of how to build a company, how to build a product. but neither three of us had done anything of this nature. And we initially built a tool like ChatGPT where it'd be like a Slack bot, and then we would be able to ask it a bunch of questions, and it would just give you the answer by opening a new thread in response to the question you ask. And we realized that it would hallucinate a lot. It would just make up stuff, and it was not trustworthy. and my co-founder had this idea that okay like what if you ground it in real web links they said okay we both are academics what do academics do what do journalists do when they write a paper or write a story they only write stuff for which they actually have a reference you're not allowed to say and make up stuff right that's the ethics in journalism and that's the ethics in academia what if we bring that personality into a ai chatbot like a chat gpt that always looks up sources that was the idea and that ended up becoming a very useful bot of its own that we made it a discord bot with a new discord server and then we tested early feedback and then the people who used it in the beginning they were all like hey look i know that you just built it as a cool tool but i actually find this to be better than google the the users said it so when we released the real like hey look guys we've we made this cool new tool that's like a chat gpt that's always accurate because it it uses real links and facts and you get citations that's how we released it but then people were like hey look this is not really a challenge to be competitor because chat gpt is very useful of its own even if it doesn't use real links it's very useful because it it's a brainstorming buddy so i still want to be just using it for for generating content where hallucinations was a feature not a bug whereas your product it's more like hallucinations is a bug you you want it to be as accurate as possible and that's why you're using real links and you're always retrieving and then generating so i feel like you're more of a Google competitor because you are trying to save time.

21:06You're trying to save people time from reading a bunch of web pages and trying to distill the content in those web pages on your own. Instead, the AI does their work for you now. And then there's no need to type in keywords anymore. You just directly get to the point. So you're actually doing a paradigm shift to Google. So that's how we realized that after rolling out the product, we realized that we have something actually much bigger than what we thought and then the further reasoning this is something we never had any idea about in the beginning which is hey it's not just a better product experience over google it actually disrupts google's business model like like like like you can have something better than google on search and get killed right away right if google just rolls out the same thing example let's say you roll out a feature where let's say google hasn't shipped speech to speech experiences and you roll out a feature that literally is google plus voice to voice you can get killed that's fine but if you uh fundamentally say i don't need links anymore i just you know i'm just going to give you the answer directly then that disrupts their display advertising business model because they they basically tied their business model with their front end with their ui the 10 blue links which link gets shown at the top how many people view a link how many people click on a link this is analytics that's fed to an advertiser so and and when i'm telling you hey that ui is going to go away and now you're going to see something else and answer is the prominent part of the ui now that is a very advantageous position to be in because you are playing to their weakness which is the deep tie with this ui on which you're making trillion dollar market cap like if you remember two weeks ago um they went their advertising revenue increased but it was a billion dollar short of wall street expectations which expected them to increase even more and even though they made up for falling short of the expectations by making up revenue in subscription and cloud the stock price went down by six percent now that is the that is their problem that is the that is their problem and that's why we are able to operate this innovator dilemma is playing out and and then we decided okay look we stumbled upon a magical idea everybody wanted to take on google for the last 20 years nobody succeeded because there was no technology that enabled you to change the ui of search and now finally it's getting changed and what's more magical is that technology is in better shape outside google than internally inside google for the first time.

24:13Ever since the last 10 years, like for every single time in the last 10 years, the best AI models always lived inside Google. So even if there was a way to disrupt the Google experience, only Google could do it and therefore they were safe. But because of open AI, because of Meta, because of all these open source AI models, suddenly like the rest of the world has leapfrogged Google and AI and therefore a startup like us can actually boldly try to attempt to bring in this disruption. And so we got very lucky to be in the right place at the right time, right after chat GPT, GPT 3.5, Lama 2, all these things were happening.

24:54And also at the same time, Google was going through bureaucracy and political problems, slow progress. And so we were so lucky to just take advantage of the timing. You're searching a curated set of sources. That's how you ensure preciseness or correctness. Bing already does the same thing. They search. They use Bing search. They list the links just like you guys do. And you set it to be more precise, and it's pretty precise. So you guys hit a moment where you were able to raise money, but how do you survive in competing against Bing, in competing against Gemini, which is most certainly going to do the same thing?

25:50How do you deal with all that? All right. Let me go one by one, right? So the first point about Bing, um but there was a point when i was very scared about ping which was on the day i signed the term sheet for a series a uh literally we had a handshake with our series a investor and we were chilling in a cafe in pala alto because that week was so stressful and we see the words uh leaking screenshots of bing the new version of bing and we are like okay that's it the vc is going to go back on their word and you know there's a 30 day due diligence period where the vcs check the market and like make a decision on whether to wire the money and they're not going to do it and i would not you know i just have to go and like hope microsoft or google buy us off for like a little bit money so that i at least save my face here but that was the extent of fear at that time but bing bought they tried to do a free form chatbot and a search bot all in one one single product they did not explicitly focus on the search use case they rolled out they had advantage of having access to the most cutting model at the time gpt4 whereas everybody else in the world had only gpt 3.5 despite that despite all those advantages all the distribution advantages marketing pr they bottled it because of poor product execution and the fact that bing has always been associated with a bad consumer product the brand of bing has always been associated with something that was like second tier The third reason that they're not taking off is they gated users to install the edge browser or have a Microsoft account to be able to use Bing chat.

27:53So all these things stopped them from taking the first mover advantage and gave the opportunity for people like us to still exist. And outside of that, people view perplexity as a much faster and better done consumer product than bing our mobile apps are much more natively done on swift ui whereas bing apps are built on flutter so the apps are much bigger in terms of memory consumption and size slower uh less well executed their focus is actually very different their focus is not really on delivering the best experience their focus is just getting browser market share it's a very microsoft way of going about things hey, I just want to get more edge users than Chrome.

28:40So what I'm going to do, I'm going to offer people who are edge users a lot of freebies like GPT-4 and then like DALI or like GPT-4 Vision or like PDF summaries on the side, all these kinds of things. That way, I'm hoping that all my AI ownership of OpenAI translates to browser market share, which might translate to advertising revenue increase. so it's a very microsoft way of thinking about it whereas honestly what is the real thing you should do deliver the best possible answer engine the best end-to-end product experience the user now they don't have any incentive to do that because they're a trillion dollar company that's only trying to make a difference in terms of tens of billions of dollars in revenue for us no revenue small startup focus on product nail it get the user base incrementally right so that's why like we did much better than bing and we could and you know to to to add uh fuel to my arguments they even changed bing chat to call it copilot they realized it didn't work and now they're trying to adopt the branding of some word microsoft copilot which is typically associated with an enterprise sas product not doesn't work for a consumer so it didn't really work for them and our bet ended up being right now as for gemini uh i guess what i'll say is like google is making the same mistakes they've been making making for a while and messaging apps by constantly changing and rebranding things and having multiple competing products existing at the same time there is google search generative experience sge that is on google.com itself and arbitrarily fires and doesn't fire based on whether you turn it off or not and um and doesn't fire for any commercial intent queries actually and then um there's google bard which is now rebranding to gemini and there's do it ai that's also trying to do something like that for enterprises so there's all all sorts of like different versions of the same product that are being rolled out and when people think about bard the branding that google has gone for is like your thought partner your buddy or something like that more as a competition for chat gpt trying to do one single chatbot for all purposes freeform or search doesn't matter but they're trying to promote it more for freeform chat brainstorming whereas perplexity is just purely focus on search like i'm not even trying to be a chatbot alternative i'm trying to be the most accurate chatbot ever like most accurate answer bot so when people think about perplexity think about accuracy and fast reliable answers they think about so many things at once so that is a key difference and the other thing i would say is google has no incentive like like honestly look why would if you were google why would you build user base from zero for a new product like that's basically you not making use of all your distribution advantages right if you were if you were google you would just directly roll out to the billions that you've already accumulated over two decades but bard built user base from zero of course they had the benefit of Google marketing and PR, but it's like a very suboptimal move for a big tech company to do that.

32:18And why did they do that? Because they cannot do the other thing. They cannot do the other thing of changing Google.com to look like Bart. Okay, if Google.com looked exactly like Bart today, perplexity is dead. We are done. But Google is also dead. We are building our own search. I mean, we've been building our own search ever since the beginning. We rely on some ranking signals that we can scrape from google search result pages but we're not like overly dependent on them in any way that uh our service wouldn't be able to run without them or something when you fire a query uh the back end doesn't send it to google so google has no api you can scrape search result pages of google there is no official api for google now and if you only scrape search your search result pages of Google, your latency will be very bad.

33:08It'll be very slow. And you have the same problems that Google has in terms of SEO, spam, and ads, and things like that. So the fact that we don't suffer from these problems clearly indicates that we are doing a lot more than just hitting Google API, which of course doesn't exist. But even if you consider hitting Google API, meaning just getting whatever the search result pages offer, clearly we're doing a lot more than that, right? And we have been building our own infrastructure for picking results from our own pages. And if you just hit Google API, you're bottlenecked by what the snippets are given to you from the search result page, which are going to be low quality and less informative and will lead to a lot of hallucinations.

33:50So you have to build your own index. You have to go and crawl the web yourself, and you have to build your own ranking, and And you have to build your own page rank of the web for which domains to use and not use. And we've been doing this ever since the beginning. Now, this is a multi-year journey. I'm not telling you we're done. I'm not giving you any indication that we have something that rivals Google. All I'm telling you is that this is going to take a while. And the only way to win here is to have a product that's already being used by people on a day-to-day basis. and use all that data flywheel to build an amazing ranking and index, which will end up being like technological notes for you.

34:34And so on this crawling the web and indexing sites for use by perplexity, how is that? I mean, that's got to be automated, right? uh how how do you how what what metrics are you using to rank uh authority or or accuracy because that's what it comes down to is having a curated list of of uh of sites i would say that initially we started off doing a lot of manual work for this like i literally wrote wrote down like some domains that we just had to like not use and use and stuff like that but obviously the process is not scalable um and um we have a more sophisticated process that uses our own aggregate data in terms of like you know what domains get linked at the top as citations this you can think about what domains get cited as a different signal for page rank of the web and then the frequency of citations can decide like how often you should you know prioritize using a domain's results versus another domain and then um honestly that's that alone is not enough you're going you have to use a ton of other signals in terms of prioritizing the ranking including the your confidence of like how good the snippet from the domain will be based on your own crawling infrastructure so we are constantly upgrading and you know updating these systems as we speak you know we have a very very very talented search and ranking team that's working all the time here and i believe like whatever solution we'll implement will constantly need to get upgraded because the web keeps changing every day and there are like different ways to um you know like like fall into the trap of poor quality results that's why this company's hard like sometimes i think like why why should this even be a company versus just being a project the difference between a project and the company the project gets done there is a deliverable and it's over this um this process of of indexing plexity when i use it every now and then i'll ask it something and it doesn't have an answer presumably because the the pages in index don't include that.

37:04And then I switch, frankly, to Bing Copilot. When you switch to Bing Copilot, do you get the answer? Usually, yeah. There are other times when it's the reverse, when I have to ask Bing twice to get the answer. It won't find it the first time. But how, I mean, I don't know how you measure it but how how much have you indexed uh and if you know based on how much you want to index are you 10 there are you uh two percent there 50 there i would say we are 70 to 80 there in terms of like really important pages on the web we are we're able to like prioritize the citations like basically they're okay here maybe i should add nuance here i don't want to say we have 70 of the whole webs index captured um what i want to say is like there is a good part of the web a small portion of the web is actually useful now in that portion we have captured 70 to 80 of the value already there is a long tail even there now the web is incredibly long tail like you only want the head of the distribution even within the head of the distribution there is a long tail so that's what we're going to try to capture and if we have captured the basic head of the whole web I feel like it's a game set match for us in the next generation of way people consuming information because nobody wants the whole web anymore and then another question you know my career was with the New York Times and there's this lawsuit with OpenAI.

38:56How do you feel about that? Because if you're indexing and pulling data from sites, you know, personally, I think fair use will win out. But how do you feel about that? because the New York Times argues that you're stealing the, not you, I mean that OpenAI is stealing the revenue from New York Times because the users see the text response, they don't see the links, or if they do see the links, they don't see the advertising on the links. yeah i i would say that uh open ai steals other people's revenue a lot more uh is is uh an argument that you can empathize with because they they don't actually display the sources and they've trained all these models and all the data and uh the leaving aside the training even the end product the chat gpt end product doesn't display the sources right um so then you if it literally reproduces the content that was there in the original publisher's site without actually attributing it to them um i do feel like that's something should be taken seriously so the new york times has some valid arguments there um now in our case we attribute the sources directly so we're not actually like saying this content is from perplexity it's actually just the users that the publisher's content that we are just making sure it reaches more users when they're asking something else so the content publisher should feel good about it and like we can still drive traffic to them and perplexity does drive traffic to others.

41:02Have you signed licensing agreements with any of the sites that you index? We don't have licensing agreements. So we haven't formally had any partnerships with anybody. But we do pay some data providers for API access. And I anticipate doing a lot more of that nature. Did the idea of building a vector database ever occur to you guys as not obviously as the main source, but as an auxiliary knowledge base? A vector database can be constructed as a proprietary vector embeddings that you have. And then the algorithm is called fast retrieval, fast key value retrieval. Now, the second part doesn't need anything proprietary from your end.

41:54And people outside are doing it really well. And we use this thing called Quadrant, which is really fast and very good. But the embeddings that are used for Quadrant are trained by ourselves. And so in some sense, you can see we have built a vector database, the end-to-end vector database. But we rely on the infrastructure that's been done by other people. And Twitter X is also using Quadrant, I believe. Yeah, and the things that the embeddings for you in Quadrant are the URLs of trusted sites or they're the content? Yeah, so the traditional key value database is URLs, and you don't need vector database for that.

42:37Now, the actual content itself is chunked into pieces, and that requires your vector database, vector embeddings. Yeah, that's fascinating. And so you're aware of the looming threats, but you think you have an opportunity to scoot through. You've got plenty of funding. I was interested that Bezos invested because Amazon's invested in Anthropic. And again, it seems as though Anthropic together with Amazon could steamroll you guys. But how do you feel about that? Anthropic is still kind of trying to find its market fit. So I would disagree with your conclusion that Anthropic can steamroll us with Amazon because they don't have any expertise in search technology.

43:36They're only working on a chatbot and Claude, the model that's powering the chatbot. And even leaving a side search, their work so far and building a first party consumer product has been like not that great compared to even players like us. So I'm not really worried about Anthropic. And also Jeff's investment in us has nothing to do with Amazon's investment in Anthropic. And so what's next? You continue indexing. Yeah, it's very simple, actually. A lot of people ask me this question, what is next? What's one of the two key things that will change your destiny? And I give them the very boring answer is we just need to keep growing.

44:20And the best way to grow is to make the product better. And what does it mean to make the product better is to make it faster, more accurate, and make the answers more readable. So faster, more accurate, more readable, more presentable. And if we just deliver on this, then our growth and retention will automatically improve. And that will continue to lead to strong word-of-mouth growth as well as marketing opportunities. And that will further drive home more awareness. and then that'll make all the underlying models and indexes better bigger and more useful and this will feed this will continue to like feed each other the the work we do on making the product better and the work we do on making the index better so they'll feed each other and then if we can create that ever self-fulfilling cycle flywheel um and keep hiring well like i feel like we have a real shot at success so that's my job and some machine keeps running yeah the is is do you have much of an api business is that uh a revenue yeah we we have our own proprietary pplx api that has a first of its kind online llms without any knowledge cut off so it always retrieves from our index and writes the answer.

45:55And I believe that API will see a lot of adoption as we increase the rate limits. Currently, the rate limits are very strict because I would say reduce the rate limits, basically make it possible for more people to submit more requests per second or minute. Because right now, it's severely rate-limited because of our own infrastructure. We don't have that many GPUs. So now that we have more funding, we can actually expand access. And then once we do that, I believe our API business will also flourish. And I think that's very important so that it doesn't put us in a pressure to figure out advertising revenue soon.

46:42Imagine if Google had Google Cloud figured out before Amazon figured out AWS. they wouldn't have needed to go this deep into advertising. So considering the hindsight of how the world has played out, I think we have the luxury of trying many different business models at once. Yeah.

47:08What kind of companies are using the API at this point? I do not have enterprise contracts signed with big companies yet. I would say we have like around 10 ,000 developers that are building with our APIs spread across so many different companies. And I think we'll have to mature more as a company to like do large enterprise contracts with like established companies. Yeah. You know, at the beginning you were saying, and I've read that you're building on GBT 3.5. now you have gpt4 in your in your paid version i guess uh but and then you have uh llama 2 or or of you fine-tuned the model off of llama what is llama what role does llama does the llama 2 model play yeah so we we build on top of two open source models llama 2 and then mistraw mistral and mixed It's a new company called Mistral that's published a bunch of cool models.

48:12And the role that these models play is give startups like us a chance to build on a stack that has no big company's influence. So, for example, OpenAI, we are competing with them on, you know, consumer product like ChatGPT and Perplexity are both trying to, you know, compete for the same user subscription revenue. so having alternatives to model access outside open ai is very important to us so that that's the role being played by anthropics claw or this is a closed source model or open source models like llama and mistraw the the additional advantage of open source obviously is the fact that you have access to debates and so you have the flexibility to change these models even more to suit your product and further differentiate your product so that it doesn't exactly respond in the same style and idiosyncrasies of gpt 3.5 and 4 doesn't moralize people has a lot less restricted and therefore can like you know attract a different user base or the same user base even that might want different kind of behavior.

49:30So it's very good. It's very good for the world that Meta and Mistral are focused on open sourcing models. Yeah, yeah, no, I agree. And so these open source models, Lama and Mistral, are playing the same role as GPT 3.5. It's just do you split a query when it comes in and send some of it to the open source and some of it to GPT 3.5 and then put them back together when you get the response or do you, depending on rate levels or traffic, do you... Yeah, yeah. Our router infrastructure is very sophisticated today that I don't even know what we are currently doing. It's something that keeps intelligently changing based on the throughput and the latency and whether some existing API endpoint is down or up.